2021-01-06T21:45:37.917 INFO:root:teuthology version: 1.0.0-77817aa4 2021-01-06T21:45:37.919 DEBUG:teuthology.run:Teuthology command: teuthology --block --name pdonnell-2021-01-06_20:20:52-fs-wip-pdonnell-testing-20210105.221014-distro-basic-smithi --verbose --archive /home/teuthworker/archive/pdonnell-2021-01-06_20:20:52-fs-wip-pdonnell-testing-20210105.221014-distro-basic-smithi/5760088 --lock --owner scheduled_pdonnell@teuthology --description fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} -- /tmp/teuthology-worker.7cgq34yb.tmp 2021-01-06T21:45:37.970 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-01-06T21:45:38.041 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2021-01-06_20:20:52-fs-wip-pdonnell-testing-20210105.221014-distro-basic-smithi/5760088 branch: wip-pdonnell-testing-20210105.221014 description: fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} email: pdonnell@redhat.com first_in_suite: false job_id: '5760088' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2021-01-06_20:20:52-fs-wip-pdonnell-testing-20210105.221014-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: wip-pdonnell-testing-20210105.221014 ceph: cephfs: max_mds: 3 session_timeout: 300 conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 15m rados osd op timeout: 15m global: mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: osd mds ms inject socket failures: 2500 mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - SLOW_OPS - slow request log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: d0ed162b51928c50f20cee111f8292828eda755e ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: sha1: d0ed162b51928c50f20cee111f8292828eda755e thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20210105.221014 sha1: d0ed162b51928c50f20cee111f8292828eda755e owner: scheduled_pdonnell@teuthology priority: 69 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 sha1: d0ed162b51928c50f20cee111f8292828eda755e sleep_before_teardown: 0 suite: fs suite_branch: wip-pdonnell-testing-20210105.221014 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20210105.221014/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: d0ed162b51928c50f20cee111f8292828eda755e tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - mds_thrash: null - workunit: clients: all: - suites/ffsb.sh teuthology_branch: master timestamp: 2021-01-06_20:20:52 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.15477 2021-01-06T21:45:38.042 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20210105.221014/qa; will attempt to use it 2021-01-06T21:45:38.043 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20210105.221014/qa/tasks 2021-01-06T21:45:38.043 INFO:teuthology.run_tasks:Running task internal.check_packages... 2021-01-06T21:45:38.045 INFO:teuthology.task.internal:Checking packages... 2021-01-06T21:45:38.068 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash 'd0ed162b51928c50f20cee111f8292828eda755e' 2021-01-06T21:45:38.068 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-01-06T21:45:38.069 INFO:teuthology.packaging:ref: None 2021-01-06T21:45:38.069 INFO:teuthology.packaging:tag: None 2021-01-06T21:45:38.069 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20210105.221014 2021-01-06T21:45:38.070 INFO:teuthology.packaging:sha1: d0ed162b51928c50f20cee111f8292828eda755e 2021-01-06T21:45:38.070 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=wip-pdonnell-testing-20210105.221014 2021-01-06T21:45:38.247 INFO:teuthology.task.internal:Found packages for ceph version 16.0.0-8719-gd0ed162b-1bionic 2021-01-06T21:45:38.248 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2021-01-06T21:45:38.256 INFO:teuthology.task.internal:no buildpackages task found 2021-01-06T21:45:38.256 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2021-01-06T21:45:38.456 INFO:teuthology.task.internal.lock_machines:Locking machines... 2021-01-06T21:45:38.457 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-01-06T21:45:38.513 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-06T21:45:48.531 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-06T21:45:58.542 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-06T21:46:08.557 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-06T21:46:18.575 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-06T21:46:28.591 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-06T21:46:38.607 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-06T21:46:48.623 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-06T21:46:58.639 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-06T21:47:08.654 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-06T21:47:18.671 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-06T21:47:28.688 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2021-01-06T21:47:38.704 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2021-01-06T21:47:48.720 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2021-01-06T21:47:58.738 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_pdonnell@teuthology', 'count': 2, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/pdonnell-2021-01-06_20:20:52-fs-wip-pdonnell-testing-20210105.221014-distro-basic-smithi/5760088'} 2021-01-06T21:47:58.758 DEBUG:teuthology.lock.ops:locked ubuntu@smithi203.front.sepia.ceph.com, ubuntu@smithi091.front.sepia.ceph.com 2021-01-06T21:47:58.807 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f20e008cbf8>, signals=[15]) 2021-01-06T21:47:59.009 INFO:teuthology.provision.fog.smithi203:Scheduling deploy of ubuntu 18.04 2021-01-06T21:47:59.012 INFO:teuthology.provision.fog.smithi091:Scheduling deploy of ubuntu 18.04 2021-01-06T21:47:59.456 INFO:teuthology.orchestra.console:Power off smithi091 2021-01-06T21:47:59.457 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2021-01-06T21:47:59.477 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-06T21:47:59.491 INFO:teuthology.orchestra.console:Power off smithi203 2021-01-06T21:47:59.491 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2021-01-06T21:47:59.511 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-06T21:47:59.600 INFO:teuthology.orchestra.console:Power off for smithi091 completed 2021-01-06T21:47:59.701 INFO:teuthology.orchestra.console:Power on smithi091 2021-01-06T21:47:59.701 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2021-01-06T21:47:59.724 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-06T21:48:03.535 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-06T21:48:03.745 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-06T21:48:03.959 INFO:teuthology.orchestra.console:Power on for smithi091 completed 2021-01-06T21:48:04.061 INFO:teuthology.provision.fog.smithi091:Waiting for deploy to finish 2021-01-06T21:48:07.646 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-06T21:48:07.855 INFO:teuthology.orchestra.console:Power off for smithi203 completed 2021-01-06T21:48:07.957 INFO:teuthology.orchestra.console:Power on smithi203 2021-01-06T21:48:07.957 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2021-01-06T21:48:07.978 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-06T21:48:12.001 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-06T21:48:12.211 INFO:teuthology.orchestra.console:Power on for smithi203 completed 2021-01-06T21:48:12.312 INFO:teuthology.provision.fog.smithi203:Waiting for deploy to finish 2021-01-06T21:51:06.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:51:15.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:51:24.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:51:28.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:51:33.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:51:37.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:51:42.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:51:47.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:51:51.476 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:51:56.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:52:00.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:52:05.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:52:14.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:52:23.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:53:06.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:53:06.664 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-01-06T21:53:06.666 DEBUG:teuthology.orchestra.run.smithi091:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2021-01-06T21:53:07.031 INFO:teuthology.provision.fog.smithi091:Node is ready 2021-01-06T21:53:07.032 DEBUG:teuthology.orchestra.run.smithi091:> hostname 2021-01-06T21:53:07.080 INFO:teuthology.orchestra.run.smithi091.stdout:smithi091.front.sepia.ceph.com 2021-01-06T21:53:07.081 DEBUG:teuthology.orchestra.run.smithi091:> grep smithi091.front.sepia.ceph.com /etc/hosts 2021-01-06T21:53:07.129 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.15.91 smithi091.front.sepia.ceph.com smithi091 2021-01-06T21:53:07.129 DEBUG:teuthology.orchestra.run.smithi091:> sudo hostname smithi091 2021-01-06T21:53:07.254 DEBUG:teuthology.orchestra.run.smithi091:> sudo sed -i -e 's/smithi091.front.sepia.ceph.com/smithi091/g' /etc/hosts 2021-01-06T21:53:07.271 DEBUG:teuthology.orchestra.run.smithi091:> sudo sed -i -e 's/172.21.15.91/172.21.15.91/g' /etc/hosts 2021-01-06T21:53:07.328 DEBUG:teuthology.orchestra.run.smithi091:> sudo sed -i -e 's/smithi091.front.sepia.ceph.com/smithi091/g' /etc/hostname 2021-01-06T21:53:07.387 DEBUG:teuthology.orchestra.run.smithi091:> sudo hostname smithi091 2021-01-06T21:53:07.446 INFO:teuthology.provision.fog.smithi091:Deploy complete! 2021-01-06T21:53:07.544 DEBUG:teuthology.parallel:result is None 2021-01-06T21:53:29.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:53:29.385 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-01-06T21:53:29.386 DEBUG:teuthology.orchestra.run.smithi203:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2021-01-06T21:53:29.715 INFO:teuthology.provision.fog.smithi203:Node is ready 2021-01-06T21:53:29.715 DEBUG:teuthology.orchestra.run.smithi203:> hostname 2021-01-06T21:53:29.763 INFO:teuthology.orchestra.run.smithi203.stdout:smithi203.front.sepia.ceph.com 2021-01-06T21:53:30.676 DEBUG:teuthology.orchestra.run.smithi203:> grep smithi203.front.sepia.ceph.com /etc/hosts 2021-01-06T21:53:30.683 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.15.203 smithi203.front.sepia.ceph.com smithi203 2021-01-06T21:53:30.684 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname smithi203 2021-01-06T21:53:30.742 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i -e 's/smithi203.front.sepia.ceph.com/smithi203/g' /etc/hosts 2021-01-06T21:53:30.800 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i -e 's/172.21.15.203/172.21.15.203/g' /etc/hosts 2021-01-06T21:53:30.857 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i -e 's/smithi203.front.sepia.ceph.com/smithi203/g' /etc/hostname 2021-01-06T21:53:30.918 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname smithi203 2021-01-06T21:53:31.327 INFO:teuthology.provision.fog.smithi203:Deploy complete! 2021-01-06T21:53:31.327 DEBUG:teuthology.parallel:result is None 2021-01-06T21:53:31.457 INFO:teuthology.lock.ops:Checking smithi203.front.sepia.ceph.com 2021-01-06T21:53:31.458 INFO:teuthology.lock.ops:New key found. Updating... 2021-01-06T21:53:31.474 INFO:teuthology.lock.ops:Checking smithi091.front.sepia.ceph.com 2021-01-06T21:53:31.474 INFO:teuthology.lock.ops:New key found. Updating... 2021-01-06T21:53:31.489 INFO:teuthology.lock.ops:Updating [smithi203.front.sepia.ceph.com]: set os type and version on server 2021-01-06T21:53:31.489 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-06T21:53:31.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:53:31.566 DEBUG:teuthology.orchestra.run.smithi203:> true 2021-01-06T21:53:31.759 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2021-01-06T21:53:31.806 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2021-01-06T21:53:31.807 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2021-01-06T21:53:31.856 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="Ubuntu" 2021-01-06T21:53:31.856 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-01-06T21:53:31.856 INFO:teuthology.orchestra.run.smithi203.stdout:ID=ubuntu 2021-01-06T21:53:31.857 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE=debian 2021-01-06T21:53:31.857 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-01-06T21:53:31.857 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="18.04" 2021-01-06T21:53:31.857 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://www.ubuntu.com/" 2021-01-06T21:53:31.858 INFO:teuthology.orchestra.run.smithi203.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-01-06T21:53:31.858 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-01-06T21:53:31.858 INFO:teuthology.orchestra.run.smithi203.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-01-06T21:53:31.858 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_CODENAME=bionic 2021-01-06T21:53:31.859 INFO:teuthology.orchestra.run.smithi203.stdout:UBUNTU_CODENAME=bionic 2021-01-06T21:53:31.860 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2021-01-06T21:53:31.874 INFO:teuthology.lock.ops:Updating [smithi091.front.sepia.ceph.com]: set os type and version on server 2021-01-06T21:53:31.874 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-06T21:53:31.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:53:31.950 DEBUG:teuthology.orchestra.run.smithi091:> true 2021-01-06T21:53:32.197 DEBUG:teuthology.orchestra.run.smithi091:> uname -m 2021-01-06T21:53:32.253 INFO:teuthology.orchestra.run.smithi091.stdout:x86_64 2021-01-06T21:53:32.254 DEBUG:teuthology.orchestra.run.smithi091:> cat /etc/os-release 2021-01-06T21:53:32.307 INFO:teuthology.orchestra.run.smithi091.stdout:NAME="Ubuntu" 2021-01-06T21:53:32.308 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-01-06T21:53:32.309 INFO:teuthology.orchestra.run.smithi091.stdout:ID=ubuntu 2021-01-06T21:53:32.309 INFO:teuthology.orchestra.run.smithi091.stdout:ID_LIKE=debian 2021-01-06T21:53:32.310 INFO:teuthology.orchestra.run.smithi091.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-01-06T21:53:32.310 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION_ID="18.04" 2021-01-06T21:53:32.311 INFO:teuthology.orchestra.run.smithi091.stdout:HOME_URL="https://www.ubuntu.com/" 2021-01-06T21:53:32.311 INFO:teuthology.orchestra.run.smithi091.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-01-06T21:53:32.312 INFO:teuthology.orchestra.run.smithi091.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-01-06T21:53:32.312 INFO:teuthology.orchestra.run.smithi091.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-01-06T21:53:32.313 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION_CODENAME=bionic 2021-01-06T21:53:32.313 INFO:teuthology.orchestra.run.smithi091.stdout:UBUNTU_CODENAME=bionic 2021-01-06T21:53:32.315 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server 2021-01-06T21:53:32.333 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2021-01-06T21:53:32.391 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi091.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxD2AnNqm9fQ9ntMAHZi1hz7TkCqFG+G2H17VCIENzH2xNWbd+PIj9VLD01/nlr0QeUu7U77/ROiQkPQZjIt4QbBtEb8Y989DzZz5Lo4smSz4IDy+ud54pvq39hQPkDUaTGe834I87W8FVlOzpdEaNKWy0M6vP/UZXeI+2C8OKd+E05ocr7ycUJK094JTAhiL50/7okOhQtUz1Dha+/apxYfKleVBb6A6j1qc+gO61IeVIRuWMZJ809vOmgaC/puV86opCpqmzV680aQC3KAMhuz8M7EsKpKlA7eqlTRZCVRbFpkgbfsI4D1ml9jjIS+7SehwL4NcKDuDAv/ogDm0f smithi203.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYX2aW5kSZE4K8H3f9mq0QhiOgFdcVZIPZTH/OdlVTmb/0CxzDjZk+mT1Cv8kBWgASGhIWzXz9DjE3qA5yVw06s3OdhwHfr6mCAIYHaT3suYc6wDjhn2n+t7vH6fzriQVnSrtlb8V42iRkLCRXk9nU2CwsoweRATMDsyo9uBCJ5Lhzi8r39asEJwib6EkVf29udO+FBNns0Xse5oesTZdSA+8miMv9oQ1/g8fF+hZciz8csOFvQlDbK1nx8Z0vpqR/PiRsUEglXBL4QWDNTiqRBnjSeMYMBWfo3QPvR1Bzjkx8n8f4ckNOYh6itU+jVhFOUD6+wm0gKxE5JiunGK4V 2021-01-06T21:53:32.391 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-01-06T21:53:32.452 INFO:teuthology.run_tasks:Running task internal.save_config... 2021-01-06T21:53:32.473 INFO:teuthology.task.internal:Saving configuration 2021-01-06T21:53:32.524 INFO:teuthology.run_tasks:Running task internal.check_lock... 2021-01-06T21:53:32.557 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-01-06T21:53:32.579 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-01-06 21:47:58.744736', 'description': '/home/teuthworker/archive/pdonnell-2021-01-06_20:20:52-fs-wip-pdonnell-testing-20210105.221014-distro-basic-smithi/5760088', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIItfF3pSYVF/Ydt3jJbGAEK9HJh2giDhh26Ba5BBtF7c', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi203.front.sepia.ceph.com'} 2021-01-06T21:53:32.606 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-01-06 21:47:58.744755', 'description': '/home/teuthworker/archive/pdonnell-2021-01-06_20:20:52-fs-wip-pdonnell-testing-20210105.221014-distro-basic-smithi/5760088', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPvPcU+Pv4cU65f3JMG+Hh9amPpz0crYr3YvJo5hCRJj', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi091.front.sepia.ceph.com'} 2021-01-06T21:53:32.607 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2021-01-06T21:53:33.226 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2021-01-06T21:53:33.227 INFO:teuthology.task.internal:roles: ubuntu@smithi091.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2021-01-06T21:53:33.227 INFO:teuthology.run_tasks:Running task console_log... 2021-01-06T21:53:33.540 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f20e01441e0>, signals=[15]) 2021-01-06T21:53:33.541 INFO:teuthology.run_tasks:Running task internal.connect... 2021-01-06T21:53:33.922 INFO:teuthology.task.internal:Opening connections... 2021-01-06T21:53:33.928 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2021-01-06T21:53:33.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:53:34.078 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi091.front.sepia.ceph.com 2021-01-06T21:53:34.080 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:53:34.209 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2021-01-06T21:53:34.480 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2021-01-06T21:53:34.487 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2021-01-06T21:53:34.487 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2021-01-06T21:53:34.535 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="Ubuntu" 2021-01-06T21:53:34.535 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-01-06T21:53:34.535 INFO:teuthology.orchestra.run.smithi203.stdout:ID=ubuntu 2021-01-06T21:53:34.536 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE=debian 2021-01-06T21:53:34.536 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-01-06T21:53:34.536 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="18.04" 2021-01-06T21:53:34.536 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://www.ubuntu.com/" 2021-01-06T21:53:34.537 INFO:teuthology.orchestra.run.smithi203.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-01-06T21:53:34.537 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-01-06T21:53:34.537 INFO:teuthology.orchestra.run.smithi203.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-01-06T21:53:34.537 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_CODENAME=bionic 2021-01-06T21:53:34.537 INFO:teuthology.orchestra.run.smithi203.stdout:UBUNTU_CODENAME=bionic 2021-01-06T21:53:34.538 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2021-01-06T21:53:34.553 DEBUG:teuthology.orchestra.run.smithi091:> uname -m 2021-01-06T21:53:34.559 INFO:teuthology.orchestra.run.smithi091.stdout:x86_64 2021-01-06T21:53:34.560 DEBUG:teuthology.orchestra.run.smithi091:> cat /etc/os-release 2021-01-06T21:53:34.608 INFO:teuthology.orchestra.run.smithi091.stdout:NAME="Ubuntu" 2021-01-06T21:53:34.608 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-01-06T21:53:34.608 INFO:teuthology.orchestra.run.smithi091.stdout:ID=ubuntu 2021-01-06T21:53:34.608 INFO:teuthology.orchestra.run.smithi091.stdout:ID_LIKE=debian 2021-01-06T21:53:34.609 INFO:teuthology.orchestra.run.smithi091.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-01-06T21:53:34.609 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION_ID="18.04" 2021-01-06T21:53:34.609 INFO:teuthology.orchestra.run.smithi091.stdout:HOME_URL="https://www.ubuntu.com/" 2021-01-06T21:53:34.609 INFO:teuthology.orchestra.run.smithi091.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-01-06T21:53:34.609 INFO:teuthology.orchestra.run.smithi091.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-01-06T21:53:34.610 INFO:teuthology.orchestra.run.smithi091.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-01-06T21:53:34.610 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION_CODENAME=bionic 2021-01-06T21:53:34.610 INFO:teuthology.orchestra.run.smithi091.stdout:UBUNTU_CODENAME=bionic 2021-01-06T21:53:34.610 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server 2021-01-06T21:53:34.623 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2021-01-06T21:53:36.297 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2021-01-06T21:53:36.305 INFO:teuthology.task.internal:Checking for old test directory... 2021-01-06T21:53:36.306 DEBUG:teuthology.orchestra.run.smithi091:> test '!' -e /home/ubuntu/cephtest 2021-01-06T21:53:36.309 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2021-01-06T21:53:36.314 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2021-01-06T21:53:36.467 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2021-01-06T21:53:36.468 DEBUG:teuthology.orchestra.run.smithi091:> test -z $(ls -A /var/lib/ceph) 2021-01-06T21:53:36.473 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2021-01-06T21:53:36.487 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2021-01-06T21:53:37.578 INFO:teuthology.run_tasks:Running task kernel... 2021-01-06T21:53:37.593 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'mds.a': {'kdb': True, 'sha1': 'distro'}, 'mds.c': {'kdb': True, 'sha1': 'distro'}, 'mds.e': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'mds.b': {'kdb': True, 'sha1': 'distro'}, 'mds.d': {'kdb': True, 'sha1': 'distro'}, 'mds.f': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}} 2021-01-06T21:53:37.593 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2021-01-06T21:53:37.594 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2021-01-06T21:53:37.600 INFO:teuthology.orchestra.run.smithi203.stdout:4.15.0-128-generic 2021-01-06T21:53:37.601 INFO:teuthology.task.kernel:Running kernel on smithi203: 4.15.0-128-generic 2021-01-06T21:53:37.601 DEBUG:teuthology.orchestra.run.smithi203:> sudo apt-get clean 2021-01-06T21:53:37.706 DEBUG:teuthology.orchestra.run.smithi203:> sudo apt-get update 2021-01-06T21:53:37.954 INFO:teuthology.orchestra.run.smithi203.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-01-06T21:53:37.974 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-01-06T21:53:38.036 INFO:teuthology.orchestra.run.smithi203.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-01-06T21:53:38.296 INFO:teuthology.orchestra.run.smithi203.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-01-06T21:53:38.365 INFO:teuthology.orchestra.run.smithi203.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,478 kB] 2021-01-06T21:53:38.499 INFO:teuthology.orchestra.run.smithi203.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,181 kB] 2021-01-06T21:53:38.838 INFO:teuthology.orchestra.run.smithi203.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [880 kB] 2021-01-06T21:53:38.839 INFO:teuthology.orchestra.run.smithi203.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,814 kB] 2021-01-06T21:53:38.874 INFO:teuthology.orchestra.run.smithi203.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [288 kB] 2021-01-06T21:53:38.886 INFO:teuthology.orchestra.run.smithi203.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [192 kB] 2021-01-06T21:53:38.894 INFO:teuthology.orchestra.run.smithi203.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [25.5 kB] 2021-01-06T21:53:38.895 INFO:teuthology.orchestra.run.smithi203.stdout:Get:12 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [969 kB] 2021-01-06T21:53:38.943 INFO:teuthology.orchestra.run.smithi203.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [379 kB] 2021-01-06T21:53:38.954 INFO:teuthology.orchestra.run.smithi203.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,097 kB] 2021-01-06T21:53:38.959 INFO:teuthology.orchestra.run.smithi203.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [17.4 kB] 2021-01-06T21:53:38.960 INFO:teuthology.orchestra.run.smithi203.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [214 kB] 2021-01-06T21:53:38.969 INFO:teuthology.orchestra.run.smithi203.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [29.1 kB] 2021-01-06T21:53:38.970 INFO:teuthology.orchestra.run.smithi203.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,700 kB] 2021-01-06T21:53:39.009 INFO:teuthology.orchestra.run.smithi203.stdout:Get:19 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [245 kB] 2021-01-06T21:53:39.014 INFO:teuthology.orchestra.run.smithi203.stdout:Get:20 http://security.ubuntu.com/ubuntu bionic-security/multiverse i386 Packages [4,400 B] 2021-01-06T21:53:39.014 INFO:teuthology.orchestra.run.smithi203.stdout:Get:21 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [12.5 kB] 2021-01-06T21:53:39.015 INFO:teuthology.orchestra.run.smithi203.stdout:Get:22 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [2,644 B] 2021-01-06T21:53:39.016 INFO:teuthology.orchestra.run.smithi203.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,549 kB] 2021-01-06T21:53:39.069 INFO:teuthology.orchestra.run.smithi203.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [358 kB] 2021-01-06T21:53:39.078 INFO:teuthology.orchestra.run.smithi203.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [14.0 kB] 2021-01-06T21:53:39.079 INFO:teuthology.orchestra.run.smithi203.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [35.4 kB] 2021-01-06T21:53:39.080 INFO:teuthology.orchestra.run.smithi203.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,980 B] 2021-01-06T21:53:40.461 INFO:teuthology.orchestra.run.smithi203.stdout:Fetched 12.7 MB in 3s (4,959 kB/s) 2021-01-06T21:53:41.944 INFO:teuthology.orchestra.run.smithi203.stdout:Reading package lists... 2021-01-06T21:53:41.993 DEBUG:teuthology.orchestra.run.smithi203:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-01-06T21:53:42.072 INFO:teuthology.orchestra.run.smithi203.stdout:Reading package lists... 2021-01-06T21:53:42.302 INFO:teuthology.orchestra.run.smithi203.stdout:Building dependency tree... 2021-01-06T21:53:42.303 INFO:teuthology.orchestra.run.smithi203.stdout:Reading state information... 2021-01-06T21:53:42.382 INFO:teuthology.orchestra.run.smithi203.stderr:E: Unable to locate package linux-image-current-generic 2021-01-06T21:53:42.383 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-01-06T21:53:42.383 DEBUG:teuthology.orchestra.run.smithi203:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-01-06T21:53:42.465 INFO:teuthology.orchestra.run.smithi203.stdout:Reading package lists... 2021-01-06T21:53:42.703 INFO:teuthology.orchestra.run.smithi203.stdout:Building dependency tree... 2021-01-06T21:53:42.704 INFO:teuthology.orchestra.run.smithi203.stdout:Reading state information... 2021-01-06T21:53:42.875 INFO:teuthology.orchestra.run.smithi203.stdout:The following additional packages will be installed: 2021-01-06T21:53:42.875 INFO:teuthology.orchestra.run.smithi203.stdout: linux-generic linux-headers-4.15.0-129 linux-headers-4.15.0-129-generic 2021-01-06T21:53:42.876 INFO:teuthology.orchestra.run.smithi203.stdout: linux-headers-generic linux-image-4.15.0-129-generic 2021-01-06T21:53:42.876 INFO:teuthology.orchestra.run.smithi203.stdout: linux-modules-4.15.0-129-generic linux-modules-extra-4.15.0-129-generic 2021-01-06T21:53:42.877 INFO:teuthology.orchestra.run.smithi203.stdout:Suggested packages: 2021-01-06T21:53:42.878 INFO:teuthology.orchestra.run.smithi203.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2021-01-06T21:53:42.878 INFO:teuthology.orchestra.run.smithi203.stdout:Recommended packages: 2021-01-06T21:53:42.878 INFO:teuthology.orchestra.run.smithi203.stdout: thermald 2021-01-06T21:53:42.911 INFO:teuthology.orchestra.run.smithi203.stdout:The following NEW packages will be installed: 2021-01-06T21:53:42.912 INFO:teuthology.orchestra.run.smithi203.stdout: linux-headers-4.15.0-129 linux-headers-4.15.0-129-generic 2021-01-06T21:53:42.912 INFO:teuthology.orchestra.run.smithi203.stdout: linux-image-4.15.0-129-generic linux-modules-4.15.0-129-generic 2021-01-06T21:53:42.913 INFO:teuthology.orchestra.run.smithi203.stdout: linux-modules-extra-4.15.0-129-generic 2021-01-06T21:53:42.913 INFO:teuthology.orchestra.run.smithi203.stdout:The following packages will be upgraded: 2021-01-06T21:53:42.914 INFO:teuthology.orchestra.run.smithi203.stdout: linux-generic linux-headers-generic linux-image-generic 2021-01-06T21:53:43.094 INFO:teuthology.orchestra.run.smithi203.stdout:3 upgraded, 5 newly installed, 0 to remove and 9 not upgraded. 2021-01-06T21:53:43.095 INFO:teuthology.orchestra.run.smithi203.stdout:Need to get 66.0 MB of archives. 2021-01-06T21:53:43.095 INFO:teuthology.orchestra.run.smithi203.stdout:After this operation, 334 MB of additional disk space will be used. 2021-01-06T21:53:43.096 INFO:teuthology.orchestra.run.smithi203.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-129-generic amd64 4.15.0-129.132 [13.1 MB] 2021-01-06T21:53:44.104 INFO:teuthology.orchestra.run.smithi203.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-129-generic amd64 4.15.0-129.132 [8,016 kB] 2021-01-06T21:53:44.310 INFO:teuthology.orchestra.run.smithi203.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-129-generic amd64 4.15.0-129.132 [32.8 MB] 2021-01-06T21:53:45.312 INFO:teuthology.orchestra.run.smithi203.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.129.116 [1,872 B] 2021-01-06T21:53:45.312 INFO:teuthology.orchestra.run.smithi203.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.129.116 [2,580 B] 2021-01-06T21:53:45.314 INFO:teuthology.orchestra.run.smithi203.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-129 all 4.15.0-129.132 [10.9 MB] 2021-01-06T21:53:45.614 INFO:teuthology.orchestra.run.smithi203.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-129-generic amd64 4.15.0-129.132 [1,106 kB] 2021-01-06T21:53:45.642 INFO:teuthology.orchestra.run.smithi203.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.129.116 [2,480 B] 2021-01-06T21:53:46.134 INFO:teuthology.orchestra.run.smithi203.stdout:Fetched 66.0 MB in 3s (24.3 MB/s) 2021-01-06T21:53:46.267 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package linux-modules-4.15.0-129-generic. 2021-01-06T21:53:46.309 INFO:teuthology.orchestra.run.smithi203.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102079 files and directories currently installed.) 2021-01-06T21:53:46.310 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../0-linux-modules-4.15.0-129-generic_4.15.0-129.132_amd64.deb ... 2021-01-06T21:53:46.365 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking linux-modules-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:53:48.006 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package linux-image-4.15.0-129-generic. 2021-01-06T21:53:48.018 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../1-linux-image-4.15.0-129-generic_4.15.0-129.132_amd64.deb ... 2021-01-06T21:53:48.119 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking linux-image-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:53:48.627 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-129-generic. 2021-01-06T21:53:48.640 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-129-generic_4.15.0-129.132_amd64.deb ... 2021-01-06T21:53:48.677 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking linux-modules-extra-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:53:52.321 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../3-linux-generic_4.15.0.129.116_amd64.deb ... 2021-01-06T21:53:52.448 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking linux-generic (4.15.0.129.116) over (4.15.0.128.115) ... 2021-01-06T21:53:52.799 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.129.116_amd64.deb ... 2021-01-06T21:53:52.921 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking linux-image-generic (4.15.0.129.116) over (4.15.0.128.115) ... 2021-01-06T21:53:53.258 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package linux-headers-4.15.0-129. 2021-01-06T21:53:53.270 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../5-linux-headers-4.15.0-129_4.15.0-129.132_all.deb ... 2021-01-06T21:53:53.308 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking linux-headers-4.15.0-129 (4.15.0-129.132) ... 2021-01-06T21:53:58.530 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package linux-headers-4.15.0-129-generic. 2021-01-06T21:53:58.552 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../6-linux-headers-4.15.0-129-generic_4.15.0-129.132_amd64.deb ... 2021-01-06T21:53:58.598 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking linux-headers-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:54:01.047 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.129.116_amd64.deb ... 2021-01-06T21:54:01.171 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking linux-headers-generic (4.15.0.129.116) over (4.15.0.128.115) ... 2021-01-06T21:54:01.516 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up linux-modules-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:54:02.077 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up linux-image-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:54:02.524 INFO:teuthology.orchestra.run.smithi203.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-129-generic 2021-01-06T21:54:02.524 INFO:teuthology.orchestra.run.smithi203.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-129-generic 2021-01-06T21:54:02.655 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up linux-headers-4.15.0-129 (4.15.0-129.132) ... 2021-01-06T21:54:02.781 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up linux-modules-extra-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:54:03.283 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up linux-headers-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:54:03.409 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up linux-image-generic (4.15.0.129.116) ... 2021-01-06T21:54:03.518 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up linux-headers-generic (4.15.0.129.116) ... 2021-01-06T21:54:03.653 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up linux-generic (4.15.0.129.116) ... 2021-01-06T21:54:03.779 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for linux-image-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:54:03.987 INFO:teuthology.orchestra.run.smithi203.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-01-06T21:54:03.988 INFO:teuthology.orchestra.run.smithi203.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-129-generic 2021-01-06T21:54:19.620 INFO:teuthology.orchestra.run.smithi203.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-01-06T21:54:19.620 INFO:teuthology.orchestra.run.smithi203.stdout:Sourcing file `/etc/default/grub' 2021-01-06T21:54:19.643 INFO:teuthology.orchestra.run.smithi203.stdout:Generating grub configuration file ... 2021-01-06T21:54:19.996 INFO:teuthology.orchestra.run.smithi203.stdout:Found linux image: /boot/vmlinuz-4.15.0-129-generic 2021-01-06T21:54:20.031 INFO:teuthology.orchestra.run.smithi203.stdout:Found initrd image: /boot/initrd.img-4.15.0-129-generic 2021-01-06T21:54:20.371 INFO:teuthology.orchestra.run.smithi203.stdout:Found linux image: /boot/vmlinuz-4.15.0-128-generic 2021-01-06T21:54:20.402 INFO:teuthology.orchestra.run.smithi203.stdout:Found initrd image: /boot/initrd.img-4.15.0-128-generic 2021-01-06T21:54:22.060 INFO:teuthology.orchestra.run.smithi203.stdout:done 2021-01-06T21:54:22.498 DEBUG:teuthology.orchestra.run.smithi203:> dpkg -s linux-image-generic 2021-01-06T21:54:22.521 INFO:teuthology.orchestra.run.smithi203.stdout:Package: linux-image-generic 2021-01-06T21:54:22.521 INFO:teuthology.orchestra.run.smithi203.stdout:Status: install ok installed 2021-01-06T21:54:22.521 INFO:teuthology.orchestra.run.smithi203.stdout:Priority: optional 2021-01-06T21:54:22.522 INFO:teuthology.orchestra.run.smithi203.stdout:Section: kernel 2021-01-06T21:54:22.522 INFO:teuthology.orchestra.run.smithi203.stdout:Installed-Size: 17 2021-01-06T21:54:22.522 INFO:teuthology.orchestra.run.smithi203.stdout:Maintainer: Ubuntu Kernel Team 2021-01-06T21:54:22.522 INFO:teuthology.orchestra.run.smithi203.stdout:Architecture: amd64 2021-01-06T21:54:22.523 INFO:teuthology.orchestra.run.smithi203.stdout:Source: linux-meta 2021-01-06T21:54:22.523 INFO:teuthology.orchestra.run.smithi203.stdout:Version: 4.15.0.129.116 2021-01-06T21:54:22.523 INFO:teuthology.orchestra.run.smithi203.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-01-06T21:54:22.523 INFO:teuthology.orchestra.run.smithi203.stdout:Depends: linux-image-4.15.0-129-generic, linux-modules-extra-4.15.0-129-generic, linux-firmware, intel-microcode, amd64-microcode 2021-01-06T21:54:22.523 INFO:teuthology.orchestra.run.smithi203.stdout:Recommends: thermald 2021-01-06T21:54:22.524 INFO:teuthology.orchestra.run.smithi203.stdout:Description: Generic Linux kernel image 2021-01-06T21:54:22.524 INFO:teuthology.orchestra.run.smithi203.stdout: This package will always depend on the latest generic kernel image 2021-01-06T21:54:22.524 INFO:teuthology.orchestra.run.smithi203.stdout: available. 2021-01-06T21:54:22.525 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-128-generic Expected: 4.15.0-129-generic 2021-01-06T21:54:22.525 DEBUG:teuthology.orchestra.run.smithi091:> uname -r 2021-01-06T21:54:22.532 INFO:teuthology.orchestra.run.smithi091.stdout:4.15.0-128-generic 2021-01-06T21:54:22.533 INFO:teuthology.task.kernel:Running kernel on smithi091: 4.15.0-128-generic 2021-01-06T21:54:22.533 DEBUG:teuthology.orchestra.run.smithi091:> sudo apt-get clean 2021-01-06T21:54:22.865 DEBUG:teuthology.orchestra.run.smithi091:> sudo apt-get update 2021-01-06T21:54:23.138 INFO:teuthology.orchestra.run.smithi091.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-01-06T21:54:23.139 INFO:teuthology.orchestra.run.smithi091.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-01-06T21:54:23.207 INFO:teuthology.orchestra.run.smithi091.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-01-06T21:54:23.478 INFO:teuthology.orchestra.run.smithi091.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-01-06T21:54:23.539 INFO:teuthology.orchestra.run.smithi091.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,478 kB] 2021-01-06T21:54:23.675 INFO:teuthology.orchestra.run.smithi091.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,181 kB] 2021-01-06T21:54:23.988 INFO:teuthology.orchestra.run.smithi091.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [880 kB] 2021-01-06T21:54:24.006 INFO:teuthology.orchestra.run.smithi091.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [288 kB] 2021-01-06T21:54:24.011 INFO:teuthology.orchestra.run.smithi091.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [192 kB] 2021-01-06T21:54:24.015 INFO:teuthology.orchestra.run.smithi091.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [25.5 kB] 2021-01-06T21:54:24.015 INFO:teuthology.orchestra.run.smithi091.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [969 kB] 2021-01-06T21:54:24.053 INFO:teuthology.orchestra.run.smithi091.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,814 kB] 2021-01-06T21:54:24.094 INFO:teuthology.orchestra.run.smithi091.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,097 kB] 2021-01-06T21:54:24.123 INFO:teuthology.orchestra.run.smithi091.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [245 kB] 2021-01-06T21:54:24.128 INFO:teuthology.orchestra.run.smithi091.stdout:Get:15 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [12.5 kB] 2021-01-06T21:54:24.129 INFO:teuthology.orchestra.run.smithi091.stdout:Get:16 http://security.ubuntu.com/ubuntu bionic-security/multiverse i386 Packages [4,400 B] 2021-01-06T21:54:24.129 INFO:teuthology.orchestra.run.smithi091.stdout:Get:17 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [2,644 B] 2021-01-06T21:54:24.170 INFO:teuthology.orchestra.run.smithi091.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [379 kB] 2021-01-06T21:54:24.190 INFO:teuthology.orchestra.run.smithi091.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [214 kB] 2021-01-06T21:54:24.220 INFO:teuthology.orchestra.run.smithi091.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [17.4 kB] 2021-01-06T21:54:24.221 INFO:teuthology.orchestra.run.smithi091.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [29.1 kB] 2021-01-06T21:54:24.222 INFO:teuthology.orchestra.run.smithi091.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,549 kB] 2021-01-06T21:54:24.259 INFO:teuthology.orchestra.run.smithi091.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,700 kB] 2021-01-06T21:54:24.316 INFO:teuthology.orchestra.run.smithi091.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [358 kB] 2021-01-06T21:54:24.324 INFO:teuthology.orchestra.run.smithi091.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [14.0 kB] 2021-01-06T21:54:24.325 INFO:teuthology.orchestra.run.smithi091.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [35.4 kB] 2021-01-06T21:54:24.327 INFO:teuthology.orchestra.run.smithi091.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,980 B] 2021-01-06T21:54:25.671 INFO:teuthology.orchestra.run.smithi091.stdout:Fetched 12.7 MB in 3s (4,912 kB/s) 2021-01-06T21:54:27.133 INFO:teuthology.orchestra.run.smithi091.stdout:Reading package lists... 2021-01-06T21:54:27.180 DEBUG:teuthology.orchestra.run.smithi091:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-01-06T21:54:27.257 INFO:teuthology.orchestra.run.smithi091.stdout:Reading package lists... 2021-01-06T21:54:27.484 INFO:teuthology.orchestra.run.smithi091.stdout:Building dependency tree... 2021-01-06T21:54:27.485 INFO:teuthology.orchestra.run.smithi091.stdout:Reading state information... 2021-01-06T21:54:27.564 INFO:teuthology.orchestra.run.smithi091.stderr:E: Unable to locate package linux-image-current-generic 2021-01-06T21:54:27.567 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-01-06T21:54:27.567 DEBUG:teuthology.orchestra.run.smithi091:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-01-06T21:54:27.645 INFO:teuthology.orchestra.run.smithi091.stdout:Reading package lists... 2021-01-06T21:54:27.899 INFO:teuthology.orchestra.run.smithi091.stdout:Building dependency tree... 2021-01-06T21:54:27.900 INFO:teuthology.orchestra.run.smithi091.stdout:Reading state information... 2021-01-06T21:54:28.072 INFO:teuthology.orchestra.run.smithi091.stdout:The following additional packages will be installed: 2021-01-06T21:54:28.072 INFO:teuthology.orchestra.run.smithi091.stdout: linux-generic linux-headers-4.15.0-129 linux-headers-4.15.0-129-generic 2021-01-06T21:54:28.073 INFO:teuthology.orchestra.run.smithi091.stdout: linux-headers-generic linux-image-4.15.0-129-generic 2021-01-06T21:54:28.073 INFO:teuthology.orchestra.run.smithi091.stdout: linux-modules-4.15.0-129-generic linux-modules-extra-4.15.0-129-generic 2021-01-06T21:54:28.074 INFO:teuthology.orchestra.run.smithi091.stdout:Suggested packages: 2021-01-06T21:54:28.075 INFO:teuthology.orchestra.run.smithi091.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2021-01-06T21:54:28.075 INFO:teuthology.orchestra.run.smithi091.stdout:Recommended packages: 2021-01-06T21:54:28.075 INFO:teuthology.orchestra.run.smithi091.stdout: thermald 2021-01-06T21:54:28.108 INFO:teuthology.orchestra.run.smithi091.stdout:The following NEW packages will be installed: 2021-01-06T21:54:28.108 INFO:teuthology.orchestra.run.smithi091.stdout: linux-headers-4.15.0-129 linux-headers-4.15.0-129-generic 2021-01-06T21:54:28.108 INFO:teuthology.orchestra.run.smithi091.stdout: linux-image-4.15.0-129-generic linux-modules-4.15.0-129-generic 2021-01-06T21:54:28.109 INFO:teuthology.orchestra.run.smithi091.stdout: linux-modules-extra-4.15.0-129-generic 2021-01-06T21:54:28.110 INFO:teuthology.orchestra.run.smithi091.stdout:The following packages will be upgraded: 2021-01-06T21:54:28.111 INFO:teuthology.orchestra.run.smithi091.stdout: linux-generic linux-headers-generic linux-image-generic 2021-01-06T21:54:28.292 INFO:teuthology.orchestra.run.smithi091.stdout:3 upgraded, 5 newly installed, 0 to remove and 9 not upgraded. 2021-01-06T21:54:28.293 INFO:teuthology.orchestra.run.smithi091.stdout:Need to get 66.0 MB of archives. 2021-01-06T21:54:28.293 INFO:teuthology.orchestra.run.smithi091.stdout:After this operation, 334 MB of additional disk space will be used. 2021-01-06T21:54:28.293 INFO:teuthology.orchestra.run.smithi091.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-129-generic amd64 4.15.0-129.132 [13.1 MB] 2021-01-06T21:54:29.330 INFO:teuthology.orchestra.run.smithi091.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-129-generic amd64 4.15.0-129.132 [8,016 kB] 2021-01-06T21:54:29.536 INFO:teuthology.orchestra.run.smithi091.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-129-generic amd64 4.15.0-129.132 [32.8 MB] 2021-01-06T21:54:30.597 INFO:teuthology.orchestra.run.smithi091.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.129.116 [1,872 B] 2021-01-06T21:54:30.597 INFO:teuthology.orchestra.run.smithi091.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.129.116 [2,580 B] 2021-01-06T21:54:30.597 INFO:teuthology.orchestra.run.smithi091.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-129 all 4.15.0-129.132 [10.9 MB] 2021-01-06T21:54:30.904 INFO:teuthology.orchestra.run.smithi091.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-129-generic amd64 4.15.0-129.132 [1,106 kB] 2021-01-06T21:54:30.943 INFO:teuthology.orchestra.run.smithi091.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.129.116 [2,480 B] 2021-01-06T21:54:31.406 INFO:teuthology.orchestra.run.smithi091.stdout:Fetched 66.0 MB in 3s (23.4 MB/s) 2021-01-06T21:54:31.732 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package linux-modules-4.15.0-129-generic. 2021-01-06T21:54:31.733 INFO:teuthology.orchestra.run.smithi091.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102079 files and directories currently installed.) 2021-01-06T21:54:31.733 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../0-linux-modules-4.15.0-129-generic_4.15.0-129.132_amd64.deb ... 2021-01-06T21:54:31.733 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking linux-modules-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:54:33.345 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package linux-image-4.15.0-129-generic. 2021-01-06T21:54:33.356 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../1-linux-image-4.15.0-129-generic_4.15.0-129.132_amd64.deb ... 2021-01-06T21:54:33.449 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking linux-image-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:54:33.948 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-129-generic. 2021-01-06T21:54:33.960 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-129-generic_4.15.0-129.132_amd64.deb ... 2021-01-06T21:54:33.998 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking linux-modules-extra-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:54:37.696 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../3-linux-generic_4.15.0.129.116_amd64.deb ... 2021-01-06T21:54:37.832 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking linux-generic (4.15.0.129.116) over (4.15.0.128.115) ... 2021-01-06T21:54:38.139 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.129.116_amd64.deb ... 2021-01-06T21:54:38.268 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking linux-image-generic (4.15.0.129.116) over (4.15.0.128.115) ... 2021-01-06T21:54:38.597 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package linux-headers-4.15.0-129. 2021-01-06T21:54:38.609 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../5-linux-headers-4.15.0-129_4.15.0-129.132_all.deb ... 2021-01-06T21:54:38.647 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking linux-headers-4.15.0-129 (4.15.0-129.132) ... 2021-01-06T21:54:43.210 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package linux-headers-4.15.0-129-generic. 2021-01-06T21:54:43.234 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../6-linux-headers-4.15.0-129-generic_4.15.0-129.132_amd64.deb ... 2021-01-06T21:54:43.269 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking linux-headers-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:54:45.614 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.129.116_amd64.deb ... 2021-01-06T21:54:45.732 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking linux-headers-generic (4.15.0.129.116) over (4.15.0.128.115) ... 2021-01-06T21:54:46.073 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up linux-modules-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:54:46.603 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up linux-image-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:54:47.040 INFO:teuthology.orchestra.run.smithi091.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-129-generic 2021-01-06T21:54:47.040 INFO:teuthology.orchestra.run.smithi091.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-129-generic 2021-01-06T21:54:47.180 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up linux-headers-4.15.0-129 (4.15.0-129.132) ... 2021-01-06T21:54:47.306 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up linux-modules-extra-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:54:47.791 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up linux-headers-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:54:47.917 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up linux-image-generic (4.15.0.129.116) ... 2021-01-06T21:54:48.043 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up linux-headers-generic (4.15.0.129.116) ... 2021-01-06T21:54:48.177 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up linux-generic (4.15.0.129.116) ... 2021-01-06T21:54:48.304 INFO:teuthology.orchestra.run.smithi091.stdout:Processing triggers for linux-image-4.15.0-129-generic (4.15.0-129.132) ... 2021-01-06T21:54:48.499 INFO:teuthology.orchestra.run.smithi091.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-01-06T21:54:48.499 INFO:teuthology.orchestra.run.smithi091.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-129-generic 2021-01-06T21:55:03.604 INFO:teuthology.orchestra.run.smithi091.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-01-06T21:55:03.604 INFO:teuthology.orchestra.run.smithi091.stdout:Sourcing file `/etc/default/grub' 2021-01-06T21:55:03.633 INFO:teuthology.orchestra.run.smithi091.stdout:Generating grub configuration file ... 2021-01-06T21:55:04.033 INFO:teuthology.orchestra.run.smithi091.stdout:Found linux image: /boot/vmlinuz-4.15.0-129-generic 2021-01-06T21:55:04.065 INFO:teuthology.orchestra.run.smithi091.stdout:Found initrd image: /boot/initrd.img-4.15.0-129-generic 2021-01-06T21:55:04.446 INFO:teuthology.orchestra.run.smithi091.stdout:Found linux image: /boot/vmlinuz-4.15.0-128-generic 2021-01-06T21:55:04.475 INFO:teuthology.orchestra.run.smithi091.stdout:Found initrd image: /boot/initrd.img-4.15.0-128-generic 2021-01-06T21:55:06.084 INFO:teuthology.orchestra.run.smithi091.stdout:done 2021-01-06T21:55:06.475 DEBUG:teuthology.orchestra.run.smithi091:> dpkg -s linux-image-generic 2021-01-06T21:55:06.499 INFO:teuthology.orchestra.run.smithi091.stdout:Package: linux-image-generic 2021-01-06T21:55:06.499 INFO:teuthology.orchestra.run.smithi091.stdout:Status: install ok installed 2021-01-06T21:55:06.499 INFO:teuthology.orchestra.run.smithi091.stdout:Priority: optional 2021-01-06T21:55:06.500 INFO:teuthology.orchestra.run.smithi091.stdout:Section: kernel 2021-01-06T21:55:06.500 INFO:teuthology.orchestra.run.smithi091.stdout:Installed-Size: 17 2021-01-06T21:55:06.500 INFO:teuthology.orchestra.run.smithi091.stdout:Maintainer: Ubuntu Kernel Team 2021-01-06T21:55:06.500 INFO:teuthology.orchestra.run.smithi091.stdout:Architecture: amd64 2021-01-06T21:55:06.500 INFO:teuthology.orchestra.run.smithi091.stdout:Source: linux-meta 2021-01-06T21:55:06.501 INFO:teuthology.orchestra.run.smithi091.stdout:Version: 4.15.0.129.116 2021-01-06T21:55:06.501 INFO:teuthology.orchestra.run.smithi091.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-01-06T21:55:06.501 INFO:teuthology.orchestra.run.smithi091.stdout:Depends: linux-image-4.15.0-129-generic, linux-modules-extra-4.15.0-129-generic, linux-firmware, intel-microcode, amd64-microcode 2021-01-06T21:55:06.501 INFO:teuthology.orchestra.run.smithi091.stdout:Recommends: thermald 2021-01-06T21:55:06.502 INFO:teuthology.orchestra.run.smithi091.stdout:Description: Generic Linux kernel image 2021-01-06T21:55:06.502 INFO:teuthology.orchestra.run.smithi091.stdout: This package will always depend on the latest generic kernel image 2021-01-06T21:55:06.502 INFO:teuthology.orchestra.run.smithi091.stdout: available. 2021-01-06T21:55:06.502 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-128-generic Expected: 4.15.0-129-generic 2021-01-06T21:55:06.503 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2021-01-06T21:55:06.503 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-01-06T21:55:06.503 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-01-06T21:55:06.504 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2021-01-06T21:55:06.504 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi203.front.sepia.ceph.com, path=None, version=distro) 2021-01-06T21:55:06.504 DEBUG:teuthology.orchestra.run.smithi203:> sudo apt-get clean 2021-01-06T21:55:06.575 DEBUG:teuthology.orchestra.run.smithi203:> sudo apt-get update 2021-01-06T21:55:06.826 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-01-06T21:55:06.827 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:2 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-01-06T21:55:06.896 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-01-06T21:55:06.977 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-01-06T21:55:08.691 INFO:teuthology.orchestra.run.smithi203.stdout:Reading package lists... 2021-01-06T21:55:08.733 DEBUG:teuthology.orchestra.run.smithi203:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-01-06T21:55:08.850 INFO:teuthology.orchestra.run.smithi203.stdout:Reading package lists... 2021-01-06T21:55:09.096 INFO:teuthology.orchestra.run.smithi203.stdout:Building dependency tree... 2021-01-06T21:55:09.096 INFO:teuthology.orchestra.run.smithi203.stdout:Reading state information... 2021-01-06T21:55:09.176 INFO:teuthology.orchestra.run.smithi203.stderr:E: Unable to locate package linux-image-current-generic 2021-01-06T21:55:09.177 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-01-06T21:55:09.178 DEBUG:teuthology.orchestra.run.smithi203:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-01-06T21:55:09.257 INFO:teuthology.orchestra.run.smithi203.stdout:Reading package lists... 2021-01-06T21:55:09.507 INFO:teuthology.orchestra.run.smithi203.stdout:Building dependency tree... 2021-01-06T21:55:09.508 INFO:teuthology.orchestra.run.smithi203.stdout:Reading state information... 2021-01-06T21:55:09.706 INFO:teuthology.orchestra.run.smithi203.stdout:linux-image-generic is already the newest version (4.15.0.129.116). 2021-01-06T21:55:09.706 INFO:teuthology.orchestra.run.smithi203.stdout:0 upgraded, 0 newly installed, 0 to remove and 9 not upgraded. 2021-01-06T21:55:09.707 DEBUG:teuthology.orchestra.run.smithi203:> dpkg -s linux-image-generic 2021-01-06T21:55:09.731 INFO:teuthology.orchestra.run.smithi203.stdout:Package: linux-image-generic 2021-01-06T21:55:09.731 INFO:teuthology.orchestra.run.smithi203.stdout:Status: install ok installed 2021-01-06T21:55:09.731 INFO:teuthology.orchestra.run.smithi203.stdout:Priority: optional 2021-01-06T21:55:09.731 INFO:teuthology.orchestra.run.smithi203.stdout:Section: kernel 2021-01-06T21:55:09.732 INFO:teuthology.orchestra.run.smithi203.stdout:Installed-Size: 17 2021-01-06T21:55:09.732 INFO:teuthology.orchestra.run.smithi203.stdout:Maintainer: Ubuntu Kernel Team 2021-01-06T21:55:09.732 INFO:teuthology.orchestra.run.smithi203.stdout:Architecture: amd64 2021-01-06T21:55:09.732 INFO:teuthology.orchestra.run.smithi203.stdout:Source: linux-meta 2021-01-06T21:55:09.732 INFO:teuthology.orchestra.run.smithi203.stdout:Version: 4.15.0.129.116 2021-01-06T21:55:09.733 INFO:teuthology.orchestra.run.smithi203.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-01-06T21:55:09.733 INFO:teuthology.orchestra.run.smithi203.stdout:Depends: linux-image-4.15.0-129-generic, linux-modules-extra-4.15.0-129-generic, linux-firmware, intel-microcode, amd64-microcode 2021-01-06T21:55:09.733 INFO:teuthology.orchestra.run.smithi203.stdout:Recommends: thermald 2021-01-06T21:55:09.733 INFO:teuthology.orchestra.run.smithi203.stdout:Description: Generic Linux kernel image 2021-01-06T21:55:09.733 INFO:teuthology.orchestra.run.smithi203.stdout: This package will always depend on the latest generic kernel image 2021-01-06T21:55:09.734 INFO:teuthology.orchestra.run.smithi203.stdout: available. 2021-01-06T21:55:09.734 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2021-01-06T21:55:09.781 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.bPzDAuqPcR 2021-01-06T21:55:09.781 DEBUG:teuthology.orchestra.run.smithi203:> sudo cp /boot/grub/grub.cfg /tmp/tmp.bPzDAuqPcR 2021-01-06T21:55:09.836 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.bPzDAuqPcR 2021-01-06T21:55:09.968 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.bPzDAuqPcR is 8KB 2021-01-06T21:55:09.982 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.bPzDAuqPcR 2021-01-06T21:55:09.988 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-01-06T21:55:10.044 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T21:55:10.044 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-01-06T21:55:10.045 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-01-06T21:55:10.112 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-129-generic 2021-01-06T21:55:10.113 DEBUG:teuthology.orchestra.run.smithi203:> sudo update-grub 2021-01-06T21:55:11.496 INFO:teuthology.orchestra.run.smithi203.stderr:Sourcing file `/etc/default/grub' 2021-01-06T21:55:11.514 INFO:teuthology.orchestra.run.smithi203.stderr:Generating grub configuration file ... 2021-01-06T21:55:11.758 INFO:teuthology.orchestra.run.smithi203.stderr:Found linux image: /boot/vmlinuz-4.15.0-129-generic 2021-01-06T21:55:11.769 INFO:teuthology.orchestra.run.smithi203.stderr:Found initrd image: /boot/initrd.img-4.15.0-129-generic 2021-01-06T21:55:12.074 INFO:teuthology.orchestra.run.smithi203.stderr:Found linux image: /boot/vmlinuz-4.15.0-128-generic 2021-01-06T21:55:12.087 INFO:teuthology.orchestra.run.smithi203.stderr:Found initrd image: /boot/initrd.img-4.15.0-128-generic 2021-01-06T21:55:13.288 INFO:teuthology.orchestra.run.smithi203.stderr:done 2021-01-06T21:55:13.291 DEBUG:teuthology.orchestra.run.smithi203:> sudo shutdown -r now 2021-01-06T21:55:13.295 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2021-01-06T21:55:13.295 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi091.front.sepia.ceph.com, path=None, version=distro) 2021-01-06T21:55:13.296 DEBUG:teuthology.orchestra.run.smithi091:> sudo apt-get clean 2021-01-06T21:55:13.374 DEBUG:teuthology.orchestra.run.smithi091:> sudo apt-get update 2021-01-06T21:55:13.626 INFO:teuthology.orchestra.run.smithi091.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-01-06T21:55:13.682 INFO:teuthology.orchestra.run.smithi091.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-01-06T21:55:13.697 INFO:teuthology.orchestra.run.smithi091.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-01-06T21:55:13.779 INFO:teuthology.orchestra.run.smithi091.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-01-06T21:55:15.465 INFO:teuthology.orchestra.run.smithi091.stdout:Reading package lists... 2021-01-06T21:55:15.513 DEBUG:teuthology.orchestra.run.smithi091:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-01-06T21:55:16.758 INFO:teuthology.orchestra.run.smithi091.stdout:Reading package lists... 2021-01-06T21:55:16.998 INFO:teuthology.orchestra.run.smithi091.stdout:Building dependency tree... 2021-01-06T21:55:16.999 INFO:teuthology.orchestra.run.smithi091.stdout:Reading state information... 2021-01-06T21:55:17.083 INFO:teuthology.orchestra.run.smithi091.stderr:E: Unable to locate package linux-image-current-generic 2021-01-06T21:55:17.084 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-01-06T21:55:17.085 DEBUG:teuthology.orchestra.run.smithi091:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-01-06T21:55:17.193 INFO:teuthology.orchestra.run.smithi091.stdout:Reading package lists... 2021-01-06T21:55:17.446 INFO:teuthology.orchestra.run.smithi091.stdout:Building dependency tree... 2021-01-06T21:55:17.447 INFO:teuthology.orchestra.run.smithi091.stdout:Reading state information... 2021-01-06T21:55:17.646 INFO:teuthology.orchestra.run.smithi091.stdout:linux-image-generic is already the newest version (4.15.0.129.116). 2021-01-06T21:55:17.646 INFO:teuthology.orchestra.run.smithi091.stdout:0 upgraded, 0 newly installed, 0 to remove and 9 not upgraded. 2021-01-06T21:55:17.648 DEBUG:teuthology.orchestra.run.smithi091:> dpkg -s linux-image-generic 2021-01-06T21:55:17.720 INFO:teuthology.orchestra.run.smithi091.stdout:Package: linux-image-generic 2021-01-06T21:55:17.720 INFO:teuthology.orchestra.run.smithi091.stdout:Status: install ok installed 2021-01-06T21:55:17.720 INFO:teuthology.orchestra.run.smithi091.stdout:Priority: optional 2021-01-06T21:55:17.720 INFO:teuthology.orchestra.run.smithi091.stdout:Section: kernel 2021-01-06T21:55:17.721 INFO:teuthology.orchestra.run.smithi091.stdout:Installed-Size: 17 2021-01-06T21:55:17.721 INFO:teuthology.orchestra.run.smithi091.stdout:Maintainer: Ubuntu Kernel Team 2021-01-06T21:55:17.721 INFO:teuthology.orchestra.run.smithi091.stdout:Architecture: amd64 2021-01-06T21:55:17.721 INFO:teuthology.orchestra.run.smithi091.stdout:Source: linux-meta 2021-01-06T21:55:17.721 INFO:teuthology.orchestra.run.smithi091.stdout:Version: 4.15.0.129.116 2021-01-06T21:55:17.722 INFO:teuthology.orchestra.run.smithi091.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-01-06T21:55:17.722 INFO:teuthology.orchestra.run.smithi091.stdout:Depends: linux-image-4.15.0-129-generic, linux-modules-extra-4.15.0-129-generic, linux-firmware, intel-microcode, amd64-microcode 2021-01-06T21:55:17.722 INFO:teuthology.orchestra.run.smithi091.stdout:Recommends: thermald 2021-01-06T21:55:17.722 INFO:teuthology.orchestra.run.smithi091.stdout:Description: Generic Linux kernel image 2021-01-06T21:55:17.723 INFO:teuthology.orchestra.run.smithi091.stdout: This package will always depend on the latest generic kernel image 2021-01-06T21:55:17.723 INFO:teuthology.orchestra.run.smithi091.stdout: available. 2021-01-06T21:55:17.723 DEBUG:teuthology.orchestra.run.smithi091:> mktemp 2021-01-06T21:55:17.774 INFO:teuthology.orchestra.run.smithi091.stdout:/tmp/tmp.jJN8PkEo75 2021-01-06T21:55:17.775 DEBUG:teuthology.orchestra.run.smithi091:> sudo cp /boot/grub/grub.cfg /tmp/tmp.jJN8PkEo75 2021-01-06T21:55:17.839 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 0666 /tmp/tmp.jJN8PkEo75 2021-01-06T21:55:17.973 DEBUG:teuthology.orchestra.remote:smithi091:/tmp/tmp.jJN8PkEo75 is 8KB 2021-01-06T21:55:17.997 DEBUG:teuthology.orchestra.run.smithi091:> rm -fr /tmp/tmp.jJN8PkEo75 2021-01-06T21:55:18.005 DEBUG:teuthology.orchestra.run.smithi091:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-01-06T21:55:18.065 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T21:55:18.068 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-01-06T21:55:18.071 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-01-06T21:55:18.173 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-129-generic 2021-01-06T21:55:18.174 DEBUG:teuthology.orchestra.run.smithi091:> sudo update-grub 2021-01-06T21:55:19.583 INFO:teuthology.orchestra.run.smithi091.stderr:Sourcing file `/etc/default/grub' 2021-01-06T21:55:19.597 INFO:teuthology.orchestra.run.smithi091.stderr:Generating grub configuration file ... 2021-01-06T21:55:19.868 INFO:teuthology.orchestra.run.smithi091.stderr:Found linux image: /boot/vmlinuz-4.15.0-129-generic 2021-01-06T21:55:19.881 INFO:teuthology.orchestra.run.smithi091.stderr:Found initrd image: /boot/initrd.img-4.15.0-129-generic 2021-01-06T21:55:20.217 INFO:teuthology.orchestra.run.smithi091.stderr:Found linux image: /boot/vmlinuz-4.15.0-128-generic 2021-01-06T21:55:20.230 INFO:teuthology.orchestra.run.smithi091.stderr:Found initrd image: /boot/initrd.img-4.15.0-128-generic 2021-01-06T21:55:21.388 INFO:teuthology.orchestra.run.smithi091.stderr:done 2021-01-06T21:55:21.390 DEBUG:teuthology.orchestra.run.smithi091:> sudo shutdown -r now 2021-01-06T21:55:51.418 INFO:teuthology.misc:Re-opening connections... 2021-01-06T21:55:51.420 INFO:teuthology.misc:trying to connect to ubuntu@smithi203.front.sepia.ceph.com 2021-01-06T21:55:51.421 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-06T21:55:51.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:56:09.794 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2021-01-06T21:56:09.795 INFO:teuthology.misc:trying to connect to ubuntu@smithi091.front.sepia.ceph.com 2021-01-06T21:56:09.796 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-06T21:56:09.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:57:09.798 DEBUG:teuthology.orchestra.remote:timed out 2021-01-06T21:57:09.800 DEBUG:teuthology.misc:waited 78.37996935844421 2021-01-06T21:57:10.801 INFO:teuthology.misc:trying to connect to ubuntu@smithi203.front.sepia.ceph.com 2021-01-06T21:57:10.801 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-06T21:57:10.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:57:10.881 DEBUG:teuthology.orchestra.run.smithi203:> true 2021-01-06T21:57:11.211 DEBUG:teuthology.misc:waited 79.79075646400452 2021-01-06T21:57:12.213 INFO:teuthology.misc:trying to connect to ubuntu@smithi091.front.sepia.ceph.com 2021-01-06T21:57:12.213 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-06T21:57:12.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:57:12.294 DEBUG:teuthology.orchestra.run.smithi091:> true 2021-01-06T21:57:12.651 DEBUG:teuthology.misc:waited 81.23102855682373 2021-01-06T21:57:13.652 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2021-01-06T21:57:14.069 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.15.0-129-generic"... 2021-01-06T21:57:14.070 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2021-01-06T21:57:14.077 INFO:teuthology.orchestra.run.smithi203.stdout:4.15.0-129-generic 2021-01-06T21:57:14.078 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-129-generic vs 4.15.0-129-generic 2021-01-06T21:57:14.078 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-01-06T21:57:14.078 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-01-06T21:57:14.079 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2021-01-06T21:57:14.079 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.15.0-129-generic"... 2021-01-06T21:57:14.079 DEBUG:teuthology.orchestra.run.smithi091:> uname -r 2021-01-06T21:57:14.087 INFO:teuthology.orchestra.run.smithi091.stdout:4.15.0-129-generic 2021-01-06T21:57:14.087 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-129-generic vs 4.15.0-129-generic 2021-01-06T21:57:14.087 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-01-06T21:57:14.088 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-01-06T21:57:15.089 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2021-01-06T21:57:15.090 DEBUG:teuthology.orchestra.run.smithi203:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-01-06T21:57:15.187 INFO:teuthology.orchestra.run.smithi203.stdout:ttyS1 2021-01-06T21:57:15.205 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2021-01-06T21:57:15.205 DEBUG:teuthology.orchestra.run.smithi091:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-01-06T21:57:15.515 INFO:teuthology.orchestra.run.smithi091.stdout:ttyS1 2021-01-06T21:57:15.538 INFO:teuthology.run_tasks:Running task internal.base... 2021-01-06T21:57:15.546 INFO:teuthology.task.internal:Creating test directory... 2021-01-06T21:57:15.546 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-01-06T21:57:15.549 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-01-06T21:57:15.555 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2021-01-06T21:57:15.702 INFO:teuthology.run_tasks:Running task internal.archive... 2021-01-06T21:57:15.744 INFO:teuthology.task.internal:Creating archive directory... 2021-01-06T21:57:15.745 DEBUG:teuthology.orchestra.run.smithi091:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-01-06T21:57:15.749 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-01-06T21:57:15.768 INFO:teuthology.run_tasks:Running task internal.coredump... 2021-01-06T21:57:16.116 INFO:teuthology.task.internal:Enabling coredump saving... 2021-01-06T21:57:16.117 DEBUG:teuthology.orchestra.run.smithi091:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-01-06T21:57:16.122 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-01-06T21:57:16.147 INFO:teuthology.orchestra.run.smithi091.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-01-06T21:57:16.152 INFO:teuthology.orchestra.run.smithi091.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-01-06T21:57:16.161 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-01-06T21:57:16.169 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-01-06T21:57:16.171 INFO:teuthology.run_tasks:Running task internal.sudo... 2021-01-06T21:57:16.179 INFO:teuthology.task.internal:Configuring sudo... 2021-01-06T21:57:16.180 DEBUG:teuthology.orchestra.run.smithi091:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-01-06T21:57:16.197 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-01-06T21:57:16.229 INFO:teuthology.run_tasks:Running task internal.syslog... 2021-01-06T21:57:16.240 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2021-01-06T21:57:16.241 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-01-06T21:57:16.254 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-01-06T21:57:16.276 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-01-06T21:57:16.324 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-01-06T21:57:16.372 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T21:57:17.192 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-01-06T21:57:17.216 DEBUG:teuthology.orchestra.run.smithi091:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-01-06T21:57:17.223 DEBUG:teuthology.orchestra.run.smithi091:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-01-06T21:57:17.270 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T21:57:17.271 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-01-06T21:57:17.478 DEBUG:teuthology.orchestra.run.smithi091:> sudo service rsyslog restart 2021-01-06T21:57:17.516 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2021-01-06T21:57:17.582 INFO:teuthology.run_tasks:Running task internal.timer... 2021-01-06T21:57:17.612 INFO:teuthology.task.internal:Starting timer... 2021-01-06T21:57:17.613 INFO:teuthology.run_tasks:Running task pcp... 2021-01-06T21:57:17.964 INFO:teuthology.run_tasks:Running task selinux... 2021-01-06T21:57:18.335 DEBUG:teuthology.task.selinux:Excluding smithi203: OS 'ubuntu' does not support SELinux 2021-01-06T21:57:18.336 DEBUG:teuthology.task.selinux:Excluding smithi091: OS 'ubuntu' does not support SELinux 2021-01-06T21:57:18.336 DEBUG:teuthology.task.selinux:Getting current SELinux state 2021-01-06T21:57:18.337 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2021-01-06T21:57:18.337 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2021-01-06T21:57:18.338 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2021-01-06T21:57:19.303 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2021-01-06T21:57:19.303 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2021-01-06T21:57:19.320 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2021-01-06T21:57:19.322 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi203.front.sepia.ceph.com,smithi091.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2021-01-06T21:57:19.692 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2021-01-06T21:57:20.583 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2021-01-06T21:57:23.066 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2021-01-06T21:57:23.085 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:23 +0000 (0:00:00.066) 0:00:00.066 ***** 2021-01-06T21:57:23.098 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:23 +0000 (0:00:00.013) 0:00:00.079 ***** 2021-01-06T21:57:24.662 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi091.front.sepia.ceph.com] 2021-01-06T21:57:24.737 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:24 +0000 (0:00:01.638) 0:00:01.718 ***** 2021-01-06T21:57:24.871 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] 2021-01-06T21:57:24.945 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:24 +0000 (0:00:00.208) 0:00:01.926 ***** 2021-01-06T21:57:24.981 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2021-01-06T21:57:24.983 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-01-06T21:57:25.019 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:25 +0000 (0:00:00.074) 0:00:02.000 ***** 2021-01-06T21:57:25.067 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-01-06T21:57:25.102 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:25 +0000 (0:00:00.082) 0:00:02.083 ***** 2021-01-06T21:57:25.489 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2021-01-06T21:57:25.490 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} changed: [smithi203.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-01-06T21:57:25.526 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:25 +0000 (0:00:00.424) 0:00:02.507 ***** 2021-01-06T21:57:25.561 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:25 +0000 (0:00:00.033) 0:00:02.541 ***** 2021-01-06T21:57:25.838 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.070174", "end": "2021-01-06 21:57:25.822694", "rc": 0, "start": "2021-01-06 21:57:25.752520", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.068623", "end": "2021-01-06 21:57:25.817999", "rc": 0, "start": "2021-01-06 21:57:25.749376", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2021-01-06T21:57:25.873 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:25 +0000 (0:00:00.312) 0:00:02.853 ***** 2021-01-06T21:57:25.905 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:25 +0000 (0:00:00.031) 0:00:02.885 ***** 2021-01-06T21:57:26.140 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2021-01-06T21:57:26.142 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} ok: [smithi203.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-06T21:57:26.177 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:26 +0000 (0:00:00.272) 0:00:03.158 ***** 2021-01-06T21:57:26.209 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:26 +0000 (0:00:00.031) 0:00:03.189 ***** 2021-01-06T21:57:26.928 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1609970246.1894293-279285135893850/source", "state": "file", "uid": 0} 2021-01-06T21:57:26.929 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1609970246.2248344-266424475824429/source", "state": "file", "uid": 0} 2021-01-06T21:57:26.969 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:26 +0000 (0:00:00.759) 0:00:03.949 ***** 2021-01-06T21:57:27.003 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:27 +0000 (0:00:00.034) 0:00:03.984 ***** 2021-01-06T21:57:27.274 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2021-01-06T21:57:27.275 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2021-01-06T21:57:27.277 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-01-06T21:57:27.312 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2021-01-06T21:57:27.313 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-01-06T21:57:27.445 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2021-01-06T21:57:27.452 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-01-06T21:57:27.624 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:27 +0000 (0:00:00.618) 0:00:04.602 ***** 2021-01-06T21:57:27.659 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:27 +0000 (0:00:00.037) 0:00:04.640 ***** 2021-01-06T21:57:28.360 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi091.front.sepia.ceph.com] 2021-01-06T21:57:28.453 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. Wednesday 06 January 2021 21:57:28 +0000 (0:00:00.793) 0:00:05.433 ***** 2021-01-06T21:57:28.463 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] 2021-01-06T21:57:28.549 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:28 +0000 (0:00:00.096) 0:00:05.530 ***** 2021-01-06T21:57:28.857 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-01-06T21:57:28.860 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.195213", "end": "2021-01-06 21:57:28.822404", "rc": 0, "start": "2021-01-06 21:57:28.627191", "stderr": "", "stderr_lines": [], "stdout": "0caff10a478f2674721da1965013b225f3ea5800", "stdout_lines": ["0caff10a478f2674721da1965013b225f3ea5800"]} 2021-01-06T21:57:28.900 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:28 +0000 (0:00:00.351) 0:00:05.881 ***** 2021-01-06T21:57:28.965 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.173504", "end": "2021-01-06 21:57:28.927629", "rc": 0, "start": "2021-01-06 21:57:28.754125", "stderr": "", "stderr_lines": [], "stdout": "0caff10a478f2674721da1965013b225f3ea5800", "stdout_lines": ["0caff10a478f2674721da1965013b225f3ea5800"]} 2021-01-06T21:57:29.003 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:29 +0000 (0:00:00.102) 0:00:05.983 ***** 2021-01-06T21:57:29.051 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.011899", "end": "2021-01-06 21:57:29.029213", "failed_when_result": false, "rc": 0, "start": "2021-01-06 21:57:29.017314", "stderr": "", "stderr_lines": [], "stdout": "00db66c131045494585d1009527441f429a52419", "stdout_lines": ["00db66c131045494585d1009527441f429a52419"]} 2021-01-06T21:57:29.088 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:29 +0000 (0:00:00.085) 0:00:06.069 ***** 2021-01-06T21:57:29.115 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2021-01-06T21:57:29.117 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:57:29.153 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:29 +0000 (0:00:00.064) 0:00:06.133 ***** 2021-01-06T21:57:29.181 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2021-01-06T21:57:29.183 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:57:29.225 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:29 +0000 (0:00:00.072) 0:00:06.206 ***** 2021-01-06T21:57:29.265 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:57:29.302 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:29 +0000 (0:00:00.077) 0:00:06.283 ***** 2021-01-06T21:57:29.330 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2021-01-06T21:57:29.332 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:57:29.367 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:29 +0000 (0:00:00.064) 0:00:06.348 ***** 2021-01-06T21:57:29.395 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2021-01-06T21:57:29.398 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:57:29.445 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:29 +0000 (0:00:00.077) 0:00:06.425 ***** 2021-01-06T21:57:29.451 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-01-06T21:57:29.453 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.258698", "end": "2021-01-06 21:57:29.376248", "failed_when_result": false, "rc": 0, "start": "2021-01-06 21:57:29.117550", "stderr": "", "stderr_lines": [], "stdout": "00db66c131045494585d1009527441f429a52419", "stdout_lines": ["00db66c131045494585d1009527441f429a52419"]} 2021-01-06T21:57:29.490 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:29 +0000 (0:00:00.045) 0:00:06.470 ***** 2021-01-06T21:57:29.516 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2021-01-06T21:57:29.519 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:57:29.554 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:29 +0000 (0:00:00.063) 0:00:06.534 ***** 2021-01-06T21:57:29.571 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-01-06T21:57:29.573 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-06T21:57:29.622 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:29 +0000 (0:00:00.068) 0:00:06.602 ***** 2021-01-06T21:57:29.628 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2021-01-06T21:57:29.631 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:57:29.669 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:29 +0000 (0:00:00.047) 0:00:06.650 ***** 2021-01-06T21:57:29.696 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2021-01-06T21:57:29.698 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:57:29.732 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:29 +0000 (0:00:00.062) 0:00:06.713 ***** 2021-01-06T21:57:29.769 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2021-01-06T21:57:29.771 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:57:29.804 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:29 +0000 (0:00:00.072) 0:00:06.785 ***** 2021-01-06T21:57:29.820 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2021-01-06T21:57:29.822 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-01-06T21:57:29.832 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:57:29.872 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:29 +0000 (0:00:00.068) 0:00:06.853 ***** 2021-01-06T21:57:29.952 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2021-01-06T21:57:29.954 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-01-06T21:57:29.996 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-01-06T21:57:29.998 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-06T21:57:30.036 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:30 +0000 (0:00:00.163) 0:00:07.017 ***** 2021-01-06T21:57:30.084 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2021-01-06T21:57:30.086 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-01-06T21:57:30.200 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-01-06T21:57:30.212 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-01-06T21:57:30.329 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-01-06T21:57:30.340 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-01-06T21:57:30.456 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-01-06T21:57:30.478 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-01-06T21:57:30.582 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-01-06T21:57:30.605 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-01-06T21:57:30.710 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-01-06T21:57:30.741 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-01-06T21:57:30.825 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-01-06T21:57:30.868 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-01-06T21:57:30.954 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-01-06T21:57:30.988 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-01-06T21:57:31.083 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-01-06T21:57:31.115 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-01-06T21:57:31.210 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-01-06T21:57:31.242 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-01-06T21:57:31.337 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-01-06T21:57:31.372 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-01-06T21:57:31.466 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-01-06T21:57:31.498 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-01-06T21:57:31.605 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-01-06T21:57:31.636 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-01-06T21:57:31.721 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-01-06T21:57:31.764 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-01-06T21:57:31.859 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-01-06T21:57:31.903 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-01-06T21:57:31.988 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-01-06T21:57:32.032 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-01-06T21:57:32.118 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-01-06T21:57:32.167 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-01-06T21:57:32.252 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-01-06T21:57:32.295 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-01-06T21:57:32.380 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-01-06T21:57:32.422 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-01-06T21:57:32.507 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-01-06T21:57:32.553 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-01-06T21:57:32.639 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-01-06T21:57:32.674 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-01-06T21:57:32.758 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-01-06T21:57:32.800 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-01-06T21:57:32.887 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-01-06T21:57:32.930 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-01-06T21:57:33.014 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-01-06T21:57:33.058 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-01-06T21:57:33.143 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-01-06T21:57:33.186 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-01-06T21:57:33.270 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-01-06T21:57:33.313 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-01-06T21:57:33.398 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-01-06T21:57:33.399 INFO:teuthology.task.ansible.out: 2021-01-06T21:57:33.431 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-01-06T21:57:33.527 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-01-06T21:57:33.559 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-01-06T21:57:33.654 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-01-06T21:57:33.687 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-01-06T21:57:33.771 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-01-06T21:57:33.815 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-01-06T21:57:33.898 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-01-06T21:57:33.940 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-01-06T21:57:34.015 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-01-06T21:57:34.069 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-01-06T21:57:34.143 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-01-06T21:57:34.196 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-01-06T21:57:34.261 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-01-06T21:57:34.324 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-01-06T21:57:34.389 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-01-06T21:57:34.452 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-01-06T21:57:34.507 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-01-06T21:57:34.592 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-01-06T21:57:34.635 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-01-06T21:57:34.709 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-01-06T21:57:34.762 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-01-06T21:57:34.837 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-01-06T21:57:34.890 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-01-06T21:57:34.964 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-01-06T21:57:35.006 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-01-06T21:57:35.091 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-01-06T21:57:35.134 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-01-06T21:57:35.218 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-01-06T21:57:35.260 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-01-06T21:57:35.345 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-01-06T21:57:35.388 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-01-06T21:57:35.473 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-01-06T21:57:35.505 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-01-06T21:57:35.589 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-01-06T21:57:35.643 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-01-06T21:57:35.720 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-01-06T21:57:35.764 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-01-06T21:57:35.838 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-01-06T21:57:35.891 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-01-06T21:57:35.966 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-01-06T21:57:36.019 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-01-06T21:57:36.094 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-01-06T21:57:36.137 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-01-06T21:57:36.224 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-01-06T21:57:36.265 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-01-06T21:57:36.363 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-01-06T21:57:36.397 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-01-06T21:57:36.483 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-01-06T21:57:36.516 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-01-06T21:57:36.602 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-01-06T21:57:36.656 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-01-06T21:57:36.730 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-01-06T21:57:36.784 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-01-06T21:57:36.858 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-01-06T21:57:36.911 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-01-06T21:57:36.986 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-01-06T21:57:37.039 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-01-06T21:57:37.114 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-01-06T21:57:37.157 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-01-06T21:57:37.252 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-01-06T21:57:37.285 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-01-06T21:57:37.369 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-01-06T21:57:37.412 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-01-06T21:57:37.496 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-01-06T21:57:37.530 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-01-06T21:57:37.626 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-01-06T21:57:37.659 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-01-06T21:57:37.743 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-01-06T21:57:37.801 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-01-06T21:57:37.884 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-01-06T21:57:37.928 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-01-06T21:57:38.013 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-01-06T21:57:38.067 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-01-06T21:57:38.141 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-01-06T21:57:38.186 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-01-06T21:57:38.269 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-01-06T21:57:38.312 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-01-06T21:57:38.397 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-01-06T21:57:38.441 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-01-06T21:57:38.517 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-01-06T21:57:38.571 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-01-06T21:57:38.646 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-01-06T21:57:38.700 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-01-06T21:57:38.775 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-01-06T21:57:38.829 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-01-06T21:57:38.903 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-01-06T21:57:38.956 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-01-06T21:57:39.032 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-01-06T21:57:39.085 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-01-06T21:57:39.160 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-01-06T21:57:39.214 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-01-06T21:57:39.288 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-01-06T21:57:39.341 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-01-06T21:57:39.418 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-01-06T21:57:39.472 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-01-06T21:57:39.546 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-01-06T21:57:39.602 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-01-06T21:57:39.703 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-01-06T21:57:39.844 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-01-06T21:57:39.900 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-01-06T21:57:40.040 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-01-06T21:57:40.082 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-01-06T21:57:40.237 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-01-06T21:57:40.269 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-01-06T21:57:40.407 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-01-06T21:57:40.515 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-01-06T21:57:40.559 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-01-06T21:57:40.562 INFO:teuthology.task.ansible.out: 2021-01-06T21:57:40.677 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-01-06T21:57:40.715 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-01-06T21:57:40.818 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-01-06T21:57:40.841 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-01-06T21:57:40.970 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-01-06T21:57:40.983 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-01-06T21:57:41.115 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-01-06T21:57:41.116 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-01-06T21:57:41.258 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-01-06T21:57:41.260 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-01-06T21:57:41.387 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-01-06T21:57:41.398 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-01-06T21:57:41.515 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-01-06T21:57:41.539 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-01-06T21:57:41.647 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-01-06T21:57:41.667 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-01-06T21:57:41.775 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-01-06T21:57:41.798 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-01-06T21:57:41.907 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-01-06T21:57:41.930 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-01-06T21:57:42.035 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-01-06T21:57:42.067 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-01-06T21:57:42.162 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-01-06T21:57:42.195 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-01-06T21:57:42.290 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-01-06T21:57:42.333 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-01-06T21:57:42.421 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-01-06T21:57:42.454 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-01-06T21:57:42.549 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-01-06T21:57:42.572 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-01-06T21:57:42.677 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-01-06T21:57:42.688 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-01-06T21:57:42.793 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-01-06T21:57:42.815 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-01-06T21:57:42.919 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-01-06T21:57:42.941 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-01-06T21:57:43.046 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-01-06T21:57:43.068 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-01-06T21:57:43.162 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-01-06T21:57:43.195 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-01-06T21:57:43.295 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-01-06T21:57:43.317 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-01-06T21:57:43.423 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-01-06T21:57:43.445 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-01-06T21:57:43.540 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-01-06T21:57:43.572 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-01-06T21:57:43.667 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-01-06T21:57:43.699 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-01-06T21:57:43.794 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-01-06T21:57:43.816 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-01-06T21:57:43.911 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-01-06T21:57:43.943 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-01-06T21:57:44.038 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-01-06T21:57:44.071 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-01-06T21:57:44.166 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-01-06T21:57:44.199 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-01-06T21:57:44.295 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-01-06T21:57:44.327 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-01-06T21:57:44.422 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-01-06T21:57:44.444 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-01-06T21:57:44.549 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-01-06T21:57:44.571 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-01-06T21:57:44.676 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-01-06T21:57:44.698 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-01-06T21:57:44.803 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-01-06T21:57:44.825 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-01-06T21:57:44.931 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-01-06T21:57:44.942 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-01-06T21:57:45.057 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-01-06T21:57:45.069 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-01-06T21:57:45.184 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-01-06T21:57:45.196 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-01-06T21:57:45.302 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-01-06T21:57:45.324 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-01-06T21:57:45.433 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-01-06T21:57:45.455 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-01-06T21:57:45.563 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-01-06T21:57:45.575 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-01-06T21:57:45.680 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-01-06T21:57:45.691 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-01-06T21:57:45.809 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-01-06T21:57:45.820 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-01-06T21:57:45.937 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-01-06T21:57:45.948 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-01-06T21:57:46.065 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-01-06T21:57:46.076 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-01-06T21:57:46.181 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-01-06T21:57:46.193 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-01-06T21:57:46.308 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-01-06T21:57:46.319 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-01-06T21:57:46.438 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-01-06T21:57:46.449 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-01-06T21:57:46.564 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-01-06T21:57:46.567 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-01-06T21:57:46.691 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-01-06T21:57:46.702 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-01-06T21:57:46.817 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-01-06T21:57:46.829 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-01-06T21:57:46.944 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-01-06T21:57:46.955 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-01-06T21:57:47.061 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-01-06T21:57:47.083 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-01-06T21:57:47.189 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-01-06T21:57:47.210 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-01-06T21:57:47.316 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-01-06T21:57:47.338 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-01-06T21:57:47.443 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-01-06T21:57:47.455 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-01-06T21:57:47.569 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-01-06T21:57:47.581 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-01-06T21:57:47.707 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-01-06T21:57:47.708 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-01-06T21:57:47.826 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:47 +0000 (0:00:17.790) 0:00:24.807 ***** 2021-01-06T21:57:47.844 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-01-06T21:57:47.856 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2021-01-06T21:57:47.892 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:47 +0000 (0:00:00.065) 0:00:24.872 ***** 2021-01-06T21:57:47.941 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2021-01-06T21:57:47.943 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora 2021-01-06T21:57:47.944 INFO:teuthology.task.ansible.out:rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-01-06T21:57:47.946 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancic 2021-01-06T21:57:47.948 INFO:teuthology.task.ansible.out:h", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkK 2021-01-06T21:57:47.949 INFO:teuthology.task.ansible.out:ztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 2021-01-06T21:57:47.951 INFO:teuthology.task.ansible.out:HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg 2021-01-06T21:57:47.952 INFO:teuthology.task.ansible.out:e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df 2021-01-06T21:57:47.954 INFO:teuthology.task.ansible.out:79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hm 2021-01-06T21:57:47.955 INFO:teuthology.task.ansible.out:p+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b 2021-01-06T21:57:47.957 INFO:teuthology.task.ansible.out:0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAM 2021-01-06T21:57:47.959 INFO:teuthology.task.ansible.out:Xq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jel 2021-01-06T21:57:47.960 INFO:teuthology.task.ansible.out:opez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf 2021-01-06T21:57:47.962 INFO:teuthology.task.ansible.out:0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4b 2021-01-06T21:57:47.962 INFO:teuthology.task.ansible.out:ea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}]}, "changed": false} 2021-01-06T21:57:48.004 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:48 +0000 (0:00:00.112) 0:00:24.985 ***** 2021-01-06T21:57:48.012 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-01-06T21:57:48.108 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-01-06T21:57:48.236 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-01-06T21:57:48.360 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:48 +0000 (0:00:00.355) 0:00:25.340 ***** 2021-01-06T21:57:48.386 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2021-01-06T21:57:48.425 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:48 +0000 (0:00:00.065) 0:00:25.406 ***** 2021-01-06T21:57:48.465 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2021-01-06T21:57:48.467 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora 2021-01-06T21:57:48.468 INFO:teuthology.task.ansible.out:rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-01-06T21:57:48.470 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancic 2021-01-06T21:57:48.472 INFO:teuthology.task.ansible.out:h", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkK 2021-01-06T21:57:48.473 INFO:teuthology.task.ansible.out:ztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 2021-01-06T21:57:48.475 INFO:teuthology.task.ansible.out:HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg 2021-01-06T21:57:48.477 INFO:teuthology.task.ansible.out:e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6 2021-01-06T21:57:48.478 INFO:teuthology.task.ansible.out:d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df 2021-01-06T21:57:48.480 INFO:teuthology.task.ansible.out:79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hm 2021-01-06T21:57:48.481 INFO:teuthology.task.ansible.out:p+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b 2021-01-06T21:57:48.483 INFO:teuthology.task.ansible.out:0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAM 2021-01-06T21:57:48.484 INFO:teuthology.task.ansible.out:Xq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jel 2021-01-06T21:57:48.486 INFO:teuthology.task.ansible.out:opez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf 2021-01-06T21:57:48.488 INFO:teuthology.task.ansible.out:0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4b 2021-01-06T21:57:48.489 INFO:teuthology.task.ansible.out:ea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}]}, "changed": false} 2021-01-06T21:57:48.529 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:48 +0000 (0:00:00.104) 0:00:25.510 ***** 2021-01-06T21:57:48.824 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* ok: [smithi091.front.sepia.ceph.com -> localhost] => {"after": "0caff10a478f2674721da1965013b225f3ea5800", "attempts": 1, "before": "0caff10a478f2674721da1965013b225f3ea5800", "changed": false, "remote_url_changed": false} 2021-01-06T21:57:48.861 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:48 +0000 (0:00:00.331) 0:00:25.842 ***** 2021-01-06T21:57:49.067 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-01-06T21:57:49.069 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-01-06T21:57:49.100 INFO:teuthology.task.ansible.out: 2021-01-06T21:57:49.102 INFO:teuthology.task.ansible.out:TASK [users : Clone the keys repo] ********************************************* ok: [smithi203.front.sepia.ceph.com -> localhost] => {"after": "0caff10a478f2674721da1965013b225f3ea5800", "attempts": 1, "before": "0caff10a478f2674721da1965013b225f3ea5800", "changed": false, "remote_url_changed": false} 2021-01-06T21:57:49.141 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:57:49 +0000 (0:00:00.279) 0:00:26.121 ***** 2021-01-06T21:57:49.210 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-01-06T21:57:49.212 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-01-06T21:57:49.336 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-01-06T21:57:49.378 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-01-06T21:57:49.462 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2021-01-06T21:57:49.464 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-01-06T21:57:49.515 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-01-06T21:57:49.600 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-01-06T21:57:49.655 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-01-06T21:57:49.728 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2021-01-06T21:57:49.729 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-01-06T21:57:49.780 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2021-01-06T21:57:49.782 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-01-06T21:57:49.843 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-01-06T21:57:49.906 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-01-06T21:57:49.969 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-01-06T21:57:50.031 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2021-01-06T21:57:50.033 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-01-06T21:57:50.094 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-01-06T21:57:50.105 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:57:50.161 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-01-06T21:57:50.223 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2021-01-06T21:57:50.225 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-01-06T21:57:50.287 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-01-06T21:57:50.350 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemer 2021-01-06T21:57:50.352 INFO:teuthology.task.ansible.out:son", "validate_certs": true} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:57:50.361 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:57:50.403 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-01-06T21:57:50.415 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:57:50.488 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-01-06T21:57:50.542 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2021-01-06T21:57:50.544 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-01-06T21:57:50.604 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2021-01-06T21:57:50.606 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-01-06T21:57:50.668 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemer 2021-01-06T21:57:50.670 INFO:teuthology.task.ansible.out:son", "validate_certs": true} 2021-01-06T21:57:50.679 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:57:50.681 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:57:50.733 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-01-06T21:57:50.807 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-01-06T21:57:50.849 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2021-01-06T21:57:50.932 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2021-01-06T21:57:50.934 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-01-06T21:57:50.975 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-01-06T21:57:51.059 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-01-06T21:57:51.101 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-01-06T21:57:51.185 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2021-01-06T21:57:51.218 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-01-06T21:57:51.300 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-01-06T21:57:51.352 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2021-01-06T21:57:51.426 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-01-06T21:57:51.468 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2021-01-06T21:57:51.470 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-01-06T21:57:51.552 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-01-06T21:57:51.595 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2021-01-06T21:57:51.596 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-01-06T21:57:51.678 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2021-01-06T21:57:51.720 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-01-06T21:57:51.804 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2021-01-06T21:57:51.806 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-01-06T21:57:51.836 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-01-06T21:57:51.930 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2021-01-06T21:57:51.932 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-01-06T21:57:51.962 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2021-01-06T21:57:51.964 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-01-06T21:57:52.056 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-01-06T21:57:52.077 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-01-06T21:57:52.171 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-01-06T21:57:52.204 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-01-06T21:57:52.298 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2021-01-06T21:57:52.300 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-01-06T21:57:52.320 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-01-06T21:57:52.415 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-01-06T21:57:52.437 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-01-06T21:57:52.542 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-01-06T21:57:52.564 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-01-06T21:57:52.670 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-01-06T21:57:52.681 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2021-01-06T21:57:52.683 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-01-06T21:57:52.796 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-01-06T21:57:52.807 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-01-06T21:57:52.913 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-01-06T21:57:52.924 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-01-06T21:57:53.038 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2021-01-06T21:57:53.040 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-01-06T21:57:53.049 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-01-06T21:57:53.164 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-01-06T21:57:53.176 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-01-06T21:57:53.281 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-01-06T21:57:53.292 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-01-06T21:57:53.407 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-01-06T21:57:53.408 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-01-06T21:57:53.533 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-01-06T21:57:53.545 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-01-06T21:57:53.659 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-01-06T21:57:53.681 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-01-06T21:57:53.785 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-01-06T21:57:53.817 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-01-06T21:57:53.900 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-01-06T21:57:53.932 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2021-01-06T21:57:53.934 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-01-06T21:57:54.026 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-01-06T21:57:54.048 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2021-01-06T21:57:54.050 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-01-06T21:57:54.155 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-01-06T21:57:54.178 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-01-06T21:57:54.276 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2021-01-06T21:57:54.278 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-01-06T21:57:54.299 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-01-06T21:57:54.405 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2021-01-06T21:57:54.407 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-01-06T21:57:54.428 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-01-06T21:57:54.524 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-01-06T21:57:54.556 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2021-01-06T21:57:54.558 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-01-06T21:57:54.643 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-01-06T21:57:54.697 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2021-01-06T21:57:54.699 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2021-01-06T21:57:54.700 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-01-06T21:57:54.771 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-01-06T21:57:54.814 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-01-06T21:57:54.899 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2021-01-06T21:57:54.901 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-01-06T21:57:54.932 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-01-06T21:57:55.027 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2021-01-06T21:57:55.029 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2021-01-06T21:57:55.030 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-01-06T21:57:55.048 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-01-06T21:57:55.143 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-01-06T21:57:55.164 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-01-06T21:57:55.269 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-01-06T21:57:55.280 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-01-06T21:57:55.405 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2021-01-06T21:57:55.407 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-01-06T21:57:55.531 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-01-06T21:57:55.542 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-01-06T21:57:55.646 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-01-06T21:57:55.657 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-01-06T21:57:55.765 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2021-01-06T21:57:55.767 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2021-01-06T21:57:55.783 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2021-01-06T21:57:55.785 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-01-06T21:57:55.887 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-01-06T21:57:55.908 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-01-06T21:57:56.024 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-01-06T21:57:56.035 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-01-06T21:57:56.150 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2021-01-06T21:57:56.152 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9 2021-01-06T21:57:56.154 INFO:teuthology.task.ansible.out:tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gabrioux", "validate_certs": true} 2021-01-06T21:57:56.267 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-01-06T21:57:56.268 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-01-06T21:57:56.382 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-01-06T21:57:56.394 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-01-06T21:57:56.499 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2021-01-06T21:57:56.500 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-01-06T21:57:56.509 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-01-06T21:57:56.614 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-01-06T21:57:56.625 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-01-06T21:57:56.729 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-01-06T21:57:56.751 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-01-06T21:57:56.855 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-01-06T21:57:56.866 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-01-06T21:57:56.962 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-01-06T21:57:56.984 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-01-06T21:57:57.079 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-01-06T21:57:57.100 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-01-06T21:57:57.204 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-01-06T21:57:57.216 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-01-06T21:57:57.320 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-01-06T21:57:57.342 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-01-06T21:57:57.436 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-01-06T21:57:57.468 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-01-06T21:57:57.551 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-01-06T21:57:57.593 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-01-06T21:57:57.667 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-01-06T21:57:57.709 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-01-06T21:57:57.792 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-01-06T21:57:57.825 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-01-06T21:57:57.908 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-01-06T21:57:57.940 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-01-06T21:57:58.044 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-01-06T21:57:58.065 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-01-06T21:57:58.159 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-01-06T21:57:58.181 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-01-06T21:57:58.285 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-01-06T21:57:58.296 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-01-06T21:57:58.411 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-01-06T21:57:58.413 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-01-06T21:57:58.526 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-01-06T21:57:58.528 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-01-06T21:57:58.642 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-01-06T21:57:58.653 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1 2021-01-06T21:57:58.655 INFO:teuthology.task.ansible.out:lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-01-06T21:57:58.759 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-01-06T21:57:58.769 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-01-06T21:57:58.874 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-01-06T21:57:58.885 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-01-06T21:57:58.990 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1 2021-01-06T21:57:58.992 INFO:teuthology.task.ansible.out:lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-01-06T21:57:59.012 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-01-06T21:57:59.106 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-01-06T21:57:59.127 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-01-06T21:57:59.231 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-01-06T21:57:59.253 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2021-01-06T21:57:59.255 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-01-06T21:57:59.348 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-01-06T21:57:59.380 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2021-01-06T21:57:59.382 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-01-06T21:57:59.463 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-01-06T21:57:59.495 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-01-06T21:57:59.589 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2021-01-06T21:57:59.591 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-01-06T21:57:59.621 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-01-06T21:57:59.716 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2021-01-06T21:57:59.718 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-01-06T21:57:59.758 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-01-06T21:57:59.831 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-01-06T21:57:59.874 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-01-06T21:57:59.947 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-01-06T21:58:00.000 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-01-06T21:58:00.074 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-01-06T21:58:00.116 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-01-06T21:58:00.190 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-01-06T21:58:00.232 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-01-06T21:58:00.317 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-01-06T21:58:00.359 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loc 2021-01-06T21:58:00.361 INFO:teuthology.task.ansible.out:al", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-01-06T21:58:00.434 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-01-06T21:58:00.487 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-01-06T21:58:00.561 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-01-06T21:58:00.614 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-01-06T21:58:00.689 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loc 2021-01-06T21:58:00.691 INFO:teuthology.task.ansible.out:al", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-01-06T21:58:00.731 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-01-06T21:58:00.805 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-01-06T21:58:00.847 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-01-06T21:58:00.931 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-01-06T21:58:00.974 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-01-06T21:58:01.048 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-01-06T21:58:01.090 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-01-06T21:58:01.163 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-01-06T21:58:01.208 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2021-01-06T21:58:01.210 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-01-06T21:58:01.281 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-01-06T21:58:01.334 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-01-06T21:58:01.398 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-01-06T21:58:01.451 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-01-06T21:58:01.524 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2021-01-06T21:58:01.526 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-01-06T21:58:01.577 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-01-06T21:58:01.640 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-01-06T21:58:01.693 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-01-06T21:58:01.767 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-01-06T21:58:01.810 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-01-06T21:58:01.874 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-01-06T21:58:02.497 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueec 2021-01-06T21:58:02.499 INFO:teuthology.task.ansible.out:YY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": 2021-01-06T21:58:02.501 INFO:teuthology.task.ansible.out:"xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90dd 2021-01-06T21:58:02.503 INFO:teuthology.task.ansible.out:c588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UD 2021-01-06T21:58:02.511 INFO:teuthology.task.ansible.out:yAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7T 2021-01-06T21:58:02.514 INFO:teuthology.task.ansible.out:Q68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-01-06T21:58:02.520 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-01-06T21:58:02.605 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-01-06T21:58:02.648 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-01-06T21:58:02.724 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-01-06T21:58:02.767 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-01-06T21:58:02.851 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-01-06T21:58:02.904 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-01-06T21:58:02.977 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-01-06T21:58:03.030 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-01-06T21:58:03.115 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-01-06T21:58:03.148 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-01-06T21:58:03.338 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "valid 2021-01-06T21:58:03.340 INFO:teuthology.task.ansible.out:ate_certs": true} 2021-01-06T21:58:03.349 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-01-06T21:58:03.404 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-01-06T21:58:03.490 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-01-06T21:58:03.585 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjL 2021-01-06T21:58:03.586 INFO:teuthology.task.ansible.out:d+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-01-06T21:58:03.617 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-01-06T21:58:03.659 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-01-06T21:58:03.734 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-01-06T21:58:03.892 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWf 2021-01-06T21:58:03.894 INFO:teuthology.task.ansible.out:Cl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-01-06T21:58:03.904 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-01-06T21:58:03.978 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-01-06T21:58:04.032 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-01-06T21:58:04.096 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-01-06T21:58:04.288 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYA 2021-01-06T21:58:04.290 INFO:teuthology.task.ansible.out:C2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-01-06T21:58:04.340 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-01-06T21:58:04.383 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-01-06T21:58:04.470 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-01-06T21:58:04.513 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-01-06T21:58:04.589 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-01-06T21:58:04.654 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-01-06T21:58:04.719 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-01-06T21:58:04.784 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-01-06T21:58:04.837 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-01-06T21:58:04.902 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-01-06T21:58:04.977 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-01-06T21:58:05.033 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-01-06T21:58:05.108 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-01-06T21:58:05.161 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-01-06T21:58:05.226 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-01-06T21:58:05.290 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-01-06T21:58:05.507 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state 2021-01-06T21:58:05.509 INFO:teuthology.task.ansible.out:": "present", "user": "dpivonka", "validate_certs": true} ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-01-06T21:58:05.545 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-01-06T21:58:05.600 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-01-06T21:58:05.668 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com", "key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-01-06T21:58:05.801 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:05 +0000 (0:00:16.659) 0:00:42.781 ***** 2021-01-06T21:58:05.815 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-01-06T21:58:05.865 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:05.877 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:05.897 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:05.909 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:05.924 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:05.934 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:05.947 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-01-06T21:58:05.951 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:05.965 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:05.976 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.117 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com", "key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-01-06T21:58:06.206 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-01-06T21:58:06.208 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e 2021-01-06T21:58:06.210 INFO:teuthology.task.ansible.out:1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.252 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:06 +0000 (0:00:00.451) 0:00:43.233 ***** 2021-01-06T21:58:06.305 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.319 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.331 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-01-06T21:58:06.343 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.345 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.355 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.366 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.377 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.379 INFO:teuthology.task.ansible.out: 2021-01-06T21:58:06.401 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.495 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-01-06T21:58:06.506 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.508 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.518 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.529 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.540 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.553 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-01-06T21:58:06.556 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.567 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.569 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.579 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.580 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.591 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.602 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.608 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.614 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.625 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.637 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.648 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.659 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.670 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.681 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.683 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.693 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6 2021-01-06T21:58:06.695 INFO:teuthology.task.ansible.out:hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-01-06T21:58:06.705 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.707 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.716 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.727 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.738 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.740 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.750 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.761 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.772 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.774 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.784 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.795 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.806 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.817 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-01-06T21:58:06.818 INFO:teuthology.task.ansible.out: 2021-01-06T21:58:06.829 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.831 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 2021-01-06T21:58:06.832 INFO:teuthology.task.ansible.out: 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.842 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.853 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.855 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.865 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.867 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.878 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.880 INFO:teuthology.task.ansible.out: skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.891 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.895 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.902 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.904 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.914 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.916 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.926 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.928 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.938 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.940 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.952 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.963 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.965 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.976 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:06.988 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.000 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.002 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.012 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.014 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.025 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.037 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.048 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.050 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.061 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.072 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.074 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.085 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.086 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.097 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.099 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.099 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.110 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.112 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.122 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.124 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.135 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.147 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.149 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.160 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.164 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.172 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.174 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.176 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.186 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.187 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.200 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.203 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.212 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.214 INFO:teuthology.task.ansible.out: skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.216 INFO:teuthology.task.ansible.out: skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.228 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.230 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.240 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.252 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.254 INFO:teuthology.task.ansible.out: skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.256 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23 2021-01-06T21:58:07.258 INFO:teuthology.task.ansible.out:F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.265 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.266 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.266 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.267 INFO:teuthology.task.ansible.out: 2021-01-06T21:58:07.278 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.280 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.291 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.302 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.304 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.314 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.325 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.336 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.338 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.348 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.351 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.361 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.363 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.373 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.375 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.386 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.388 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.398 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.399 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.410 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.412 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.422 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.424 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.434 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.436 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.448 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.460 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.472 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.485 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.496 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.498 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.499 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.499 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "ss 2021-01-06T21:58:07.500 INFO:teuthology.task.ansible.out:eshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.511 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.529 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.531 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.566 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:07 +0000 (0:00:01.314) 0:00:44.547 ***** 2021-01-06T21:58:07.574 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.577 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "z 2021-01-06T21:58:07.579 INFO:teuthology.task.ansible.out:dover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.590 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.592 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.592 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.635 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:07 +0000 (0:00:00.068) 0:00:44.616 ***** 2021-01-06T21:58:07.643 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-01-06T21:58:07.646 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.648 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.649 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thi 2021-01-06T21:58:07.652 INFO:teuthology.task.ansible.out:nkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.653 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.654 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.666 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.678 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.681 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.692 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.693 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.703 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.705 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.714 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.725 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.727 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.727 INFO:teuthology.task.ansible.out: 2021-01-06T21:58:07.739 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.750 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.761 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.776 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.783 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.794 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-06T21:58:07.796 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.806 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.817 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.819 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.829 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.840 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.851 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.873 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.874 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.884 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:07.937 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:07 +0000 (0:00:00.302) 0:00:44.918 ***** 2021-01-06T21:58:07.945 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-06T21:58:07.946 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-01-06T21:58:07.968 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:08.014 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:08 +0000 (0:00:00.076) 0:00:44.995 ***** 2021-01-06T21:58:08.045 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-06T21:58:08.047 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-01-06T21:58:08.161 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-01-06T21:58:08.174 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-01-06T21:58:08.280 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-01-06T21:58:08.292 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-01-06T21:58:08.396 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-01-06T21:58:08.408 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-01-06T21:58:08.523 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-01-06T21:58:08.534 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-01-06T21:58:08.641 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-01-06T21:58:08.652 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-01-06T21:58:08.759 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-01-06T21:58:08.771 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-01-06T21:58:08.889 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-01-06T21:58:08.901 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-01-06T21:58:08.997 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-01-06T21:58:09.020 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-01-06T21:58:09.116 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-01-06T21:58:09.138 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-01-06T21:58:09.243 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-01-06T21:58:09.254 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-01-06T21:58:09.359 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-01-06T21:58:09.381 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-01-06T21:58:09.476 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-01-06T21:58:09.499 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-01-06T21:58:09.603 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-01-06T21:58:09.626 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-01-06T21:58:09.720 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-01-06T21:58:09.742 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-01-06T21:58:09.838 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-01-06T21:58:09.860 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-01-06T21:58:09.955 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-01-06T21:58:09.987 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-01-06T21:58:10.074 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-01-06T21:58:10.104 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-01-06T21:58:10.199 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-01-06T21:58:10.221 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-01-06T21:58:10.316 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-01-06T21:58:10.347 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-01-06T21:58:10.443 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-01-06T21:58:10.475 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-01-06T21:58:10.559 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-01-06T21:58:10.591 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-01-06T21:58:10.674 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-01-06T21:58:10.717 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-01-06T21:58:10.791 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-01-06T21:58:10.845 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-01-06T21:58:10.918 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-01-06T21:58:10.960 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-01-06T21:58:11.033 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-01-06T21:58:11.085 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-01-06T21:58:11.149 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-01-06T21:58:11.201 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-01-06T21:58:11.275 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-01-06T21:58:11.317 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-01-06T21:58:11.391 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-01-06T21:58:11.443 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-01-06T21:58:11.517 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-01-06T21:58:11.560 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-01-06T21:58:11.634 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-01-06T21:58:11.677 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-01-06T21:58:11.761 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-01-06T21:58:11.804 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-01-06T21:58:11.877 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-01-06T21:58:11.919 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-01-06T21:58:11.993 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-01-06T21:58:12.046 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-01-06T21:58:12.121 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-01-06T21:58:12.163 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-01-06T21:58:12.236 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-01-06T21:58:12.279 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-01-06T21:58:12.352 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-01-06T21:58:12.406 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-01-06T21:58:12.480 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-01-06T21:58:12.522 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-01-06T21:58:12.596 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-01-06T21:58:12.649 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-01-06T21:58:12.723 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-01-06T21:58:12.775 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-01-06T21:58:12.839 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-01-06T21:58:12.902 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-01-06T21:58:12.965 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-01-06T21:58:13.018 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-01-06T21:58:13.080 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-01-06T21:58:13.143 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-01-06T21:58:13.206 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-01-06T21:58:13.259 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-01-06T21:58:13.322 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-01-06T21:58:13.375 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-01-06T21:58:13.448 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-01-06T21:58:13.506 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:13 +0000 (0:00:05.491) 0:00:50.487 ***** 2021-01-06T21:58:13.513 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-01-06T21:58:13.608 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-01-06T21:58:13.734 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-01-06T21:58:13.840 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-01-06T21:58:13.841 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "checksum": "e880c991f1562524752e643d227b8575ac6e2ed1", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "fd5444fc24d885aa1caaed5c7ef528f8", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1609970293.5264316-67597034584281/source", "state": "file", "uid": 0} 2021-01-06T21:58:13.876 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:13 +0000 (0:00:00.370) 0:00:50.857 ***** 2021-01-06T21:58:13.883 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-06T21:58:13.885 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-01-06T21:58:13.939 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:13 +0000 (0:00:00.062) 0:00:50.920 ***** 2021-01-06T21:58:13.944 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-01-06T21:58:13.948 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => { "msg": "Host smithi091.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2021-01-06T21:58:13.985 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:13 +0000 (0:00:00.046) 0:00:50.966 ***** 2021-01-06T21:58:14.022 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-01-06T21:58:14.023 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-01-06T21:58:14.060 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.074) 0:00:51.040 ***** 2021-01-06T21:58:14.086 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-01-06T21:58:14.088 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:14.122 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.062) 0:00:51.103 ***** 2021-01-06T21:58:14.149 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:14.183 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.061) 0:00:51.164 ***** 2021-01-06T21:58:14.210 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-01-06T21:58:14.212 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:14.246 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.063) 0:00:51.227 ***** 2021-01-06T21:58:14.276 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-01-06T21:58:14.278 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:14.318 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.071) 0:00:51.299 ***** 2021-01-06T21:58:14.327 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-01-06T21:58:14.328 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "checksum": "e880c991f1562524752e643d227b8575ac6e2ed1", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "fd5444fc24d885aa1caaed5c7ef528f8", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1609970293.9610648-121246044513382/source", "state": "file", "uid": 0} 2021-01-06T21:58:14.375 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.056) 0:00:51.356 ***** 2021-01-06T21:58:14.383 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-01-06T21:58:14.385 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:14.425 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.050) 0:00:51.406 ***** 2021-01-06T21:58:14.434 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-01-06T21:58:14.436 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => { "msg": "Host smithi203.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2021-01-06T21:58:14.478 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.051) 0:00:51.458 ***** 2021-01-06T21:58:14.485 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:14.522 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.043) 0:00:51.502 ***** 2021-01-06T21:58:14.527 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-01-06T21:58:14.529 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-01-06T21:58:14.564 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.042) 0:00:51.545 ***** 2021-01-06T21:58:14.571 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-01-06T21:58:14.573 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:14.607 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.042) 0:00:51.588 ***** 2021-01-06T21:58:14.620 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:14.654 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.046) 0:00:51.634 ***** 2021-01-06T21:58:14.659 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-01-06T21:58:14.661 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:14.696 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.042) 0:00:51.677 ***** 2021-01-06T21:58:14.704 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-01-06T21:58:14.706 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:14.768 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.071) 0:00:51.749 ***** 2021-01-06T21:58:14.778 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-01-06T21:58:14.780 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi091.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-06T21:58:14.816 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.047) 0:00:51.797 ***** 2021-01-06T21:58:14.822 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-01-06T21:58:14.824 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:14.859 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.042) 0:00:51.840 ***** 2021-01-06T21:58:14.865 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-01-06T21:58:14.866 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:14.902 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.043) 0:00:51.883 ***** 2021-01-06T21:58:14.908 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-01-06T21:58:14.910 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:14.945 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.042) 0:00:51.926 ***** 2021-01-06T21:58:14.951 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-01-06T21:58:14.953 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:14.986 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:14 +0000 (0:00:00.041) 0:00:51.967 ***** 2021-01-06T21:58:14.991 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-01-06T21:58:14.993 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:15.029 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.042) 0:00:52.009 ***** 2021-01-06T21:58:15.035 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-01-06T21:58:15.037 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-06T21:58:15.071 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.042) 0:00:52.052 ***** 2021-01-06T21:58:15.077 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-01-06T21:58:15.079 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:15.113 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.041) 0:00:52.094 ***** 2021-01-06T21:58:15.120 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-01-06T21:58:15.122 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:15.155 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.042) 0:00:52.136 ***** 2021-01-06T21:58:15.161 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-01-06T21:58:15.164 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:15.198 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.042) 0:00:52.179 ***** 2021-01-06T21:58:15.203 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-01-06T21:58:15.205 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-06T21:58:15.240 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.042) 0:00:52.221 ***** 2021-01-06T21:58:15.246 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-01-06T21:58:15.249 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:15.286 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.045) 0:00:52.267 ***** 2021-01-06T21:58:15.291 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-01-06T21:58:15.293 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:15.328 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.042) 0:00:52.309 ***** 2021-01-06T21:58:15.333 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-01-06T21:58:15.335 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi203.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-06T21:58:15.369 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.041) 0:00:52.350 ***** 2021-01-06T21:58:15.375 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-01-06T21:58:15.377 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:15.413 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.043) 0:00:52.394 ***** 2021-01-06T21:58:15.419 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-01-06T21:58:15.421 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:15.460 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.046) 0:00:52.440 ***** 2021-01-06T21:58:15.466 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-01-06T21:58:15.468 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:15.510 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.050) 0:00:52.490 ***** 2021-01-06T21:58:15.516 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-01-06T21:58:15.518 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:15.555 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.045) 0:00:52.536 ***** 2021-01-06T21:58:15.564 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-01-06T21:58:15.566 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:15.610 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.055) 0:00:52.591 ***** 2021-01-06T21:58:15.620 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-01-06T21:58:15.622 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-06T21:58:15.666 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.055) 0:00:52.647 ***** 2021-01-06T21:58:15.672 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-01-06T21:58:15.674 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-06T21:58:15.709 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.042) 0:00:52.689 ***** 2021-01-06T21:58:15.717 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:15.752 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.044) 0:00:52.733 ***** 2021-01-06T21:58:15.758 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-01-06T21:58:15.759 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:15.797 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.044) 0:00:52.777 ***** 2021-01-06T21:58:15.803 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-01-06T21:58:15.805 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-06T21:58:15.840 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.043) 0:00:52.821 ***** 2021-01-06T21:58:15.846 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-01-06T21:58:15.848 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:15.882 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.041) 0:00:52.863 ***** 2021-01-06T21:58:15.888 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-01-06T21:58:15.890 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:15.927 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.044) 0:00:52.908 ***** 2021-01-06T21:58:15.933 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-01-06T21:58:15.935 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:15.969 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:15 +0000 (0:00:00.042) 0:00:52.950 ***** 2021-01-06T21:58:15.976 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-01-06T21:58:15.978 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.012 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.042) 0:00:52.993 ***** 2021-01-06T21:58:16.018 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.055 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.042) 0:00:53.035 ***** 2021-01-06T21:58:16.061 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-01-06T21:58:16.063 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.099 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.043) 0:00:53.079 ***** 2021-01-06T21:58:16.104 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-01-06T21:58:16.106 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.141 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.042) 0:00:53.122 ***** 2021-01-06T21:58:16.147 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-01-06T21:58:16.149 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.186 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.042) 0:00:53.164 ***** 2021-01-06T21:58:16.190 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.227 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.043) 0:00:53.208 ***** 2021-01-06T21:58:16.234 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-01-06T21:58:16.235 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-06T21:58:16.272 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.045) 0:00:53.253 ***** 2021-01-06T21:58:16.279 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-01-06T21:58:16.281 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-06T21:58:16.316 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.043) 0:00:53.297 ***** 2021-01-06T21:58:16.324 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.359 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.043) 0:00:53.340 ***** 2021-01-06T21:58:16.366 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-01-06T21:58:16.367 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.403 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.043) 0:00:53.384 ***** 2021-01-06T21:58:16.410 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-01-06T21:58:16.413 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.448 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.044) 0:00:53.428 ***** 2021-01-06T21:58:16.454 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.491 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.043) 0:00:53.472 ***** 2021-01-06T21:58:16.497 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-01-06T21:58:16.499 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.539 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.047) 0:00:53.519 ***** 2021-01-06T21:58:16.545 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-01-06T21:58:16.547 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.599 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.060) 0:00:53.580 ***** 2021-01-06T21:58:16.608 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-01-06T21:58:16.610 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.646 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.046) 0:00:53.627 ***** 2021-01-06T21:58:16.652 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.654 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.689 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.042) 0:00:53.670 ***** 2021-01-06T21:58:16.695 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-01-06T21:58:16.697 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.733 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.044) 0:00:53.714 ***** 2021-01-06T21:58:16.738 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-01-06T21:58:16.741 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.777 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.043) 0:00:53.757 ***** 2021-01-06T21:58:16.783 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-01-06T21:58:16.784 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.819 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.042) 0:00:53.800 ***** 2021-01-06T21:58:16.826 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-01-06T21:58:16.827 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.863 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.043) 0:00:53.844 ***** 2021-01-06T21:58:16.870 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-01-06T21:58:16.873 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-06T21:58:16.907 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.044) 0:00:53.888 ***** 2021-01-06T21:58:16.934 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-01-06T21:58:16.936 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:16.971 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:16 +0000 (0:00:00.063) 0:00:53.952 ***** 2021-01-06T21:58:16.999 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-01-06T21:58:17.001 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:17.037 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.065) 0:00:54.017 ***** 2021-01-06T21:58:17.044 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-01-06T21:58:17.046 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.014535", "end": "2021-01-06 21:58:16.988267", "rc": 0, "start": "2021-01-06 21:58:16.973732", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-01-06T21:58:17.083 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.045) 0:00:54.063 ***** 2021-01-06T21:58:17.091 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-01-06T21:58:17.093 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:17.128 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.046) 0:00:54.109 ***** 2021-01-06T21:58:17.136 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:17.175 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.046) 0:00:54.156 ***** 2021-01-06T21:58:17.182 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-01-06T21:58:17.184 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:17.226 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.050) 0:00:54.207 ***** 2021-01-06T21:58:17.233 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-01-06T21:58:17.235 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:17.274 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.047) 0:00:54.255 ***** 2021-01-06T21:58:17.281 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-01-06T21:58:17.283 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-01-06T21:58:17.321 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.046) 0:00:54.301 ***** 2021-01-06T21:58:17.328 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-01-06T21:58:17.330 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:17.367 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.045) 0:00:54.347 ***** 2021-01-06T21:58:17.374 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-01-06T21:58:17.376 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:17.412 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.045) 0:00:54.392 ***** 2021-01-06T21:58:17.419 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-01-06T21:58:17.421 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:17.458 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.046) 0:00:54.439 ***** 2021-01-06T21:58:17.486 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-01-06T21:58:17.488 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:17.523 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.064) 0:00:54.504 ***** 2021-01-06T21:58:17.550 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-01-06T21:58:17.552 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone] *************************************** ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.017827", "end": "2021-01-06 21:58:17.543059", "rc": 0, "start": "2021-01-06 21:58:17.525232", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-01-06T21:58:17.586 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.063) 0:00:54.567 ***** 2021-01-06T21:58:17.620 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.033) 0:00:54.601 ***** 2021-01-06T21:58:17.627 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:17.663 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.042) 0:00:54.644 ***** 2021-01-06T21:58:17.670 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-01-06T21:58:17.672 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:17.707 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.044) 0:00:54.688 ***** 2021-01-06T21:58:17.734 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-01-06T21:58:17.736 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:17.769 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.062) 0:00:54.750 ***** 2021-01-06T21:58:17.796 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-01-06T21:58:17.798 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:17.831 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.061) 0:00:54.812 ***** 2021-01-06T21:58:17.858 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-01-06T21:58:17.859 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:17.893 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.061) 0:00:54.874 ***** 2021-01-06T21:58:17.920 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:17.959 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:17 +0000 (0:00:00.065) 0:00:54.939 ***** 2021-01-06T21:58:17.985 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-01-06T21:58:17.987 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:18.020 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:18 +0000 (0:00:00.061) 0:00:55.001 ***** 2021-01-06T21:58:18.047 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-01-06T21:58:18.048 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:18.085 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:18 +0000 (0:00:00.064) 0:00:55.066 ***** 2021-01-06T21:58:18.517 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-01-06T21:58:18.519 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.306525", "end": "2021-01-06 21:58:18.500443", "rc": 0, "start": "2021-01-06 21:58:18.193918", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.734494", "end": "2021-01-06 21:58:18.500369", "rc": 0, "start": "2021-01-06 21:58:17.765875", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:58:18.553 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:18 +0000 (0:00:00.468) 0:00:55.534 ***** 2021-01-06T21:58:18.585 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:18 +0000 (0:00:00.031) 0:00:55.566 ***** 2021-01-06T21:58:18.590 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-01-06T21:58:18.592 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:18.625 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:18 +0000 (0:00:00.040) 0:00:55.606 ***** 2021-01-06T21:58:18.634 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:18.671 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:18 +0000 (0:00:00.045) 0:00:55.652 ***** 2021-01-06T21:58:18.813 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-01-06T21:58:18.814 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.055192", "end": "2021-01-06 21:58:18.791953", "rc": 0, "start": "2021-01-06 21:58:18.736761", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:58:18.853 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:18 +0000 (0:00:00.181) 0:00:55.834 ***** 2021-01-06T21:58:18.860 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.050268", "end": "2021-01-06 21:58:18.832879", "rc": 0, "start": "2021-01-06 21:58:18.782611", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:58:18.899 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:18 +0000 (0:00:00.045) 0:00:55.879 ***** 2021-01-06T21:58:22.827 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-01-06T21:58:22.829 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1609970019, "cache_updated": false, "changed": false} 2021-01-06T21:58:22.862 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:22 +0000 (0:00:03.963) 0:00:59.843 ***** 2021-01-06T21:58:22.993 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1609970065, "cache_updated": false, "changed": false} 2021-01-06T21:58:23.029 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:23 +0000 (0:00:00.166) 0:01:00.010 ***** 2021-01-06T21:58:23.700 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-01-06T21:58:23.702 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"cache_update_time": 1609970019, "cache_updated": false, "changed": false} 2021-01-06T21:58:23.741 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:23 +0000 (0:00:00.711) 0:01:00.722 ***** 2021-01-06T21:58:23.874 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"cache_update_time": 1609970065, "cache_updated": false, "changed": false} 2021-01-06T21:58:24.002 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:23 +0000 (0:00:00.260) 0:01:00.983 ***** 2021-01-06T21:58:24.008 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-01-06T21:58:24.010 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:24.045 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:24 +0000 (0:00:00.042) 0:01:01.026 ***** 2021-01-06T21:58:24.051 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:24.089 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:24 +0000 (0:00:00.043) 0:01:01.069 ***** 2021-01-06T21:58:24.546 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-01-06T21:58:24.549 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-01-06T21:58:24.583 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:24 +0000 (0:00:00.494) 0:01:01.564 ***** 2021-01-06T21:58:24.590 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-01-06T21:58:24.627 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:24 +0000 (0:00:00.044) 0:01:01.608 ***** 2021-01-06T21:58:25.392 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-01-06T21:58:25.394 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"cache_update_time": 1609970065, "cache_updated": false, "changed": false} 2021-01-06T21:58:25.430 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:25 +0000 (0:00:00.803) 0:01:02.411 ***** 2021-01-06T21:58:25.468 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"cache_update_time": 1609970019, "cache_updated": false, "changed": false} 2021-01-06T21:58:25.506 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:25 +0000 (0:00:00.075) 0:01:02.487 ***** 2021-01-06T21:58:25.858 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-01-06T21:58:25.860 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-01-06T21:58:25.902 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-01-06T21:58:26.143 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-01-06T21:58:26.179 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-01-06T21:58:26.401 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-01-06T21:58:26.439 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:26 +0000 (0:00:00.932) 0:01:03.420 ***** 2021-01-06T21:58:26.456 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-01-06T21:58:26.498 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:26 +0000 (0:00:00.058) 0:01:03.478 ***** 2021-01-06T21:58:26.556 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-01-06T21:58:26.558 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-01-06T21:58:26.596 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:26 +0000 (0:00:00.098) 0:01:03.577 ***** 2021-01-06T21:58:26.624 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-01-06T21:58:26.661 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:26 +0000 (0:00:00.064) 0:01:03.642 ***** 2021-01-06T21:58:26.887 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-01-06T21:58:26.889 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-01-06T21:58:26.957 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-01-06T21:58:27.147 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-01-06T21:58:27.425 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} ok: [smithi091.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-01-06T21:58:27.435 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:27 +0000 (0:00:00.773) 0:01:04.416 ***** 2021-01-06T21:58:27.463 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-01-06T21:58:27.465 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-01-06T21:58:27.506 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:27 +0000 (0:00:00.070) 0:01:04.486 ***** 2021-01-06T21:58:27.512 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-01-06T21:58:27.514 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-01-06T21:58:27.555 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:27 +0000 (0:00:00.049) 0:01:04.536 ***** 2021-01-06T21:58:27.563 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:27.602 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:27 +0000 (0:00:00.046) 0:01:04.583 ***** 2021-01-06T21:58:27.609 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-01-06T21:58:27.611 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-01-06T21:58:27.646 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:27 +0000 (0:00:00.044) 0:01:04.627 ***** 2021-01-06T21:58:27.653 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* skipping: [smithi091.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-06T21:58:27.692 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:27 +0000 (0:00:00.045) 0:01:04.672 ***** 2021-01-06T21:58:27.769 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Wednesday 06 January 2021 21:58:27 +0000 (0:00:00.044) 0:01:04.717 ***** TASK [common : Install nrpe without epel] ************************************** skipping: [smithi091.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} TASK [common : Check for epel] ************************************************* skipping: [smithi203.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Install nrpe without epel] ************************************** skipping: [smithi091.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http: 2021-01-06T21:58:27.771 INFO:teuthology.task.ansible.out://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-01-06T21:58:27.800 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:27 +0000 (0:00:00.064) 0:01:04.781 ***** 2021-01-06T21:58:27.833 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:27 +0000 (0:00:00.033) 0:01:04.814 ***** 2021-01-06T21:58:27.840 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-01-06T21:58:27.842 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:27.877 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:27 +0000 (0:00:00.043) 0:01:04.858 ***** 2021-01-06T21:58:27.887 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-01-06T21:58:27.889 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-01-06T21:58:27.890 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-01-06T21:58:27.901 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-01-06T21:58:27.937 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:27 +0000 (0:00:00.059) 0:01:04.918 ***** 2021-01-06T21:58:27.963 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-01-06T21:58:27.965 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:27.999 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:27 +0000 (0:00:00.062) 0:01:04.980 ***** 2021-01-06T21:58:28.775 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2021-01-06T21:58:28.777 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"cache_update_time": 1609970065, "cache_updated": false, "changed": false} 2021-01-06T21:58:28.815 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:28 +0000 (0:00:00.815) 0:01:05.796 ***** 2021-01-06T21:58:28.916 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"cache_update_time": 1609970019, "cache_updated": false, "changed": false} 2021-01-06T21:58:28.958 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:28 +0000 (0:00:00.141) 0:01:05.938 ***** 2021-01-06T21:58:29.121 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-01-06T21:58:29.123 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-01-06T21:58:29.157 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:29 +0000 (0:00:00.200) 0:01:06.138 ***** 2021-01-06T21:58:29.258 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-01-06T21:58:29.300 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:29 +0000 (0:00:00.142) 0:01:06.281 ***** 2021-01-06T21:58:29.540 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-01-06T21:58:29.540 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-01-06T21:58:29.580 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:29 +0000 (0:00:00.279) 0:01:06.561 ***** 2021-01-06T21:58:29.608 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-01-06T21:58:29.658 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:29 +0000 (0:00:00.078) 0:01:06.639 ***** 2021-01-06T21:58:29.718 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-01-06T21:58:29.720 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-01-06T21:58:29.794 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-01-06T21:58:29.816 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-01-06T21:58:29.860 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:29 +0000 (0:00:00.201) 0:01:06.841 ***** 2021-01-06T21:58:29.887 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-01-06T21:58:29.889 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:29.926 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:29 +0000 (0:00:00.066) 0:01:06.907 ***** 2021-01-06T21:58:29.932 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-01-06T21:58:29.934 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-01-06T21:58:29.973 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:29 +0000 (0:00:00.047) 0:01:06.954 ***** 2021-01-06T21:58:29.979 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-01-06T21:58:29.981 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-06T21:58:30.019 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:30 +0000 (0:00:00.045) 0:01:07.000 ***** 2021-01-06T21:58:30.027 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-01-06T21:58:30.028 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:30.068 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:30 +0000 (0:00:00.048) 0:01:07.049 ***** 2021-01-06T21:58:30.077 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-01-06T21:58:30.078 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:30.114 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:30 +0000 (0:00:00.046) 0:01:07.095 ***** 2021-01-06T21:58:30.503 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** skipping: [smithi203.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} Wednesday 06 January 2021 21:58:30 +0000 (0:00:00.043) 0:01:07.139 ***** TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Wednesday 06 January 2021 21:58:30 +0000 (0:00:00.061) 0:01:07.201 ***** TASK [common : Upload nagios nrpe config.] ************************************* ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} Wednesday 06 January 2021 21:58:30 +0000 (0:00:00.241) 0:01:07.442 ***** 2021-01-06T21:58:30.538 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-01-06T21:58:30.574 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:30 +0000 (0:00:00.112) 0:01:07.555 ***** 2021-01-06T21:58:31.105 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-01-06T21:58:31.106 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Wed 2021-01-06 21:57:03 UTC", "ActiveEnterTimestampMonotonic": "20809321", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target systemd-journald.socket basic.target remote-fs.target sysinit.target var-run.mount system.slice local-fs.target systemd-tmpfiles-setup.service time-sync.target -.mount nss-lookup.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Wed 2021-01-06 21:57:03 UTC", "AssertTimestampMonotonic": "20807947", "Before": "getty@tty1.service multi-user.target plymouth-quit.service xdm.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Wed 2021-01-06 21:57:03 UTC", "ConditionTimestampMonotonic": "20807946", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2021-01-06T21:58:31.108 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1273", "ExecMainStartTimestamp": "Wed 2021-01-06 21:57:03 UTC", "ExecMainStartTimestampMonotonic": "20809250", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Wed 2021-01-06 21:57:03 UTC] ; stop_time=[n/a] ; pid=1273 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Wed 2021-01-06 21:57:03 UTC", "InactiveExitTimestampMonotonic": "20809321", "InvocationID": "dd906bc93a6d4fec9e01157fc1793b02", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2021-01-06T21:58:31.109 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128028", "LimitNPROCSoft": "128028", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1273", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2021-01-06T21:58:31.111 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Wed 2021-01-06 21:57:03 UTC", "StateChangeTimestampMonotonic": "20809321", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Wed 2021-01-06 21:57:03 UTC", "WatchdogTimestampMonotonic": "20809319", "WatchdogUSec": "0"}} ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Wed 2021-01-06 21:56:58 UTC", "ActiveEnterTimestampMonotonic": "19049630", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target systemd-journald.socket system.slice network.target local-fs.target remote-fs.target basic.target systemd-tmpfiles-setup.service 2021-01-06T21:58:31.112 INFO:teuthology.task.ansible.out: -.mount time-sync.target var-run.mount nss-lookup.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Wed 2021-01-06 21:56:58 UTC", "AssertTimestampMonotonic": "19048758", "Before": "multi-user.target getty@tty1.service xdm.service plymouth-quit.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Wed 2021-01-06 21:56:58 UTC", "ConditionTimestampMonotonic": "19048757", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1269", "ExecMainStartTimestamp": "Wed 2021-01-06 21:56:58 UTC", "ExecMai 2021-01-06T21:58:31.114 INFO:teuthology.task.ansible.out:nStartTimestampMonotonic": "19049600", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Wed 2021-01-06 21:56:58 UTC] ; stop_time=[n/a] ; pid=1269 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Wed 2021-01-06 21:56:58 UTC", "InactiveExitTimestampMonotonic": "19049630", "InvocationID": "48a8e59118924779ad3c2e684bbf30d6", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft 2021-01-06T21:58:31.116 INFO:teuthology.task.ansible.out:": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128028", "LimitNPROCSoft": "128028", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1269", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "St 2021-01-06T21:58:31.117 INFO:teuthology.task.ansible.out:artLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Wed 2021-01-06 21:56:58 UTC", "StateChangeTimestampMonotonic": "19049630", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Wed 2021-01-06 21:56:58 UTC", "WatchdogTimestampMonotonic": "19049629", "WatchdogUSec": "0"}} 2021-01-06T21:58:31.146 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.572) 0:01:08.127 ***** 2021-01-06T21:58:31.182 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.035) 0:01:08.163 ***** 2021-01-06T21:58:31.187 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-01-06T21:58:31.189 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.226 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.042) 0:01:08.205 ***** 2021-01-06T21:58:31.231 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.267 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.043) 0:01:08.248 ***** 2021-01-06T21:58:31.272 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-01-06T21:58:31.274 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.310 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.043) 0:01:08.291 ***** 2021-01-06T21:58:31.317 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.359 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.048) 0:01:08.340 ***** 2021-01-06T21:58:31.416 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.043) 0:01:08.384 ***** skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.444 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.041) 0:01:08.425 ***** 2021-01-06T21:58:31.450 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-01-06T21:58:31.452 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.486 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.041) 0:01:08.467 ***** 2021-01-06T21:58:31.492 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.527 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.040) 0:01:08.508 ***** 2021-01-06T21:58:31.533 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-01-06T21:58:31.535 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.570 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.043) 0:01:08.551 ***** 2021-01-06T21:58:31.577 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.616 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.045) 0:01:08.597 ***** 2021-01-06T21:58:31.622 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-01-06T21:58:31.624 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.659 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.042) 0:01:08.640 ***** 2021-01-06T21:58:31.666 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.708 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.048) 0:01:08.688 ***** 2021-01-06T21:58:31.716 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.766 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.059) 0:01:08.747 ***** 2021-01-06T21:58:31.775 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.814 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.047) 0:01:08.794 ***** 2021-01-06T21:58:31.820 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.856 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.042) 0:01:08.837 ***** 2021-01-06T21:58:31.862 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.898 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.042) 0:01:08.879 ***** 2021-01-06T21:58:31.904 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-01-06T21:58:31.906 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.942 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.043) 0:01:08.922 ***** 2021-01-06T21:58:31.948 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:31.989 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:31 +0000 (0:00:00.047) 0:01:08.970 ***** 2021-01-06T21:58:31.991 INFO:teuthology.task.ansible.out: 2021-01-06T21:58:31.997 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-01-06T21:58:31.999 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:32.034 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.044) 0:01:09.015 ***** 2021-01-06T21:58:32.040 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:32.080 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.044) 0:01:09.059 ***** 2021-01-06T21:58:32.084 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-01-06T21:58:32.086 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-01-06T21:58:32.122 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.043) 0:01:09.103 ***** 2021-01-06T21:58:32.128 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-01-06T21:58:32.166 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.044) 0:01:09.147 ***** 2021-01-06T21:58:32.172 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-01-06T21:58:32.174 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-01-06T21:58:32.211 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.045) 0:01:09.192 ***** 2021-01-06T21:58:32.219 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-01-06T21:58:32.262 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.050) 0:01:09.243 ***** 2021-01-06T21:58:32.268 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-01-06T21:58:32.270 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-01-06T21:58:32.308 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.046) 0:01:09.289 ***** 2021-01-06T21:58:32.316 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-01-06T21:58:32.355 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.046) 0:01:09.336 ***** 2021-01-06T21:58:32.361 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-01-06T21:58:32.364 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-01-06T21:58:32.399 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.044) 0:01:09.380 ***** 2021-01-06T21:58:32.406 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-01-06T21:58:32.453 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.053) 0:01:09.433 ***** 2021-01-06T21:58:32.527 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-01-06T21:58:32.529 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-01-06T21:58:32.566 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.113) 0:01:09.547 ***** 2021-01-06T21:58:32.584 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-01-06T21:58:32.620 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.053) 0:01:09.601 ***** 2021-01-06T21:58:32.704 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-01-06T21:58:32.742 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.122) 0:01:09.723 ***** 2021-01-06T21:58:32.760 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-01-06T21:58:32.805 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.062) 0:01:09.786 ***** 2021-01-06T21:58:32.811 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-01-06T21:58:32.813 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:32.860 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.055) 0:01:09.841 ***** 2021-01-06T21:58:32.869 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:32.922 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.061) 0:01:09.902 ***** 2021-01-06T21:58:32.928 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-01-06T21:58:32.930 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:32.968 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:32 +0000 (0:00:00.046) 0:01:09.949 ***** 2021-01-06T21:58:32.974 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:33.011 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:33 +0000 (0:00:00.043) 0:01:09.992 ***** 2021-01-06T21:58:33.130 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-01-06T21:58:33.132 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-01-06T21:58:33.172 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:33 +0000 (0:00:00.160) 0:01:10.152 ***** 2021-01-06T21:58:33.178 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-01-06T21:58:33.215 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:33 +0000 (0:00:00.043) 0:01:10.196 ***** 2021-01-06T21:58:33.572 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-01-06T21:58:33.574 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-01-06T21:58:33.623 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:33 +0000 (0:00:00.407) 0:01:10.604 ***** 2021-01-06T21:58:33.663 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-01-06T21:58:33.666 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi091"}, "changed": false} TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-01-06T21:58:33.702 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:33 +0000 (0:00:00.078) 0:01:10.682 ***** 2021-01-06T21:58:33.735 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:33 +0000 (0:00:00.032) 0:01:10.715 ***** 2021-01-06T21:58:33.762 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-01-06T21:58:33.764 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi203"}, "changed": false} 2021-01-06T21:58:33.798 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:33 +0000 (0:00:00.063) 0:01:10.779 ***** 2021-01-06T21:58:34.579 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-01-06T21:58:34.581 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi203.front.sepia.ceph.com", "ansible_hostname": "smithi203", "ansible_nodename": "smithi203"}, "changed": true, "name": "smithi203"} 2021-01-06T21:58:34.614 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:34 +0000 (0:00:00.816) 0:01:11.595 ***** 2021-01-06T21:58:34.631 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi091.front.sepia.ceph.com", "ansible_hostname": "smithi091", "ansible_nodename": "smithi091"}, "changed": true, "name": "smithi091"} 2021-01-06T21:58:34.668 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:34 +0000 (0:00:00.054) 0:01:11.649 ***** 2021-01-06T21:58:34.925 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-01-06T21:58:34.927 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-01-06T21:58:34.962 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:34 +0000 (0:00:00.293) 0:01:11.943 ***** 2021-01-06T21:58:34.968 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-01-06T21:58:35.005 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:35 +0000 (0:00:00.042) 0:01:11.985 ***** 2021-01-06T21:58:35.286 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-01-06T21:58:35.288 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-01-06T21:58:35.325 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:35 +0000 (0:00:00.320) 0:01:12.306 ***** 2021-01-06T21:58:35.332 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-01-06T21:58:35.368 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:35 +0000 (0:00:00.043) 0:01:12.349 ***** 2021-01-06T21:58:35.897 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-01-06T21:58:35.899 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-01-06T21:58:35.949 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:35 +0000 (0:00:00.581) 0:01:12.930 ***** 2021-01-06T21:58:35.956 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-01-06T21:58:35.994 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:35 +0000 (0:00:00.044) 0:01:12.975 ***** 2021-01-06T21:58:35.999 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-01-06T21:58:36.001 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:36.035 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.041) 0:01:13.016 ***** 2021-01-06T21:58:36.041 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:36.080 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.045) 0:01:13.061 ***** 2021-01-06T21:58:36.086 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-01-06T21:58:36.088 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:36.123 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.042) 0:01:13.104 ***** 2021-01-06T21:58:36.129 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:36.165 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.041) 0:01:13.146 ***** 2021-01-06T21:58:36.170 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-01-06T21:58:36.172 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:36.209 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.044) 0:01:13.190 ***** 2021-01-06T21:58:36.217 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:36.255 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.045) 0:01:13.236 ***** 2021-01-06T21:58:36.262 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-01-06T21:58:36.264 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:36.297 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.042) 0:01:13.278 ***** 2021-01-06T21:58:36.304 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:36.345 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.047) 0:01:13.326 ***** 2021-01-06T21:58:36.351 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-01-06T21:58:36.353 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:36.387 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.042) 0:01:13.368 ***** 2021-01-06T21:58:36.394 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:36.430 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.043) 0:01:13.411 ***** 2021-01-06T21:58:36.439 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:36.475 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.044) 0:01:13.456 ***** 2021-01-06T21:58:36.481 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:36.518 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.042) 0:01:13.499 ***** 2021-01-06T21:58:36.524 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-01-06T21:58:36.525 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:36.561 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.042) 0:01:13.542 ***** 2021-01-06T21:58:36.567 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:36.625 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.063) 0:01:13.606 ***** 2021-01-06T21:58:36.633 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-01-06T21:58:36.634 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:36.670 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.045) 0:01:13.651 ***** 2021-01-06T21:58:36.676 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:36.714 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.044) 0:01:13.695 ***** 2021-01-06T21:58:36.721 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-01-06T21:58:36.758 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.043) 0:01:13.738 ***** 2021-01-06T21:58:36.807 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.048) 0:01:13.787 ***** 2021-01-06T21:58:36.815 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-01-06T21:58:36.853 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.046) 0:01:13.834 ***** 2021-01-06T21:58:36.896 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.042) 0:01:13.877 ***** 2021-01-06T21:58:36.903 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-01-06T21:58:36.950 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:36 +0000 (0:00:00.053) 0:01:13.931 ***** 2021-01-06T21:58:37.011 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.060) 0:01:13.991 ***** 2021-01-06T21:58:37.017 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-01-06T21:58:37.020 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.068 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.057) 0:01:14.049 ***** 2021-01-06T21:58:37.076 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.114 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.045) 0:01:14.094 ***** 2021-01-06T21:58:37.120 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-01-06T21:58:37.121 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.160 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.046) 0:01:14.141 ***** 2021-01-06T21:58:37.166 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.168 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.204 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.044) 0:01:14.185 ***** 2021-01-06T21:58:37.211 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.248 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.043) 0:01:14.229 ***** 2021-01-06T21:58:37.254 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.294 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.046) 0:01:14.275 ***** 2021-01-06T21:58:37.301 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-06T21:58:37.303 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.341 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.046) 0:01:14.322 ***** 2021-01-06T21:58:37.348 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.387 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.045) 0:01:14.367 ***** 2021-01-06T21:58:37.393 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-01-06T21:58:37.395 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.431 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.044) 0:01:14.412 ***** 2021-01-06T21:58:37.437 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.473 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.042) 0:01:14.454 ***** 2021-01-06T21:58:37.482 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-01-06T21:58:37.483 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.517 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.044) 0:01:14.498 ***** 2021-01-06T21:58:37.524 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.561 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.043) 0:01:14.541 ***** 2021-01-06T21:58:37.568 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.604 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.043) 0:01:14.585 ***** 2021-01-06T21:58:37.610 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.649 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.045) 0:01:14.630 ***** 2021-01-06T21:58:37.655 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-01-06T21:58:37.657 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.693 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.043) 0:01:14.674 ***** 2021-01-06T21:58:37.700 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.736 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.043) 0:01:14.717 ***** 2021-01-06T21:58:37.743 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-01-06T21:58:37.744 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.779 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.043) 0:01:14.760 ***** 2021-01-06T21:58:37.785 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.821 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.042) 0:01:14.802 ***** 2021-01-06T21:58:37.827 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-01-06T21:58:37.829 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.864 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.043) 0:01:14.845 ***** 2021-01-06T21:58:37.870 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.908 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.043) 0:01:14.889 ***** 2021-01-06T21:58:37.915 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-01-06T21:58:37.917 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.952 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.043) 0:01:14.933 ***** 2021-01-06T21:58:37.958 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:37.994 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:37 +0000 (0:00:00.041) 0:01:14.975 ***** 2021-01-06T21:58:38.001 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-01-06T21:58:38.003 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.036 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.042) 0:01:15.017 ***** 2021-01-06T21:58:38.042 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.080 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.042) 0:01:15.060 ***** 2021-01-06T21:58:38.085 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-01-06T21:58:38.087 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.122 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.042) 0:01:15.103 ***** 2021-01-06T21:58:38.128 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.163 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.041) 0:01:15.144 ***** 2021-01-06T21:58:38.169 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-01-06T21:58:38.171 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.205 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.041) 0:01:15.186 ***** 2021-01-06T21:58:38.211 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.247 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.041) 0:01:15.227 ***** 2021-01-06T21:58:38.259 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-01-06T21:58:38.261 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.294 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.047) 0:01:15.275 ***** 2021-01-06T21:58:38.300 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.337 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.042) 0:01:15.318 ***** 2021-01-06T21:58:38.343 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-01-06T21:58:38.344 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.378 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.041) 0:01:15.359 ***** 2021-01-06T21:58:38.384 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.421 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.043) 0:01:15.402 ***** 2021-01-06T21:58:38.428 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-01-06T21:58:38.429 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.463 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.041) 0:01:15.444 ***** 2021-01-06T21:58:38.468 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.504 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.040) 0:01:15.485 ***** 2021-01-06T21:58:38.510 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.545 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.041) 0:01:15.526 ***** 2021-01-06T21:58:38.553 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.588 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.043) 0:01:15.569 ***** 2021-01-06T21:58:38.595 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-01-06T21:58:38.596 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.631 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.043) 0:01:15.612 ***** 2021-01-06T21:58:38.637 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.678 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.046) 0:01:15.659 ***** 2021-01-06T21:58:38.684 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-01-06T21:58:38.686 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.721 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.042) 0:01:15.701 ***** 2021-01-06T21:58:38.726 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.763 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.041) 0:01:15.743 ***** 2021-01-06T21:58:38.769 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-01-06T21:58:38.771 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.809 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.046) 0:01:15.789 ***** 2021-01-06T21:58:38.815 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.854 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.045) 0:01:15.835 ***** 2021-01-06T21:58:38.869 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-01-06T21:58:38.871 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.906 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.051) 0:01:15.887 ***** 2021-01-06T21:58:38.912 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:38.950 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:38 +0000 (0:00:00.044) 0:01:15.931 ***** 2021-01-06T21:58:39.042 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-01-06T21:58:39.044 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005171", "end": "2021-01-06 21:58:39.026786", "rc": 0, "start": "2021-01-06 21:58:39.021615", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:58:39.084 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:39 +0000 (0:00:00.133) 0:01:16.065 ***** 2021-01-06T21:58:39.091 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.003235", "end": "2021-01-06 21:58:39.068398", "rc": 0, "start": "2021-01-06 21:58:39.065163", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:58:39.128 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:39 +0000 (0:00:00.044) 0:01:16.109 ***** 2021-01-06T21:58:39.135 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-01-06T21:58:39.191 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:39 +0000 (0:00:00.062) 0:01:16.172 ***** 2021-01-06T21:58:39.245 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:39 +0000 (0:00:00.052) 0:01:16.225 ***** 2021-01-06T21:58:39.539 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-01-06T21:58:39.593 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:39 +0000 (0:00:00.348) 0:01:16.573 ***** 2021-01-06T21:58:39.607 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-01-06T21:58:39.646 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:39 +0000 (0:00:00.053) 0:01:16.627 ***** 2021-01-06T21:58:39.652 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-01-06T21:58:39.654 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:39.693 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:39 +0000 (0:00:00.045) 0:01:16.673 ***** 2021-01-06T21:58:39.701 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:39.754 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:39 +0000 (0:00:00.062) 0:01:16.735 ***** 2021-01-06T21:58:41.047 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-01-06T21:58:41.049 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-01-06T21:58:41.069 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-01-06T21:58:41.975 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-01-06T21:58:42.013 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:42 +0000 (0:00:02.258) 0:01:18.993 ***** 2021-01-06T21:58:42.019 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-01-06T21:58:42.058 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:42 +0000 (0:00:00.045) 0:01:19.039 ***** 2021-01-06T21:58:42.922 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-01-06T21:58:42.924 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"cache_update_time": 1609970065, "cache_updated": false, "changed": false} 2021-01-06T21:58:42.963 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:42 +0000 (0:00:00.904) 0:01:19.944 ***** 2021-01-06T21:58:42.970 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"cache_update_time": 1609970019, "cache_updated": false, "changed": false} 2021-01-06T21:58:43.009 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:43 +0000 (0:00:00.045) 0:01:19.990 ***** 2021-01-06T21:58:43.017 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-01-06T21:58:43.055 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:43 +0000 (0:00:00.046) 0:01:20.036 ***** 2021-01-06T21:58:43.106 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:43 +0000 (0:00:00.048) 0:01:20.084 ***** 2021-01-06T21:58:43.235 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** [WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo 2021-01-06T21:58:43.237 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.033881", "end": "2021-01-06 21:58:43.210109", "rc": 0, "start": "2021-01-06 21:58:43.176228", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:58:43.271 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:43 +0000 (0:00:00.167) 0:01:20.252 ***** 2021-01-06T21:58:43.288 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.034821", "end": "2021-01-06 21:58:43.261562", "rc": 0, "start": "2021-01-06 21:58:43.226741", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:58:43.325 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:43 +0000 (0:00:00.053) 0:01:20.306 ***** 2021-01-06T21:58:46.200 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-01-06T21:58:46.201 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"cache_update_time": 1609970065, "cache_updated": false, "changed": false} 2021-01-06T21:58:46.236 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:46 +0000 (0:00:02.911) 0:01:23.217 ***** 2021-01-06T21:58:46.253 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"cache_update_time": 1609970019, "cache_updated": false, "changed": false} 2021-01-06T21:58:46.291 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:46 +0000 (0:00:00.054) 0:01:23.272 ***** 2021-01-06T21:58:55.542 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 9 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 2021-01-06T21:58:55.544 INFO:teuthology.task.ansible.out: 95%\r(Reading database ... 100%\r(Reading database ... 137610 files and directories currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 9 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading d 2021-01-06T21:58:55.546 INFO:teuthology.task.ansible.out:atabase ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137610 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-01-06T21:58:55.578 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:55 +0000 (0:00:09.286) 0:01:32.559 ***** 2021-01-06T21:58:55.922 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 9 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137610 files and directories 2021-01-06T21:58:55.924 INFO:teuthology.task.ansible.out: currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 9 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2021-01-06T21:58:55.926 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137610 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-01-06T21:58:55.959 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:55 +0000 (0:00:00.381) 0:01:32.940 ***** 2021-01-06T21:58:56.487 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-01-06T21:58:56.489 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"cache_update_time": 1609970019, "cache_updated": false, "changed": false} 2021-01-06T21:58:56.525 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:56 +0000 (0:00:00.565) 0:01:33.505 ***** 2021-01-06T21:58:56.552 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-01-06T21:58:56.554 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:56.591 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:56 +0000 (0:00:00.066) 0:01:33.572 ***** 2021-01-06T21:58:56.818 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-01-06T21:58:56.820 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"cache_update_time": 1609970065, "cache_updated": false, "changed": false} 2021-01-06T21:58:56.854 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:56 +0000 (0:00:00.263) 0:01:33.835 ***** 2021-01-06T21:58:56.881 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:58:56.916 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:58:56 +0000 (0:00:00.061) 0:01:33.897 ***** 2021-01-06T21:59:05.263 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* changed: [smithi203.front.sepia.ceph.com] => {"cache_update_time": 1609970019, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 9 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.34 [40.2 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.34 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.34 [5188 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.34 [871 kB]\nFetched 10.4 MB in 1s (8846 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database 2021-01-06T21:59:05.265 INFO:teuthology.task.ansible.out:... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137468 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nProcessing triggers 2021-01-06T21:59:05.267 INFO:teuthology.task.ansible.out: for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 9 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.34 [40.2 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.34 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.34 [5188 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.34 [871 kB]", "Fetched 10.4 MB in 1s (8846 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Readin 2021-01-06T21:59:05.269 INFO:teuthology.task.ansible.out:g database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137468 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-01-06T21:59:05.328 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:05 +0000 (0:00:08.411) 0:01:42.309 ***** 2021-01-06T21:59:05.621 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"cache_update_time": 1609970065, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 9 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.34 [40.2 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.34 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.34 [5188 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.34 [871 kB]\nFetched 10.4 MB in 1s (9457 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2021-01-06T21:59:05.623 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137468 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2021-01-06T21:59:05.625 INFO:teuthology.task.ansible.out:1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 9 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.34 [40.2 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.34 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.34 [5188 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.34 [871 kB]", "Fetched 10.4 MB in 1s (9457 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2021-01-06T21:59:05.626 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137468 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.34_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-01-06T21:59:05.661 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:05 +0000 (0:00:00.333) 0:01:42.642 ***** 2021-01-06T21:59:06.249 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-01-06T21:59:06.251 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"cache_update_time": 1609970019, "cache_updated": false, "changed": false} 2021-01-06T21:59:06.291 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:06 +0000 (0:00:00.630) 0:01:43.272 ***** 2021-01-06T21:59:06.414 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-01-06T21:59:06.416 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-01-06T21:59:06.452 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:06 +0000 (0:00:00.160) 0:01:43.433 ***** 2021-01-06T21:59:06.479 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-01-06T21:59:06.481 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:06.519 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:06 +0000 (0:00:00.066) 0:01:43.500 ***** 2021-01-06T21:59:06.567 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-01-06T21:59:06.569 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"cache_update_time": 1609970065, "cache_updated": false, "changed": false} 2021-01-06T21:59:06.617 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:06 +0000 (0:00:00.098) 0:01:43.598 ***** 2021-01-06T21:59:06.655 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-01-06T21:59:06.657 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-01-06T21:59:06.697 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:06 +0000 (0:00:00.079) 0:01:43.678 ***** 2021-01-06T21:59:06.745 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-01-06T21:59:06.747 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-01-06T21:59:06.783 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:06 +0000 (0:00:00.086) 0:01:43.764 ***** 2021-01-06T21:59:06.810 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:06.845 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:06 +0000 (0:00:00.062) 0:01:43.826 ***** 2021-01-06T21:59:06.978 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-01-06T21:59:06.980 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-01-06T21:59:07.015 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:07 +0000 (0:00:00.169) 0:01:43.996 ***** 2021-01-06T21:59:07.017 INFO:teuthology.task.ansible.out: 2021-01-06T21:59:07.023 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-01-06T21:59:07.025 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-01-06T21:59:07.065 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:07 +0000 (0:00:00.049) 0:01:44.046 ***** 2021-01-06T21:59:07.221 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-01-06T21:59:07.223 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-01-06T21:59:07.259 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:07 +0000 (0:00:00.193) 0:01:44.240 ***** 2021-01-06T21:59:07.327 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-01-06T21:59:07.328 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-01-06T21:59:07.366 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:07 +0000 (0:00:00.107) 0:01:44.347 ***** 2021-01-06T21:59:07.393 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-01-06T21:59:07.395 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-06T21:59:07.430 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:07 +0000 (0:00:00.063) 0:01:44.410 ***** 2021-01-06T21:59:07.510 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-01-06T21:59:07.511 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-01-06T21:59:07.552 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:07 +0000 (0:00:00.122) 0:01:44.533 ***** 2021-01-06T21:59:07.687 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** ok: [smithi091.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-06T21:59:07.723 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:07 +0000 (0:00:00.170) 0:01:44.704 ***** 2021-01-06T21:59:07.730 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-01-06T21:59:07.731 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Wed 2021-01-06 21:56:59 UTC", "ActiveEnterTimestampMonotonic": "19495872", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "remote-fs.target -.mount system.slice nss-lookup.target basic.target network.target systemd-tmpfiles-setup.service sysinit.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Wed 2021-01-06 21:56:58 UTC", "AssertTimestampMonotonic": "19047831", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Wed 2021-01-06 21:56:58 UTC", "ConditionTimestampMonotonic": "19047831", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2021-01-06T21:59:07.733 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1339", "ExecMainStartTimestamp": "Wed 2021-01-06 21:56:59 UTC", "ExecMainStartTimestampMonotonic": "19495861", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Wed 2021-01-06 21:56:58 UTC", "InactiveExitTimestampMonotonic": "19048684", "InvocationID": "67eb5649c85741468260a401f3d52818", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2021-01-06T21:59:07.734 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128028", "LimitNPROCSoft": "128028", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1339", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2021-01-06T21:59:07.736 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Wed 2021-01-06 21:56:59 UTC", "StateChangeTimestampMonotonic": "19495872", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Wed 2021-01-06 21:56:59 UTC", "WatchdogTimestampMonotonic": "19495871", "WatchdogUSec": "0"}} 2021-01-06T21:59:07.770 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:07 +0000 (0:00:00.046) 0:01:44.751 ***** 2021-01-06T21:59:07.797 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:07.844 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:07 +0000 (0:00:00.074) 0:01:44.825 ***** 2021-01-06T21:59:07.872 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-01-06T21:59:07.874 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:07.909 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:07 +0000 (0:00:00.064) 0:01:44.889 ***** 2021-01-06T21:59:07.935 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-01-06T21:59:07.937 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:07.972 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:07 +0000 (0:00:00.063) 0:01:44.953 ***** 2021-01-06T21:59:07.999 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-01-06T21:59:08.000 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.034 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.061) 0:01:45.015 ***** 2021-01-06T21:59:08.039 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-01-06T21:59:08.041 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Wed 2021-01-06 21:57:03 UTC", "ActiveEnterTimestampMonotonic": "21318966", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target basic.target network.target system.slice systemd-tmpfiles-setup.service remote-fs.target nss-lookup.target -.mount systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Wed 2021-01-06 21:57:03 UTC", "AssertTimestampMonotonic": "20809553", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Wed 2021-01-06 21:57:03 UTC", "ConditionTimestampMonotonic": "20809553", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2021-01-06T21:59:08.043 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1333", "ExecMainStartTimestamp": "Wed 2021-01-06 21:57:03 UTC", "ExecMainStartTimestampMonotonic": "21318928", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Wed 2021-01-06 21:57:03 UTC", "InactiveExitTimestampMonotonic": "20811030", "InvocationID": "446f982a0571414ab45d2b5ac0125dbb", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2021-01-06T21:59:08.045 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128028", "LimitNPROCSoft": "128028", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1333", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2021-01-06T21:59:08.046 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Wed 2021-01-06 21:57:03 UTC", "StateChangeTimestampMonotonic": "21318966", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Wed 2021-01-06 21:57:03 UTC", "WatchdogTimestampMonotonic": "21318963", "WatchdogUSec": "0"}} 2021-01-06T21:59:08.077 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.042) 0:01:45.058 ***** 2021-01-06T21:59:08.083 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.119 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.042) 0:01:45.100 ***** 2021-01-06T21:59:08.124 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-01-06T21:59:08.126 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.160 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.041) 0:01:45.141 ***** 2021-01-06T21:59:08.166 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-01-06T21:59:08.168 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.210 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.049) 0:01:45.190 ***** 2021-01-06T21:59:08.216 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-01-06T21:59:08.217 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.253 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.043) 0:01:45.234 ***** 2021-01-06T21:59:08.261 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-01-06T21:59:08.263 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.303 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.049) 0:01:45.284 ***** 2021-01-06T21:59:08.310 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-01-06T21:59:08.313 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.346 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.042) 0:01:45.326 ***** 2021-01-06T21:59:08.351 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-01-06T21:59:08.353 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.389 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.043) 0:01:45.370 ***** 2021-01-06T21:59:08.395 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-01-06T21:59:08.397 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.433 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.043) 0:01:45.413 ***** 2021-01-06T21:59:08.438 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-06T21:59:08.440 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.474 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.042) 0:01:45.455 ***** 2021-01-06T21:59:08.480 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-01-06T21:59:08.482 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.518 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.043) 0:01:45.499 ***** 2021-01-06T21:59:08.524 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-01-06T21:59:08.526 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.561 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.043) 0:01:45.542 ***** 2021-01-06T21:59:08.567 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-01-06T21:59:08.569 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.603 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.042) 0:01:45.584 ***** 2021-01-06T21:59:08.611 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.650 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.047) 0:01:45.631 ***** 2021-01-06T21:59:08.657 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-01-06T21:59:08.660 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.696 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.045) 0:01:45.677 ***** 2021-01-06T21:59:08.702 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-01-06T21:59:08.704 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.740 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.044) 0:01:45.721 ***** 2021-01-06T21:59:08.747 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-01-06T21:59:08.748 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.786 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.044) 0:01:45.766 ***** 2021-01-06T21:59:08.792 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-01-06T21:59:08.794 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.834 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.048) 0:01:45.815 ***** 2021-01-06T21:59:08.841 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.879 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.045) 0:01:45.860 ***** 2021-01-06T21:59:08.887 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-01-06T21:59:08.889 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.924 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.044) 0:01:45.905 ***** 2021-01-06T21:59:08.930 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-01-06T21:59:08.932 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:08.968 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:08 +0000 (0:00:00.044) 0:01:45.949 ***** 2021-01-06T21:59:08.975 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.013 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.044) 0:01:45.994 ***** 2021-01-06T21:59:09.019 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-01-06T21:59:09.021 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.064 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.050) 0:01:46.045 ***** 2021-01-06T21:59:09.074 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-01-06T21:59:09.076 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.122 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.058) 0:01:46.103 ***** 2021-01-06T21:59:09.129 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-01-06T21:59:09.130 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.172 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.050) 0:01:46.153 ***** 2021-01-06T21:59:09.179 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-01-06T21:59:09.180 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.216 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.043) 0:01:46.197 ***** 2021-01-06T21:59:09.223 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-01-06T21:59:09.225 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.261 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.045) 0:01:46.242 ***** 2021-01-06T21:59:09.268 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-01-06T21:59:09.269 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.311 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.049) 0:01:46.292 ***** 2021-01-06T21:59:09.318 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-01-06T21:59:09.319 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.364 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.052) 0:01:46.345 ***** 2021-01-06T21:59:09.370 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-01-06T21:59:09.372 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.407 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.043) 0:01:46.388 ***** 2021-01-06T21:59:09.413 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-01-06T21:59:09.415 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.451 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.043) 0:01:46.432 ***** 2021-01-06T21:59:09.458 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-01-06T21:59:09.459 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.497 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.046) 0:01:46.478 ***** 2021-01-06T21:59:09.503 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-01-06T21:59:09.505 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.542 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.044) 0:01:46.523 ***** 2021-01-06T21:59:09.549 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-01-06T21:59:09.551 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.591 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.049) 0:01:46.572 ***** 2021-01-06T21:59:09.598 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-01-06T21:59:09.600 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.641 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.050) 0:01:46.622 ***** 2021-01-06T21:59:09.650 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-01-06T21:59:09.652 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.688 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.046) 0:01:46.669 ***** 2021-01-06T21:59:09.694 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-01-06T21:59:09.696 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.737 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.049) 0:01:46.718 ***** 2021-01-06T21:59:09.744 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.782 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.043) 0:01:46.762 ***** 2021-01-06T21:59:09.787 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-01-06T21:59:09.789 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.826 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.044) 0:01:46.807 ***** 2021-01-06T21:59:09.832 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-01-06T21:59:09.834 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.871 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.044) 0:01:46.852 ***** 2021-01-06T21:59:09.878 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-01-06T21:59:09.880 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.916 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.044) 0:01:46.896 ***** 2021-01-06T21:59:09.921 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-01-06T21:59:09.923 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:09.963 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:09 +0000 (0:00:00.047) 0:01:46.944 ***** 2021-01-06T21:59:09.969 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-01-06T21:59:09.971 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:10.007 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:10 +0000 (0:00:00.043) 0:01:46.987 ***** 2021-01-06T21:59:10.013 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-01-06T21:59:10.015 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:10.050 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:10 +0000 (0:00:00.043) 0:01:47.031 ***** 2021-01-06T21:59:10.056 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-01-06T21:59:10.057 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:10.093 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:10 +0000 (0:00:00.043) 0:01:47.074 ***** 2021-01-06T21:59:10.123 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:10.167 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:10 +0000 (0:00:00.073) 0:01:47.148 ***** 2021-01-06T21:59:10.175 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-01-06T21:59:10.177 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-01-06T21:59:10.213 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:10 +0000 (0:00:00.045) 0:01:47.194 ***** 2021-01-06T21:59:10.220 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-01-06T21:59:10.222 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:10.260 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:10 +0000 (0:00:00.046) 0:01:47.241 ***** 2021-01-06T21:59:10.288 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-01-06T21:59:10.289 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:10.339 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:10 +0000 (0:00:00.078) 0:01:47.320 ***** 2021-01-06T21:59:10.370 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:10.409 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:10 +0000 (0:00:00.070) 0:01:47.390 ***** 2021-01-06T21:59:10.531 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-01-06T21:59:10.533 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-01-06T21:59:10.572 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:10 +0000 (0:00:00.162) 0:01:47.553 ***** 2021-01-06T21:59:10.578 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-01-06T21:59:10.579 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-01-06T21:59:10.615 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:10 +0000 (0:00:00.042) 0:01:47.595 ***** 2021-01-06T21:59:10.622 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-01-06T21:59:10.624 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:10.658 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:10 +0000 (0:00:00.043) 0:01:47.639 ***** 2021-01-06T21:59:10.915 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-01-06T21:59:10.917 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-01-06T21:59:10.951 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:10 +0000 (0:00:00.292) 0:01:47.931 ***** 2021-01-06T21:59:10.977 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-01-06T21:59:10.979 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:11.013 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:11 +0000 (0:00:00.062) 0:01:47.994 ***** 2021-01-06T21:59:11.020 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-01-06T21:59:11.022 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-01-06T21:59:11.059 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:11 +0000 (0:00:00.046) 0:01:48.040 ***** 2021-01-06T21:59:11.326 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-01-06T21:59:11.363 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:11 +0000 (0:00:00.304) 0:01:48.344 ***** 2021-01-06T21:59:11.369 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-01-06T21:59:11.371 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-01-06T21:59:11.413 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:11 +0000 (0:00:00.049) 0:01:48.394 ***** 2021-01-06T21:59:11.668 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-01-06T21:59:11.703 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:11 +0000 (0:00:00.290) 0:01:48.684 ***** 2021-01-06T21:59:11.720 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-01-06T21:59:11.722 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-01-06T21:59:11.756 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:11 +0000 (0:00:00.053) 0:01:48.737 ***** 2021-01-06T21:59:11.888 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** ok: [smithi203.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-06T21:59:11.925 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:11 +0000 (0:00:00.168) 0:01:48.906 ***** 2021-01-06T21:59:11.951 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-01-06T21:59:11.953 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:11.988 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:11 +0000 (0:00:00.062) 0:01:48.969 ***** 2021-01-06T21:59:12.004 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-01-06T21:59:12.006 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-01-06T21:59:12.041 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.053) 0:01:49.022 ***** 2021-01-06T21:59:12.047 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-01-06T21:59:12.053 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.085 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.043) 0:01:49.066 ***** 2021-01-06T21:59:12.111 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-01-06T21:59:12.113 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.146 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.060) 0:01:49.127 ***** 2021-01-06T21:59:12.163 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-01-06T21:59:12.165 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-06T21:59:12.199 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.053) 0:01:49.180 ***** 2021-01-06T21:59:12.205 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-01-06T21:59:12.207 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.242 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.042) 0:01:49.223 ***** 2021-01-06T21:59:12.248 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-01-06T21:59:12.249 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.284 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.042) 0:01:49.265 ***** 2021-01-06T21:59:12.290 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-01-06T21:59:12.292 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.329 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.044) 0:01:49.310 ***** 2021-01-06T21:59:12.335 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-01-06T21:59:12.337 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.380 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.050) 0:01:49.360 ***** 2021-01-06T21:59:12.387 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi203.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-01-06T21:59:12.427 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.047) 0:01:49.408 ***** 2021-01-06T21:59:12.435 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-01-06T21:59:12.436 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.474 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.046) 0:01:49.455 ***** 2021-01-06T21:59:12.482 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-01-06T21:59:12.484 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.496 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.505 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-01-06T21:59:12.507 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi203.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.543 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.068) 0:01:49.524 ***** 2021-01-06T21:59:12.549 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-01-06T21:59:12.551 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.553 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.563 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.565 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.599 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.055) 0:01:49.579 ***** 2021-01-06T21:59:12.605 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-01-06T21:59:12.607 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.647 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.048) 0:01:49.628 ***** 2021-01-06T21:59:12.653 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-06T21:59:12.655 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-01-06T21:59:12.690 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.042) 0:01:49.670 ***** 2021-01-06T21:59:12.738 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-01-06T21:59:12.740 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.751 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.752 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.763 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.776 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.786 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.797 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-01-06T21:59:12.808 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.810 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.819 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.830 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.841 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-01-06T21:59:12.842 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004107", "end": "2021-01-06 21:59:12.813318", "rc": 0, "start": "2021-01-06 21:59:12.809211", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi091.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.881 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.191) 0:01:49.862 ***** 2021-01-06T21:59:12.886 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.888 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.931 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.049) 0:01:49.912 ***** 2021-01-06T21:59:12.937 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-01-06T21:59:12.939 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:12.980 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:12 +0000 (0:00:00.048) 0:01:49.960 ***** 2021-01-06T21:59:13.016 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-01-06T21:59:13.018 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:13.053 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:13 +0000 (0:00:00.073) 0:01:50.034 ***** 2021-01-06T21:59:13.091 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:13.129 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:13 +0000 (0:00:00.075) 0:01:50.109 ***** 2021-01-06T21:59:13.135 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-01-06T21:59:13.137 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003251", "end": "2021-01-06 21:59:13.097540", "rc": 0, "start": "2021-01-06 21:59:13.094289", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2021-01-06T21:59:13.184 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:13 +0000 (0:00:00.055) 0:01:50.165 ***** 2021-01-06T21:59:13.222 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-01-06T21:59:13.223 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:13.261 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:13 +0000 (0:00:00.076) 0:01:50.242 ***** 2021-01-06T21:59:13.287 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-01-06T21:59:13.289 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:13.324 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:13 +0000 (0:00:00.062) 0:01:50.304 ***** 2021-01-06T21:59:13.350 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-01-06T21:59:13.351 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:13.385 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:13 +0000 (0:00:00.062) 0:01:50.366 ***** 2021-01-06T21:59:14.036 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-01-06T21:59:14.038 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"cache_update_time": 1609970019, "cache_updated": false, "changed": false} 2021-01-06T21:59:14.073 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:14 +0000 (0:00:00.687) 0:01:51.054 ***** 2021-01-06T21:59:14.099 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-01-06T21:59:14.101 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:14.136 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:14 +0000 (0:00:00.063) 0:01:51.117 ***** 2021-01-06T21:59:14.173 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-01-06T21:59:14.175 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item={'block_used': 13754039, 'uuid': '03d36009-ad2e-4bf1-9745-015a33230619', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226341069, 'size_available': 927093018624, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226341069, "block_size": 4096, "block_total": 240095108, "block_used": 13754039, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927093018624, "size_total": 983429562368, "uuid": "03d36009-ad2e-4bf1-9745-015a33230619"}} 2021-01-06T21:59:14.212 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:14 +0000 (0:00:00.075) 0:01:51.193 ***** 2021-01-06T21:59:14.271 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* ok: [smithi091.front.sepia.ceph.com] => {"cache_update_time": 1609970065, "cache_updated": false, "changed": false} 2021-01-06T21:59:14.308 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:14 +0000 (0:00:00.095) 0:01:51.289 ***** 2021-01-06T21:59:14.335 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-01-06T21:59:14.337 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:14.370 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:14 +0000 (0:00:00.062) 0:01:51.351 ***** 2021-01-06T21:59:14.377 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-01-06T21:59:14.378 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.009248", "end": "2021-01-06 21:59:14.338265", "rc": 0, "start": "2021-01-06 21:59:14.329017", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:14.417 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:14 +0000 (0:00:00.046) 0:01:51.398 ***** 2021-01-06T21:59:14.422 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-01-06T21:59:14.424 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'block_used': 13754046, 'uuid': '03d36009-ad2e-4bf1-9745-015a33230619', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226341062, 'size_available': 927092989952, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226341062, "block_size": 4096, "block_total": 240095108, "block_used": 13754046, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927092989952, "size_total": 983429562368, "uuid": "03d36009-ad2e-4bf1-9745-015a33230619"}} 2021-01-06T21:59:14.461 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:14 +0000 (0:00:00.044) 0:01:51.442 ***** 2021-01-06T21:59:14.467 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-01-06T21:59:14.468 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'block_used': 13754039, 'uuid': '03d36009-ad2e-4bf1-9745-015a33230619', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226341069, 'size_available': 927093018624, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226341069, "block_size": 4096, "block_total": 240095108, "block_used": 13754039, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927093018624, "size_total": 983429562368, "uuid": "03d36009-ad2e-4bf1-9745-015a33230619"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:14.505 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:14 +0000 (0:00:00.043) 0:01:51.486 ***** 2021-01-06T21:59:14.533 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-01-06T21:59:14.568 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:14 +0000 (0:00:00.062) 0:01:51.549 ***** 2021-01-06T21:59:14.595 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-01-06T21:59:14.597 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.011064", "end": "2021-01-06 21:59:14.577418", "rc": 0, "start": "2021-01-06 21:59:14.566354", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:14.632 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:14 +0000 (0:00:00.064) 0:01:51.613 ***** 2021-01-06T21:59:14.639 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-01-06T21:59:14.641 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:14.673 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [smithi091.front.sepia.ceph.com] => (item={'block_used': 13754046, 'uuid': '03d36009-ad2e-4bf1-9745-015a33230619', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226341062, 'size_available': 927092989952, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226341062, "block_size": 4096, "block_total": 240095108, "block_used": 13754046, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927092989952, "size_total": 983429562368, "uuid": "03d36009-ad2e-4bf1-9745-015a33230619"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:14.709 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:14 +0000 (0:00:00.076) 0:01:51.689 ***** 2021-01-06T21:59:14.728 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-01-06T21:59:14.762 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:14 +0000 (0:00:00.053) 0:01:51.743 ***** 2021-01-06T21:59:14.799 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-01-06T21:59:14.801 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:16.881 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSg7bPMIxQbVLSCw83wC6IxEHReH3YlOoW'], 'uuids': ['a7025c6e-ae21-489a-901f-72e75d6e189c']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:02.127863", "end": "2021-01-06 21:59:16.856502", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSg7bPMIxQbVLSCw83wC6IxEHReH3YlOoW"], "labels": [], "masters": [], "uuids": ["a7025c6e-ae21-489a-901f-72e75d6e189c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:14.728639", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi203.front.sepia.ceph.com] => 2021-01-06T21:59:16.883 INFO:teuthology.task.ansible.out:(item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:16.892 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:17.298 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZ6yW7dcYegpNcIZRZ4dhX6zcgufCVXdDk'], 'uuids': ['0971abb3-b123-4178-9181-bd9187813385']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:02.365043", "end": "2021-01-06 21:59:17.267268", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZ6yW7dcYegpNcIZRZ4dhX6zcgufCVXdDk"], "labels": [], "masters": [], "uuids": ["0971abb3-b123-4178-9181-bd9187813385"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:14.902225", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-06T21:59:17.310 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:17.322 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:18.148 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSkbb1KaRfqxdtRAbGv4QridJKZL3ljIsT'], 'uuids': ['2d7cc491-c85b-47b6-984f-59ddc0dc685b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.133715", "end": "2021-01-06 21:59:18.130105", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSkbb1KaRfqxdtRAbGv4QridJKZL3ljIsT"], "labels": [], "masters": [], "uuids": ["2d7cc491-c85b-47b6-984f-59ddc0dc685b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:16.996390", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-06T21:59:18.160 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XMNM', 'wwn-0x5000c500929386c5'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '03d36009-ad2e-4bf1-9745-015a33230619', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XMNM-part1', 'wwn-0x5000c500929386c5-part1'], 'uuids': ['03d36009-ad2e-4bf1-9745-015a33230619']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500929386c5', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XMNM', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XMNM", "wwn-0x5000c500929386c5"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XMNM-part1", "wwn-0x5000c500929386c5-part1"], "labels": [], "masters": [], "uuids": ["03d36009-ad2e-4bf1-9745-015a33230619"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "03d36009-ad2e-4bf1-9745-015a33230619"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XMNM", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500929386c5"}}, "skip_reason": 2021-01-06T21:59:18.162 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-01-06T21:59:18.630 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZO4xpzkxzVcLIMo0f1FucZOyK9RBe93pE'], 'uuids': ['a4c696cf-bb70-414c-85c5-25c8813a88d2']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.188613", "end": "2021-01-06 21:59:18.613657", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZO4xpzkxzVcLIMo0f1FucZOyK9RBe93pE"], "labels": [], "masters": [], "uuids": ["a4c696cf-bb70-414c-85c5-25c8813a88d2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:17.425044", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-06T21:59:18.642 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5L78A', 'wwn-0x5000c500918dacd0'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '03d36009-ad2e-4bf1-9745-015a33230619', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5L78A-part1', 'wwn-0x5000c500918dacd0-part1'], 'uuids': ['03d36009-ad2e-4bf1-9745-015a33230619']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500918dacd0', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5L78A', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L78A", "wwn-0x5000c500918dacd0"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L78A-part1", "wwn-0x5000c500918dacd0-part1"], "labels": [], "masters": [], "uuids": ["03d36009-ad2e-4bf1-9745-015a33230619"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "03d36009-ad2e-4bf1-9745-015a33230619"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5L78A", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500918dacd0"}}, "skip_reason": 2021-01-06T21:59:18.644 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-01-06T21:59:19.466 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-sQXuJQ-y0Ts-vZt3-49CF-BJjA-sg37-VZSkhR', 'nvme-INTEL_SSDPEDMD400G4_CVFT623300GK400BGN', 'nvme-nvme.8086-43564654363233333030474b34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.188966", "end": "2021-01-06 21:59:19.449104", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-sQXuJQ-y0Ts-vZt3-49CF-BJjA-sg37-VZSkhR", "nvme-INTEL_SSDPEDMD400G4_CVFT623300GK400BGN", "nvme-nvme.8086-43564654363233333030474b34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:18.260138", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2021-01-06T21:59:19.468 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-06T21:59:19.476 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:19.494 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:19.982 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-T8tgK4-Nuoo-Qwmj-7mVJ-5z6F-FsCf-a0KmUS', 'nvme-INTEL_SSDPEDMD400G4_CVFT623300AG400BGN', 'nvme-nvme.8086-43564654363233333030414734303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.216900", "end": "2021-01-06 21:59:19.954120", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-T8tgK4-Nuoo-Qwmj-7mVJ-5z6F-FsCf-a0KmUS", "nvme-INTEL_SSDPEDMD400G4_CVFT623300AG400BGN", "nvme-nvme.8086-43564654363233333030414734303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:18.737220", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2021-01-06T21:59:19.984 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-06T21:59:20.004 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:20.007 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:20.761 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSWofu5Av35FUhIkGDBv5Bs3fOA1aLjli9'], 'uuids': ['edda5773-32ee-4e0f-bbd3-dad4c1e5e54b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.147352", "end": "2021-01-06 21:59:20.740080", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSWofu5Av35FUhIkGDBv5Bs3fOA1aLjli9"], "labels": [], "masters": [], "uuids": ["edda5773-32ee-4e0f-bbd3-dad4c1e5e54b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:19.592728", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-06T21:59:20.772 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:20.784 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:21.447 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZM1dauCSRJemqu124u3c0MTLRGGegNaUP'], 'uuids': ['2c269c94-92f4-4285-bdcc-6bd748168ec1']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.329922", "end": "2021-01-06 21:59:21.430736", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZM1dauCSRJemqu124u3c0MTLRGGegNaUP"], "labels": [], "masters": [], "uuids": ["2c269c94-92f4-4285-bdcc-6bd748168ec1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:20.100814", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-06T21:59:21.459 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:21.470 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:22.209 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSOJxGW1KaC4PRqjyJwxP9kLtEZyI6WUrv'], 'uuids': ['ae1538a5-b76d-4c7a-bbb5-17f2ace5aaeb']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.302559", "end": "2021-01-06 21:59:22.190751", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSOJxGW1KaC4PRqjyJwxP9kLtEZyI6WUrv"], "labels": [], "masters": [], "uuids": ["ae1538a5-b76d-4c7a-bbb5-17f2ace5aaeb"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:20.888192", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-06T21:59:22.221 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:22.702 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZGKP8x3cm5mbkrPDG7iLGuuBWUHRtEziT'], 'uuids': ['4176f6bf-b807-48ca-a756-b9bd8f02dc8b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.123878", "end": "2021-01-06 21:59:22.688619", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZGKP8x3cm5mbkrPDG7iLGuuBWUHRtEziT"], "labels": [], "masters": [], "uuids": ["4176f6bf-b807-48ca-a756-b9bd8f02dc8b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:21.564741", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-06T21:59:22.725 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:23.492 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSae4rXH79h8sHMW3UsVS6c1So3o1K1sWj'], 'uuids': ['f26bd22a-6869-45d1-b795-d110b48b0ab6']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.153815", "end": "2021-01-06 21:59:23.474623", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSae4rXH79h8sHMW3UsVS6c1So3o1K1sWj"], "labels": [], "masters": [], "uuids": ["f26bd22a-6869-45d1-b795-d110b48b0ab6"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:22.320808", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-06T21:59:23.538 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:23 +0000 (0:00:08.776) 0:02:00.519 ***** 2021-01-06T21:59:23.592 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:23.884 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSg7bPMIxQbVLSCw83wC6IxEHReH3YlOoW'], 'uuids': ['a7025c6e-ae21-489a-901f-72e75d6e189c']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.156589", "end": "2021-01-06 21:59:23.868125", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSg7bPMIxQbVLSCw83wC6IxEHReH3YlOoW"], "labels": [], "masters": [], "uuids": ["a7025c6e-ae21-489a-901f-72e75d6e189c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:23.711536", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.15191 s, 759 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.15191 s, 759 MB/s"], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:23.897 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:23.899 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:23.963 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZnfgqqCNSN1gfYgtYo9jWV9uZxCMKA263'], 'uuids': ['9d27d7af-d9ba-42d6-9de8-4eb5a72f7894']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.127461", "end": "2021-01-06 21:59:23.938879", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZnfgqqCNSN1gfYgtYo9jWV9uZxCMKA263"], "labels": [], "masters": [], "uuids": ["9d27d7af-d9ba-42d6-9de8-4eb5a72f7894"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:22.811418", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk u 2021-01-06T21:59:23.966 INFO:teuthology.task.ansible.out:sing fdisk or", "other utilities."]} 2021-01-06T21:59:24.019 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:24 +0000 (0:00:00.480) 0:02:01.000 ***** 2021-01-06T21:59:24.058 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-01-06T21:59:24.060 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:24.300 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSkbb1KaRfqxdtRAbGv4QridJKZL3ljIsT'], 'uuids': ['2d7cc491-c85b-47b6-984f-59ddc0dc685b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.156920", "end": "2021-01-06 21:59:24.283296", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSkbb1KaRfqxdtRAbGv4QridJKZL3ljIsT"], "labels": [], "masters": [], "uuids": ["2d7cc491-c85b-47b6-984f-59ddc0dc685b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:24.126376", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.152031 s, 759 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.152031 s, 759 MB/s"], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:24.311 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XMNM', 'wwn-0x5000c500929386c5'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '03d36009-ad2e-4bf1-9745-015a33230619', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XMNM-part1', 'wwn-0x5000c500929386c5-part1'], 'uuids': ['03d36009-ad2e-4bf1-9745-015a33230619']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500929386c5', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XMNM', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XMNM", "wwn-0x5000c500929386c5"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XMNM-part1", "wwn-0x5000c500929386c5-part1"], "labels": [], "masters": [], "uuids": ["03d36009-ad2e-4bf1-9745-015a33230619"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "03d36009-ad2e-4bf1-9745-015a33230619"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XMNM", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500929386c5"}}, "skip_reason": 2021-01-06T21:59:24.314 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-01-06T21:59:24.386 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZ6yW7dcYegpNcIZRZ4dhX6zcgufCVXdDk'], 'uuids': ['0971abb3-b123-4178-9181-bd9187813385']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.159720", "end": "2021-01-06 21:59:24.358965", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZ6yW7dcYegpNcIZRZ4dhX6zcgufCVXdDk"], "labels": [], "masters": [], "uuids": ["0971abb3-b123-4178-9181-bd9187813385"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:24.199245", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.154956 s, 744 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.154956 s, 744 MB/s"], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:24.400 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:24.423 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:24.499 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-sQXuJQ-y0Ts-vZt3-49CF-BJjA-sg37-VZSkhR', 'nvme-INTEL_SSDPEDMD400G4_CVFT623300GK400BGN', 'nvme-nvme.8086-43564654363233333030474b34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.054585", "end": "2021-01-06 21:59:24.479162", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-sQXuJQ-y0Ts-vZt3-49CF-BJjA-sg37-VZSkhR", "nvme-INTEL_SSDPEDMD400G4_CVFT623300GK400BGN", "nvme-nvme.8086-43564654363233333030474b34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:24.424577", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0500607 s, 2.3 GB/s", "std 2021-01-06T21:59:24.501 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0500607 s, 2.3 GB/s"], "stdout": "", "stdout_lines": []} skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:24.512 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:24.719 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZO4xpzkxzVcLIMo0f1FucZOyK9RBe93pE'], 'uuids': ['a4c696cf-bb70-414c-85c5-25c8813a88d2']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.161303", "end": "2021-01-06 21:59:24.682825", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZO4xpzkxzVcLIMo0f1FucZOyK9RBe93pE"], "labels": [], "masters": [], "uuids": ["a4c696cf-bb70-414c-85c5-25c8813a88d2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:24.521522", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.156549 s, 737 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.156549 s, 737 MB/s"], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:24.731 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5L78A', 'wwn-0x5000c500918dacd0'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '03d36009-ad2e-4bf1-9745-015a33230619', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5L78A-part1', 'wwn-0x5000c500918dacd0-part1'], 'uuids': ['03d36009-ad2e-4bf1-9745-015a33230619']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500918dacd0', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5L78A', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L78A", "wwn-0x5000c500918dacd0"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L78A-part1", "wwn-0x5000c500918dacd0-part1"], "labels": [], "masters": [], "uuids": ["03d36009-ad2e-4bf1-9745-015a33230619"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "03d36009-ad2e-4bf1-9745-015a33230619"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5L78A", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500918dacd0"}}, "skip_reason": 2021-01-06T21:59:24.733 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-01-06T21:59:24.800 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSWofu5Av35FUhIkGDBv5Bs3fOA1aLjli9'], 'uuids': ['edda5773-32ee-4e0f-bbd3-dad4c1e5e54b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.153739", "end": "2021-01-06 21:59:24.775559", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSWofu5Av35FUhIkGDBv5Bs3fOA1aLjli9"], "labels": [], "masters": [], "uuids": ["edda5773-32ee-4e0f-bbd3-dad4c1e5e54b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:24.621820", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.150483 s, 766 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.150483 s, 766 MB/s"], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:24.813 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:24.913 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-T8tgK4-Nuoo-Qwmj-7mVJ-5z6F-FsCf-a0KmUS', 'nvme-INTEL_SSDPEDMD400G4_CVFT623300AG400BGN', 'nvme-nvme.8086-43564654363233333030414734303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.055541", "end": "2021-01-06 21:59:24.889482", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-T8tgK4-Nuoo-Qwmj-7mVJ-5z6F-FsCf-a0KmUS", "nvme-INTEL_SSDPEDMD400G4_CVFT623300AG400BGN", "nvme-nvme.8086-43564654363233333030414734303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:24.833941", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0509203 s, 2.3 GB/s", "std 2021-01-06T21:59:24.915 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0509203 s, 2.3 GB/s"], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:24.928 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:24.940 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:25.116 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSOJxGW1KaC4PRqjyJwxP9kLtEZyI6WUrv'], 'uuids': ['ae1538a5-b76d-4c7a-bbb5-17f2ace5aaeb']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.157250", "end": "2021-01-06 21:59:25.096180", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSOJxGW1KaC4PRqjyJwxP9kLtEZyI6WUrv"], "labels": [], "masters": [], "uuids": ["ae1538a5-b76d-4c7a-bbb5-17f2ace5aaeb"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:24.938930", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.152574 s, 756 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.152574 s, 756 MB/s"], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:25.120 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:25.240 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZM1dauCSRJemqu124u3c0MTLRGGegNaUP'], 'uuids': ['2c269c94-92f4-4285-bdcc-6bd748168ec1']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.160341", "end": "2021-01-06 21:59:25.211771", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZM1dauCSRJemqu124u3c0MTLRGGegNaUP"], "labels": [], "masters": [], "uuids": ["2c269c94-92f4-4285-bdcc-6bd748168ec1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:25.051430", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.157108 s, 734 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.157108 s, 734 MB/s"], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:25.252 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:25.268 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:25.392 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSae4rXH79h8sHMW3UsVS6c1So3o1K1sWj'], 'uuids': ['f26bd22a-6869-45d1-b795-d110b48b0ab6']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.150606", "end": "2021-01-06 21:59:25.371047", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-knVREnrqmW9HDsVTFKy7bj9ySSV4FWLSae4rXH79h8sHMW3UsVS6c1So3o1K1sWj"], "labels": [], "masters": [], "uuids": ["f26bd22a-6869-45d1-b795-d110b48b0ab6"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:25.220441", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.146018 s, 790 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.146018 s, 790 MB/s"], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:25.454 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:25 +0000 (0:00:01.433) 0:02:02.434 ***** 2021-01-06T21:59:25.571 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZGKP8x3cm5mbkrPDG7iLGuuBWUHRtEziT'], 'uuids': ['4176f6bf-b807-48ca-a756-b9bd8f02dc8b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.168221", "end": "2021-01-06 21:59:25.542234", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZGKP8x3cm5mbkrPDG7iLGuuBWUHRtEziT"], "labels": [], "masters": [], "uuids": ["4176f6bf-b807-48ca-a756-b9bd8f02dc8b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:25.374013", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.163447 s, 706 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.163447 s, 706 MB/s"], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:25.603 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:25.742 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi203.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.150646", "end": "2021-01-06 21:59:25.721029", "rc": 0, "start": "2021-01-06 21:59:25.570383", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:25.782 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:25 +0000 (0:00:00.328) 0:02:02.763 ***** 2021-01-06T21:59:25.908 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-01-06T21:59:25.910 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZnfgqqCNSN1gfYgtYo9jWV9uZxCMKA263'], 'uuids': ['9d27d7af-d9ba-42d6-9de8-4eb5a72f7894']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.161424", "end": "2021-01-06 21:59:25.878978", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-zzSX1thjR3XsFqDRSO4z54yJKeX6lbTZnfgqqCNSN1gfYgtYo9jWV9uZxCMKA263"], "labels": [], "masters": [], "uuids": ["9d27d7af-d9ba-42d6-9de8-4eb5a72f7894"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-06 21:59:25.717554", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.15676 s, 736 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.15676 s, 736 MB/s"], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:25.959 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:25 +0000 (0:00:00.176) 0:02:02.940 ***** 2021-01-06T21:59:25.986 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-01-06T21:59:25.989 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.065191", "end": "2021-01-06 21:59:25.969528", "rc": 0, "start": "2021-01-06 21:59:25.904337", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:26.027 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:26 +0000 (0:00:00.068) 0:02:03.008 ***** 2021-01-06T21:59:26.047 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-01-06T21:59:26.084 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:26 +0000 (0:00:00.056) 0:02:03.065 ***** 2021-01-06T21:59:26.131 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-01-06T21:59:26.133 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:26.171 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:26 +0000 (0:00:00.087) 0:02:03.152 ***** 2021-01-06T21:59:26.208 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-01-06T21:59:26.210 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:26.243 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:26 +0000 (0:00:00.071) 0:02:03.224 ***** 2021-01-06T21:59:26.261 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-01-06T21:59:26.262 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.155896", "end": "2021-01-06 21:59:26.237867", "rc": 0, "start": "2021-01-06 21:59:26.081971", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:26.298 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:26 +0000 (0:00:00.054) 0:02:03.278 ***** 2021-01-06T21:59:26.304 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-01-06T21:59:26.306 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:26.345 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:26 +0000 (0:00:00.047) 0:02:03.325 ***** 2021-01-06T21:59:26.382 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-01-06T21:59:26.384 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:26.418 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:26 +0000 (0:00:00.073) 0:02:03.399 ***** 2021-01-06T21:59:26.519 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.088200", "end": "2021-01-06 21:59:26.499175", "rc": 0, "start": "2021-01-06 21:59:26.410975", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:26.569 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:26 +0000 (0:00:00.150) 0:02:03.549 ***** 2021-01-06T21:59:26.575 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-01-06T21:59:26.576 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-06T21:59:26.614 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:26 +0000 (0:00:00.045) 0:02:03.595 ***** 2021-01-06T21:59:26.623 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-01-06T21:59:26.665 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:26 +0000 (0:00:00.051) 0:02:03.646 ***** 2021-01-06T21:59:26.672 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-01-06T21:59:26.674 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'block_used': 13754039, 'uuid': '03d36009-ad2e-4bf1-9745-015a33230619', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226341069, 'size_available': 927093018624, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226341069, "block_size": 4096, "block_total": 240095108, "block_used": 13754039, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927093018624, "size_total": 983429562368, "uuid": "03d36009-ad2e-4bf1-9745-015a33230619"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:26.718 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:26 +0000 (0:00:00.049) 0:02:03.696 ***** 2021-01-06T21:59:26.725 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-01-06T21:59:26.726 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:26.766 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:26 +0000 (0:00:00.050) 0:02:03.747 ***** 2021-01-06T21:59:26.773 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-01-06T21:59:26.774 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:26.813 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:26 +0000 (0:00:00.047) 0:02:03.794 ***** 2021-01-06T21:59:26.821 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:26.858 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:26 +0000 (0:00:00.044) 0:02:03.839 ***** 2021-01-06T21:59:26.864 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-06T21:59:26.866 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:26.907 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:26 +0000 (0:00:00.048) 0:02:03.888 ***** 2021-01-06T21:59:26.914 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-01-06T21:59:26.916 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:26.971 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:26 +0000 (0:00:00.064) 0:02:03.952 ***** 2021-01-06T21:59:26.978 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-06T21:59:26.980 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:27.018 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:27 +0000 (0:00:00.046) 0:02:03.999 ***** 2021-01-06T21:59:27.025 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:27.068 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:27 +0000 (0:00:00.049) 0:02:04.049 ***** 2021-01-06T21:59:27.076 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-01-06T21:59:27.079 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:27.131 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:27 +0000 (0:00:00.063) 0:02:04.112 ***** 2021-01-06T21:59:27.159 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-01-06T21:59:27.160 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:27.215 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:27 +0000 (0:00:00.083) 0:02:04.195 ***** 2021-01-06T21:59:27.224 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-01-06T21:59:27.225 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-06T21:59:27.265 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:27 +0000 (0:00:00.050) 0:02:04.245 ***** 2021-01-06T21:59:27.271 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-01-06T21:59:27.273 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:27.313 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:27 +0000 (0:00:00.048) 0:02:04.294 ***** 2021-01-06T21:59:27.320 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-01-06T21:59:27.322 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'block_used': 13754046, 'uuid': '03d36009-ad2e-4bf1-9745-015a33230619', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226341062, 'size_available': 927092989952, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226341062, "block_size": 4096, "block_total": 240095108, "block_used": 13754046, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927092989952, "size_total": 983429562368, "uuid": "03d36009-ad2e-4bf1-9745-015a33230619"}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:27.363 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:27 +0000 (0:00:00.049) 0:02:04.343 ***** 2021-01-06T21:59:27.390 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-01-06T21:59:27.392 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:27.431 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:27 +0000 (0:00:00.068) 0:02:04.412 ***** 2021-01-06T21:59:27.458 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-06T21:59:27.460 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:27.495 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:27 +0000 (0:00:00.063) 0:02:04.475 ***** 2021-01-06T21:59:27.522 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-06T21:59:27.524 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:27.562 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:27 +0000 (0:00:00.067) 0:02:04.543 ***** 2021-01-06T21:59:27.592 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-01-06T21:59:27.593 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:27.629 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:27 +0000 (0:00:00.067) 0:02:04.610 ***** 2021-01-06T21:59:27.657 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-01-06T21:59:27.658 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:27.696 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:27 +0000 (0:00:00.067) 0:02:04.677 ***** 2021-01-06T21:59:27.724 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-01-06T21:59:27.725 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:27.760 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:27 +0000 (0:00:00.063) 0:02:04.741 ***** 2021-01-06T21:59:27.945 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-01-06T21:59:27.983 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:27 +0000 (0:00:00.222) 0:02:04.964 ***** 2021-01-06T21:59:28.105 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-01-06T21:59:28.143 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:28 +0000 (0:00:00.160) 0:02:05.124 ***** 2021-01-06T21:59:28.539 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-01-06T21:59:28.541 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-06T21:59:28.572 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-06T21:59:28.784 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-06T21:59:28.817 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-06T21:59:29.016 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-06T21:59:29.049 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-06T21:59:29.270 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-06T21:59:29.303 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-06T21:59:29.527 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-06T21:59:29.567 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:29 +0000 (0:00:01.423) 0:02:06.547 ***** 2021-01-06T21:59:29.573 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-06T21:59:29.616 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:29 +0000 (0:00:00.049) 0:02:06.597 ***** 2021-01-06T21:59:29.695 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-01-06T21:59:29.697 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-01-06T21:59:29.755 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:29 +0000 (0:00:00.138) 0:02:06.735 ***** 2021-01-06T21:59:29.761 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-01-06T21:59:29.798 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:29 +0000 (0:00:00.043) 0:02:06.779 ***** 2021-01-06T21:59:29.899 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-01-06T21:59:29.902 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-01-06T21:59:29.953 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-01-06T21:59:30.016 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-01-06T21:59:30.211 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-01-06T21:59:30.265 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-01-06T21:59:30.307 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-01-06T21:59:30.371 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-01-06T21:59:30.474 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:30.478 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-01-06T21:59:30.478 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:30.515 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:30 +0000 (0:00:00.716) 0:02:07.496 ***** 2021-01-06T21:59:30.549 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:30 +0000 (0:00:00.034) 0:02:07.530 ***** 2021-01-06T21:59:30.557 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-01-06T21:59:30.558 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:30.592 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:30 +0000 (0:00:00.042) 0:02:07.573 ***** 2021-01-06T21:59:30.599 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:30.635 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:30 +0000 (0:00:00.042) 0:02:07.616 ***** 2021-01-06T21:59:30.641 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-01-06T21:59:30.642 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:30.678 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:30 +0000 (0:00:00.043) 0:02:07.659 ***** 2021-01-06T21:59:30.684 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:30.723 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:30 +0000 (0:00:00.044) 0:02:07.704 ***** 2021-01-06T21:59:30.803 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-01-06T21:59:30.805 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-01-06T21:59:30.839 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:30 +0000 (0:00:00.115) 0:02:07.820 ***** 2021-01-06T21:59:30.855 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-01-06T21:59:30.889 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:30 +0000 (0:00:00.050) 0:02:07.870 ***** 2021-01-06T21:59:30.895 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-01-06T21:59:30.897 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-01-06T21:59:30.932 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:30 +0000 (0:00:00.043) 0:02:07.913 ***** 2021-01-06T21:59:30.938 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-01-06T21:59:30.973 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:30 +0000 (0:00:00.040) 0:02:07.954 ***** 2021-01-06T21:59:31.281 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi203.front.sepia.ceph.com] => {"changed": true} changed: [smithi091.front.sepia.ceph.com] => {"changed": true} 2021-01-06T21:59:31.318 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:31 +0000 (0:00:00.344) 0:02:08.299 ***** 2021-01-06T21:59:31.356 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:31 +0000 (0:00:00.037) 0:02:08.337 ***** 2021-01-06T21:59:31.655 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-01-06T21:59:31.657 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-01-06T21:59:31.690 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:31 +0000 (0:00:00.334) 0:02:08.671 ***** 2021-01-06T21:59:31.721 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:31 +0000 (0:00:00.030) 0:02:08.702 ***** 2021-01-06T21:59:31.727 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:31.765 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:31 +0000 (0:00:00.043) 0:02:08.746 ***** 2021-01-06T21:59:31.771 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:31.807 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:31 +0000 (0:00:00.042) 0:02:08.788 ***** 2021-01-06T21:59:31.813 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-01-06T21:59:31.814 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:31.851 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:31 +0000 (0:00:00.043) 0:02:08.832 ***** 2021-01-06T21:59:31.858 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:31.895 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:31 +0000 (0:00:00.043) 0:02:08.876 ***** 2021-01-06T21:59:31.901 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-01-06T21:59:31.902 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:31.939 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:31 +0000 (0:00:00.044) 0:02:08.920 ***** 2021-01-06T21:59:31.945 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:31.982 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:31 +0000 (0:00:00.041) 0:02:08.962 ***** 2021-01-06T21:59:31.987 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-01-06T21:59:31.989 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:32.023 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:32 +0000 (0:00:00.041) 0:02:09.004 ***** 2021-01-06T21:59:32.029 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:32.066 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:32 +0000 (0:00:00.043) 0:02:09.047 ***** 2021-01-06T21:59:32.166 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-01-06T21:59:32.168 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-01-06T21:59:32.203 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:32 +0000 (0:00:00.136) 0:02:09.184 ***** 2021-01-06T21:59:32.220 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-01-06T21:59:32.257 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:32 +0000 (0:00:00.054) 0:02:09.238 ***** 2021-01-06T21:59:32.336 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-01-06T21:59:32.338 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-01-06T21:59:32.386 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:32 +0000 (0:00:00.128) 0:02:09.367 ***** 2021-01-06T21:59:32.407 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-01-06T21:59:32.444 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:32 +0000 (0:00:00.058) 0:02:09.425 ***** 2021-01-06T21:59:32.700 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-01-06T21:59:32.703 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-01-06T21:59:32.736 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:32 +0000 (0:00:00.292) 0:02:09.717 ***** 2021-01-06T21:59:32.765 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-01-06T21:59:32.804 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:32 +0000 (0:00:00.067) 0:02:09.785 ***** 2021-01-06T21:59:33.041 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-01-06T21:59:33.042 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-01-06T21:59:33.078 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:33 +0000 (0:00:00.273) 0:02:10.059 ***** 2021-01-06T21:59:33.117 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-01-06T21:59:33.154 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:33 +0000 (0:00:00.075) 0:02:10.134 ***** 2021-01-06T21:59:33.980 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-01-06T21:59:33.981 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"cache_update_time": 1609970065, "cache_updated": false, "changed": false} 2021-01-06T21:59:34.017 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:34 +0000 (0:00:00.863) 0:02:10.998 ***** 2021-01-06T21:59:34.055 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"cache_update_time": 1609970019, "cache_updated": false, "changed": false} 2021-01-06T21:59:34.093 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:34 +0000 (0:00:00.075) 0:02:11.074 ***** 2021-01-06T21:59:34.199 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-01-06T21:59:34.201 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.048474", "end": "2021-01-06 21:59:34.176970", "rc": 0, "start": "2021-01-06 21:59:34.128496", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2021-01-06T21:59:34.235 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:34 +0000 (0:00:00.142) 0:02:11.216 ***** 2021-01-06T21:59:34.262 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-01-06T21:59:34.264 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.043163", "end": "2021-01-06 21:59:34.257063", "rc": 0, "start": "2021-01-06 21:59:34.213900", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2021-01-06T21:59:34.301 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:34 +0000 (0:00:00.066) 0:02:11.282 ***** 2021-01-06T21:59:34.334 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:34 +0000 (0:00:00.032) 0:02:11.315 ***** 2021-01-06T21:59:34.339 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-01-06T21:59:34.340 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:34.376 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:34 +0000 (0:00:00.042) 0:02:11.357 ***** 2021-01-06T21:59:34.383 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-01-06T21:59:34.385 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:34.420 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:34 +0000 (0:00:00.044) 0:02:11.401 ***** 2021-01-06T21:59:34.447 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-01-06T21:59:34.449 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:34.483 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:34 +0000 (0:00:00.062) 0:02:11.464 ***** 2021-01-06T21:59:35.257 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** ok: [smithi091.front.sepia.ceph.com] => {"cache_update_time": 1609970065, "cache_updated": false, "changed": false} 2021-01-06T21:59:35.294 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:35 +0000 (0:00:00.810) 0:02:12.274 ***** 2021-01-06T21:59:35.321 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"cache_update_time": 1609970019, "cache_updated": false, "changed": false} 2021-01-06T21:59:35.323 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:35.358 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:35 +0000 (0:00:00.064) 0:02:12.339 ***** 2021-01-06T21:59:35.392 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:35 +0000 (0:00:00.033) 0:02:12.372 ***** 2021-01-06T21:59:35.419 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:35.455 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:35 +0000 (0:00:00.062) 0:02:12.435 ***** 2021-01-06T21:59:35.648 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-01-06T21:59:35.650 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-01-06T21:59:35.688 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:35 +0000 (0:00:00.233) 0:02:12.669 ***** 2021-01-06T21:59:35.715 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-01-06T21:59:35.717 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:35.755 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:35 +0000 (0:00:00.067) 0:02:12.736 ***** 2021-01-06T21:59:35.761 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-01-06T21:59:35.763 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-01-06T21:59:35.802 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:35 +0000 (0:00:00.046) 0:02:12.783 ***** 2021-01-06T21:59:35.829 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-01-06T21:59:35.831 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:35.866 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:35 +0000 (0:00:00.063) 0:02:12.847 ***** 2021-01-06T21:59:35.955 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-01-06T21:59:35.956 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Wed 2021-01-06 21:57:10 UTC", "ActiveEnterTimestampMonotonic": "26957642", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "system.slice systemd-journald.socket -.mount network.target basic.target systemd-tmpfiles-setup.service sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Wed 2021-01-06 21:57:03 UTC", "AssertTimestampMonotonic": "20364184", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Wed 2021-01-06 21:57:03 UTC", "ConditionTimestampMonotonic": "20364183", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2021-01-06T21:59:35.958 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1742", "ExecMainStartTimestamp": "Wed 2021-01-06 21:57:10 UTC", "ExecMainStartTimestampMonotonic": "26957628", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Wed 2021-01-06 21:57:03 UTC", "InactiveExitTimestampMonotonic": "20365807", "InvocationID": "286f8e3053c2434d9fc2023484c50985", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128028", "LimitNPROCSoft": "128028", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2021-01-06T21:59:35.960 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1742", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Wed 2021-01-06 21:57:10 UTC", "S 2021-01-06T21:59:35.961 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "26957642", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Wed 2021-01-06 21:57:10 UTC", "WatchdogTimestampMonotonic": "26957640", "WatchdogUSec": "0"}} 2021-01-06T21:59:35.996 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:35 +0000 (0:00:00.129) 0:02:12.977 ***** 2021-01-06T21:59:36.023 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-06T21:59:36.025 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-01-06T21:59:36.058 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:36 +0000 (0:00:00.062) 0:02:13.039 ***** 2021-01-06T21:59:36.075 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-01-06T21:59:36.076 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Wed 2021-01-06 21:57:05 UTC", "ActiveEnterTimestampMonotonic": "25341572", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "system.slice systemd-journald.socket systemd-tmpfiles-setup.service network.target basic.target sysinit.target -.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Wed 2021-01-06 21:56:58 UTC", "AssertTimestampMonotonic": "18832170", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Wed 2021-01-06 21:56:58 UTC", "ConditionTimestampMonotonic": "18832170", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2021-01-06T21:59:36.078 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1761", "ExecMainStartTimestamp": "Wed 2021-01-06 21:57:05 UTC", "ExecMainStartTimestampMonotonic": "25341558", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Wed 2021-01-06 21:56:58 UTC", "InactiveExitTimestampMonotonic": "18832859", "InvocationID": "c166f6e4e9d44e7cabdb369fce1de41e", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128028", "LimitNPROCSoft": "128028", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2021-01-06T21:59:36.079 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1761", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Wed 2021-01-06 21:57:05 UTC", "S 2021-01-06T21:59:36.081 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "25341572", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Wed 2021-01-06 21:57:05 UTC", "WatchdogTimestampMonotonic": "25341570", "WatchdogUSec": "0"}} 2021-01-06T21:59:36.112 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:36 +0000 (0:00:00.054) 0:02:13.093 ***** 2021-01-06T21:59:36.118 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-06T21:59:36.120 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:36.154 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:36 +0000 (0:00:00.041) 0:02:13.135 ***** 2021-01-06T21:59:36.160 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-06T21:59:36.161 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-01-06T21:59:36.195 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:36 +0000 (0:00:00.041) 0:02:13.176 ***** 2021-01-06T21:59:36.201 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:36.237 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:36 +0000 (0:00:00.042) 0:02:13.218 ***** 2021-01-06T21:59:36.243 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-06T21:59:36.245 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:36.282 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:36 +0000 (0:00:00.044) 0:02:13.262 ***** 2021-01-06T21:59:36.308 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-01-06T21:59:36.310 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:36.344 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:36 +0000 (0:00:00.062) 0:02:13.325 ***** 2021-01-06T21:59:37.127 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* ok: [smithi091.front.sepia.ceph.com] => {"cache_update_time": 1609970065, "cache_updated": false, "changed": false} 2021-01-06T21:59:37.165 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:37 +0000 (0:00:00.820) 0:02:14.146 ***** 2021-01-06T21:59:37.193 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-01-06T21:59:37.195 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:37.230 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:37 +0000 (0:00:00.064) 0:02:14.211 ***** 2021-01-06T21:59:37.236 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-01-06T21:59:37.238 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"cache_update_time": 1609970019, "cache_updated": false, "changed": false} 2021-01-06T21:59:37.275 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:37 +0000 (0:00:00.045) 0:02:14.256 ***** 2021-01-06T21:59:37.302 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-01-06T21:59:37.303 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:37.345 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:37 +0000 (0:00:00.069) 0:02:14.325 ***** 2021-01-06T21:59:37.350 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-01-06T21:59:37.352 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-01-06T21:59:37.388 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:37 +0000 (0:00:00.043) 0:02:14.369 ***** 2021-01-06T21:59:37.469 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-01-06T21:59:37.508 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:37 +0000 (0:00:00.119) 0:02:14.489 ***** 2021-01-06T21:59:37.702 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-01-06T21:59:37.704 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-01-06T21:59:37.740 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:37 +0000 (0:00:00.231) 0:02:14.720 ***** 2021-01-06T21:59:37.936 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-01-06T21:59:37.999 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:37 +0000 (0:00:00.259) 0:02:14.980 ***** 2021-01-06T21:59:38.743 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-01-06T21:59:38.782 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:38 +0000 (0:00:00.782) 0:02:15.763 ***** 2021-01-06T21:59:38.789 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-01-06T21:59:38.830 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:38 +0000 (0:00:00.047) 0:02:15.810 ***** 2021-01-06T21:59:39.286 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-01-06T21:59:39.288 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false} 2021-01-06T21:59:39.323 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:39 +0000 (0:00:00.492) 0:02:16.303 ***** 2021-01-06T21:59:39.350 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-01-06T21:59:39.352 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2021-01-06T21:59:39.387 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:39 +0000 (0:00:00.064) 0:02:16.368 ***** 2021-01-06T21:59:39.393 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-01-06T21:59:39.395 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false} 2021-01-06T21:59:39.432 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:39 +0000 (0:00:00.044) 0:02:16.413 ***** 2021-01-06T21:59:39.470 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-01-06T21:59:39.472 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2021-01-06T21:59:39.505 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:39 +0000 (0:00:00.073) 0:02:16.486 ***** 2021-01-06T21:59:39.523 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-01-06T21:59:39.523 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1609969974.6558623, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1609969974.6518624, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 10259, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1609969974.6518624, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "4291540836", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-01-06T21:59:39.563 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:39 +0000 (0:00:00.057) 0:02:16.543 ***** 2021-01-06T21:59:39.646 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1609970000.8404007, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1609970000.8364005, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 10259, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1609970000.8364005, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "4291540836", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-01-06T21:59:39.687 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:39 +0000 (0:00:00.123) 0:02:16.667 ***** 2021-01-06T21:59:39.811 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-01-06T21:59:39.814 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-01-06T21:59:39.851 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:39 +0000 (0:00:00.164) 0:02:16.832 ***** 2021-01-06T21:59:39.859 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-01-06T21:59:39.895 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:39 +0000 (0:00:00.044) 0:02:16.876 ***** 2021-01-06T21:59:39.903 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-01-06T21:59:39.939 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:39 +0000 (0:00:00.043) 0:02:16.920 ***** 2021-01-06T21:59:39.941 INFO:teuthology.task.ansible.out: 2021-01-06T21:59:39.953 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-01-06T21:59:39.987 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:39 +0000 (0:00:00.047) 0:02:16.968 ***** 2021-01-06T21:59:39.995 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-01-06T21:59:39.997 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.032 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:40 +0000 (0:00:00.044) 0:02:17.013 ***** 2021-01-06T21:59:40.040 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.078 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:40 +0000 (0:00:00.044) 0:02:17.058 ***** 2021-01-06T21:59:40.157 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-01-06T21:59:40.159 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-01-06T21:59:40.199 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:40 +0000 (0:00:00.121) 0:02:17.179 ***** 2021-01-06T21:59:40.217 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-01-06T21:59:40.254 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:40 +0000 (0:00:00.055) 0:02:17.235 ***** 2021-01-06T21:59:40.323 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-01-06T21:59:40.325 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-01-06T21:59:40.361 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:40 +0000 (0:00:00.106) 0:02:17.341 ***** 2021-01-06T21:59:40.381 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-01-06T21:59:40.384 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.423 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:40 +0000 (0:00:00.061) 0:02:17.403 ***** 2021-01-06T21:59:40.461 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:40 +0000 (0:00:00.038) 0:02:17.442 ***** 2021-01-06T21:59:40.468 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-01-06T21:59:40.470 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.481 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.484 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.521 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:40 +0000 (0:00:00.060) 0:02:17.502 ***** 2021-01-06T21:59:40.528 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-01-06T21:59:40.529 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.570 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:40 +0000 (0:00:00.048) 0:02:17.551 ***** 2021-01-06T21:59:40.582 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-01-06T21:59:40.584 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.585 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.585 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditiona 2021-01-06T21:59:40.587 INFO:teuthology.task.ansible.out:l result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.642 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:40 +0000 (0:00:00.071) 0:02:17.623 ***** 2021-01-06T21:59:40.649 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-01-06T21:59:40.650 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.650 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.651 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.688 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:40 +0000 (0:00:00.045) 0:02:17.669 ***** 2021-01-06T21:59:40.693 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-06T21:59:40.695 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.737 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:40 +0000 (0:00:00.049) 0:02:17.718 ***** 2021-01-06T21:59:40.745 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-01-06T21:59:40.747 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.749 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi203.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.789 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:40 +0000 (0:00:00.050) 0:02:17.769 ***** 2021-01-06T21:59:40.795 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-01-06T21:59:40.797 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.831 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:40 +0000 (0:00:00.042) 0:02:17.812 ***** 2021-01-06T21:59:40.837 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-06T21:59:40.839 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.900 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:40 +0000 (0:00:00.068) 0:02:17.881 ***** 2021-01-06T21:59:40.909 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-01-06T21:59:40.911 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:40.965 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:40 +0000 (0:00:00.065) 0:02:17.946 ***** 2021-01-06T21:59:40.973 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-01-06T21:59:40.975 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:41.010 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:41 +0000 (0:00:00.045) 0:02:17.991 ***** 2021-01-06T21:59:41.017 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-01-06T21:59:41.018 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:41.055 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:41 +0000 (0:00:00.044) 0:02:18.035 ***** 2021-01-06T21:59:41.061 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-01-06T21:59:41.062 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:41.103 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:41 +0000 (0:00:00.048) 0:02:18.084 ***** 2021-01-06T21:59:41.110 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-01-06T21:59:41.112 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:41.162 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:41 +0000 (0:00:00.058) 0:02:18.143 ***** 2021-01-06T21:59:41.171 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-01-06T21:59:41.173 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:41.208 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:41 +0000 (0:00:00.046) 0:02:18.189 ***** 2021-01-06T21:59:41.214 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-01-06T21:59:41.215 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:41.252 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:41 +0000 (0:00:00.043) 0:02:18.232 ***** 2021-01-06T21:59:41.258 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-01-06T21:59:41.259 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:41.301 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:41 +0000 (0:00:00.049) 0:02:18.282 ***** 2021-01-06T21:59:41.309 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-01-06T21:59:41.312 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:41.353 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:41 +0000 (0:00:00.052) 0:02:18.334 ***** 2021-01-06T21:59:41.360 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-01-06T21:59:41.362 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:41.402 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:41 +0000 (0:00:00.049) 0:02:18.383 ***** 2021-01-06T21:59:41.409 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-01-06T21:59:41.411 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:41.449 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:41 +0000 (0:00:00.046) 0:02:18.430 ***** 2021-01-06T21:59:41.457 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-01-06T21:59:41.458 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:41.500 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:41 +0000 (0:00:00.050) 0:02:18.481 ***** 2021-01-06T21:59:41.529 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-01-06T21:59:41.532 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:41.569 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:41 +0000 (0:00:00.068) 0:02:18.550 ***** 2021-01-06T21:59:41.607 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-01-06T21:59:41.608 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.010824", "end": "2021-01-06 21:59:41.569353", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-01-06 21:59:41.558529", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:41.722 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.011123", "end": "2021-01-06 21:59:41.698200", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-01-06 21:59:41.687077", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:41.724 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006015", "end": "2021-01-06 21:59:41.696986", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-01-06 21:59:41.690971", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:41.838 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006031", "end": "2021-01-06 21:59:41.818553", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-01-06 21:59:41.812522", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:41.840 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005481", "end": "2021-01-06 21:59:41.810621", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-01-06 21:59:41.805140", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:41.945 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005515", "end": "2021-01-06 21:59:41.930739", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-01-06 21:59:41.925224", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:41.947 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.007363", "end": "2021-01-06 21:59:41.924634", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-01-06 21:59:41.917271", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:41.984 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:41 +0000 (0:00:00.414) 0:02:18.965 ***** 2021-01-06T21:59:42.012 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-01-06T21:59:42.047 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:42 +0000 (0:00:00.063) 0:02:19.028 ***** 2021-01-06T21:59:42.066 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-01-06T21:59:42.068 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.006717", "end": "2021-01-06 21:59:42.041822", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-01-06 21:59:42.035105", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:42.105 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:42 +0000 (0:00:00.057) 0:02:19.085 ***** 2021-01-06T21:59:42.131 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-01-06T21:59:42.168 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:42 +0000 (0:00:00.063) 0:02:19.149 ***** 2021-01-06T21:59:42.174 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-01-06T21:59:42.176 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-01-06T21:59:42.278 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-01-06T21:59:42.387 INFO:teuthology.task.ansible.out: PLAY [testnodes,senta,vossi,jenkins_slaves] ************************************ 2021-01-06T21:59:42.445 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:42 +0000 (0:00:00.277) 0:02:19.426 ***** 2021-01-06T21:59:42.485 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:42 +0000 (0:00:00.039) 0:02:19.466 ***** 2021-01-06T21:59:43.171 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi091.front.sepia.ceph.com] 2021-01-06T21:59:43.221 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] 2021-01-06T21:59:43.262 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:43 +0000 (0:00:00.776) 0:02:20.243 ***** 2021-01-06T21:59:43.298 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-01-06T21:59:43.335 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:43 +0000 (0:00:00.072) 0:02:20.316 ***** 2021-01-06T21:59:43.370 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:43 +0000 (0:00:00.034) 0:02:20.351 ***** 2021-01-06T21:59:43.376 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-01-06T21:59:43.379 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:43.415 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:43 +0000 (0:00:00.045) 0:02:20.396 ***** 2021-01-06T21:59:43.421 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-01-06T21:59:43.424 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-01-06T21:59:43.461 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:43 +0000 (0:00:00.045) 0:02:20.442 ***** 2021-01-06T21:59:43.467 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-01-06T21:59:43.469 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-01-06T21:59:43.509 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:43 +0000 (0:00:00.047) 0:02:20.490 ***** 2021-01-06T21:59:43.515 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-01-06T21:59:43.517 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:43.557 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:43 +0000 (0:00:00.047) 0:02:20.537 ***** 2021-01-06T21:59:43.594 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-01-06T21:59:43.597 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-01-06T21:59:43.630 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:43 +0000 (0:00:00.073) 0:02:20.611 ***** 2021-01-06T21:59:44.414 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** ok: [smithi091.front.sepia.ceph.com] => {"cache_update_time": 1609970065, "cache_updated": false, "changed": false} 2021-01-06T21:59:44.453 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:44 +0000 (0:00:00.821) 0:02:21.433 ***** 2021-01-06T21:59:44.479 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-01-06T21:59:44.481 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:44.517 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:44 +0000 (0:00:00.065) 0:02:21.498 ***** 2021-01-06T21:59:44.534 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-01-06T21:59:44.536 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"cache_update_time": 1609970019, "cache_updated": false, "changed": false} 2021-01-06T21:59:44.574 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:44 +0000 (0:00:00.056) 0:02:21.555 ***** 2021-01-06T21:59:44.579 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-01-06T21:59:44.582 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-01-06T21:59:44.623 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:44 +0000 (0:00:00.048) 0:02:21.604 ***** 2021-01-06T21:59:44.629 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-01-06T21:59:44.631 INFO:teuthology.task.ansible.out:skipping: [smithi203.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-06T21:59:44.668 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:44 +0000 (0:00:00.045) 0:02:21.649 ***** 2021-01-06T21:59:44.707 INFO:teuthology.task.ansible.out: 2021-01-06T21:59:44.709 INFO:teuthology.task.ansible.out:TASK [container-host : set_fact] *********************************************** ok: [smithi203.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-01-06T21:59:44.760 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:44 +0000 (0:00:00.091) 0:02:21.741 ***** 2021-01-06T21:59:44.778 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000] *** changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-01-06T21:59:44.824 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:44 +0000 (0:00:00.063) 0:02:21.805 ***** 2021-01-06T21:59:44.895 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-01-06T21:59:44.933 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:44 +0000 (0:00:00.108) 0:02:21.914 ***** 2021-01-06T21:59:45.148 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "checksum": "ad23d23287af05eb6dfc6847a1957988954f5ccf", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "97bf7a9f95b52e04e40f96163cd1717c", "mode": "0644", "owner": "root", "size": 2134, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1609970384.8592873-239366307603057/source", "state": "file", "uid": 0} 2021-01-06T21:59:45.186 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:45 +0000 (0:00:00.253) 0:02:22.167 ***** 2021-01-06T21:59:45.266 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "checksum": "ad23d23287af05eb6dfc6847a1957988954f5ccf", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "97bf7a9f95b52e04e40f96163cd1717c", "mode": "0644", "owner": "root", "size": 2134, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1609970384.9612806-73298154481126/source", "state": "file", "uid": 0} 2021-01-06T21:59:45.304 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:45 +0000 (0:00:00.117) 0:02:22.285 ***** 2021-01-06T21:59:48.651 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-01-06T21:59:48.652 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-TgYM8S-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-TgYM8S-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packag 2021-01-06T21:59:48.654 INFO:teuthology.task.ansible.out:es/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-01-06T21:59:48.688 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:48 +0000 (0:00:03.383) 0:02:25.669 ***** 2021-01-06T21:59:48.819 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-01-06T21:59:48.821 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-01-06T21:59:48.857 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:48 +0000 (0:00:00.169) 0:02:25.838 ***** 2021-01-06T21:59:49.020 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-01-06T21:59:49.022 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-92XOsG-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-92XOsG-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packag 2021-01-06T21:59:49.024 INFO:teuthology.task.ansible.out:es/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-01-06T21:59:49.059 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:49 +0000 (0:00:00.201) 0:02:26.040 ***** 2021-01-06T21:59:49.180 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2021-01-06T21:59:49.181 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1609970388.8908653-171162370683890/source", "state": "file", "uid": 0} 2021-01-06T21:59:49.217 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:49 +0000 (0:00:00.158) 0:02:26.198 ***** 2021-01-06T21:59:49.223 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-01-06T21:59:49.225 INFO:teuthology.task.ansible.out:ok: [smithi203.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-01-06T21:59:49.263 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:49 +0000 (0:00:00.045) 0:02:26.243 ***** 2021-01-06T21:59:49.467 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-01-06T21:59:49.469 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.119002", "end": "2021-01-06 21:59:49.451984", "rc": 0, "start": "2021-01-06 21:59:49.332982", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:49.514 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:49 +0000 (0:00:00.250) 0:02:26.495 ***** 2021-01-06T21:59:49.754 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2021-01-06T21:59:49.755 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1609970389.2974677-131588072983984/source", "state": "file", "uid": 0} 2021-01-06T21:59:49.791 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:49 +0000 (0:00:00.277) 0:02:26.772 ***** 2021-01-06T21:59:50.084 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-01-06T21:59:50.086 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.123242", "end": "2021-01-06 21:59:50.075351", "rc": 0, "start": "2021-01-06 21:59:49.952109", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-06T21:59:50.148 INFO:teuthology.task.ansible.out:Wednesday 06 January 2021 21:59:50 +0000 (0:00:00.357) 0:02:27.129 ***** 2021-01-06T21:59:52.801 INFO:teuthology.task.ansible.out: TASK [container-host : Restart docker service] ********************************* 2021-01-06T21:59:52.803 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "network-online.target systemd-journald.socket containerd.service firewalld.service basic.target system.slice sysinit.target docker.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BindsTo": "containerd.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": 2021-01-06T21:59:52.804 INFO:teuthology.task.ansible.out: "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "12802 2021-01-06T21:59:52.806 INFO:teuthology.task.ansible.out:8", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "docker.socket sysinit.target system.slice", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Wed 2021-01-06 21:58:48 UTC", "StateChangeTimestampMonotonic": "125385242", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneed 2021-01-06T21:59:52.807 INFO:teuthology.task.ansible.out:ed": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-online.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-01-06T21:59:53.739 INFO:teuthology.task.ansible.out:changed: [smithi203.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "containerd.service basic.target firewalld.service docker.socket network-online.target systemd-journald.socket sysinit.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BindsTo": "containerd.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": 2021-01-06T21:59:53.741 INFO:teuthology.task.ansible.out: "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "12802 2021-01-06T21:59:53.743 INFO:teuthology.task.ansible.out:8", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "docker.socket sysinit.target system.slice", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Wed 2021-01-06 21:58:48 UTC", "StateChangeTimestampMonotonic": "129298937", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneed 2021-01-06T21:59:53.744 INFO:teuthology.task.ansible.out:ed": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-online.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-01-06T21:59:53.840 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2021-01-06T21:59:53.845 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2021-01-06T21:59:53.849 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2021-01-06T21:59:53.850 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi091.front.sepia.ceph.com : ok=126 changed=36 unreachable=0 failed=0 skipped=183 rescued=0 ignored=0 smithi203.front.sepia.ceph.com : ok=126 changed=36 unreachable=0 failed=0 skipped=183 rescued=0 ignored=0 Wednesday 06 January 2021 21:59:53 +0000 (0:00:03.701) 0:02:30.830 ***** =============================================================================== users : Create all admin users with sudo access. ----------------------- 17.79s users : Update authorized_keys using the keys repo --------------------- 16.66s testnode : Ensure packages are not present. ----------------------------- 9.29s testnode : Zap all non-root disks --------------------------------------- 8.78s testnode : Install packages --------------------------------------------- 8.41s users : Remove revoked users -------------------------------------------- 5.49s common : Update apt cache ----------------------------------------------- 3.96s container-host : Restart docker service --------------------------------- 3.70s container-host : Install registries-conf-ctl ---------------------------- 3.38s testnode : Update apt cache. -------------------------------------------- 2.91s testnode : Install apt keys --------------------------------------------- 2.26s Gathering Facts --------------------------------------------------------- 1.64s testnode : Blow away lingering OSD data and FSIDs ----------------------- 1.43s testnode : Create logical volume(s) ------------------------------------- 1.42s users : Update authorized_keys for each user with literal keys ---------- 1.31s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 0.93s testnode : Install python-apt ------------------------------------------- 0.90s testnode : Ensure perl-doc is installed on apt systems. ----------------- 0.86s container-host : Install container 2021-01-06T21:59:53.852 INFO:teuthology.task.ansible.out: packages ----------------------------- 0.82s testnode : Install python-pip on apt based systems. --------------------- 0.82s 2021-01-06T21:59:54.152 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi203.front.sepia.ceph.com'), Remote(name='ubuntu@smithi091.front.sepia.ceph.com')] 2021-01-06T21:59:54.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-06T21:59:54.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:59:54.229 DEBUG:teuthology.orchestra.run.smithi203:> true 2021-01-06T21:59:54.427 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-06T21:59:54.428 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-06T21:59:54.549 DEBUG:teuthology.orchestra.run.smithi091:> true 2021-01-06T21:59:54.755 INFO:teuthology.run_tasks:Running task clock... 2021-01-06T21:59:54.766 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2021-01-06T21:59:54.766 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-01-06T21:59:54.767 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-01-06T21:59:54.840 INFO:teuthology.orchestra.run.smithi203.stdout: 6 Jan 21:59:54 ntpd[8278]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-01-06T21:59:54.840 INFO:teuthology.orchestra.run.smithi203.stdout: 6 Jan 21:59:54 ntpd[8278]: Command line: ntpd -gq 2021-01-06T21:59:54.842 INFO:teuthology.orchestra.run.smithi203.stdout: 6 Jan 21:59:54 ntpd[8278]: proto: precision = 0.120 usec (-23) 2021-01-06T21:59:54.842 INFO:teuthology.orchestra.run.smithi203.stdout: 6 Jan 21:59:54 ntpd[8278]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-01-06T21:59:54.842 INFO:teuthology.orchestra.run.smithi203.stdout: 6 Jan 21:59:54 ntpd[8278]: restrict ::: KOD does nothing without LIMITED. 2021-01-06T21:59:54.842 INFO:teuthology.orchestra.run.smithi203.stdout: 6 Jan 21:59:54 ntpd[8278]: Listen and drop on 0 v6wildcard [::]:123 2021-01-06T21:59:54.843 INFO:teuthology.orchestra.run.smithi203.stdout: 6 Jan 21:59:54 ntpd[8278]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-01-06T21:59:54.843 INFO:teuthology.orchestra.run.smithi203.stdout: 6 Jan 21:59:54 ntpd[8278]: Listen normally on 2 lo 127.0.0.1:123 2021-01-06T21:59:54.843 INFO:teuthology.orchestra.run.smithi203.stdout: 6 Jan 21:59:54 ntpd[8278]: Listen normally on 3 enp3s0f1 172.21.15.203:123 2021-01-06T21:59:54.843 INFO:teuthology.orchestra.run.smithi203.stdout: 6 Jan 21:59:54 ntpd[8278]: Listen normally on 4 lo [::1]:123 2021-01-06T21:59:54.843 INFO:teuthology.orchestra.run.smithi203.stdout: 6 Jan 21:59:54 ntpd[8278]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:4a23%5]:123 2021-01-06T21:59:54.844 INFO:teuthology.orchestra.run.smithi203.stdout: 6 Jan 21:59:54 ntpd[8278]: Listening on routing socket on fd #22 for interface updates 2021-01-06T21:59:54.844 INFO:teuthology.orchestra.run.smithi203.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-01-06T21:59:54.844 INFO:teuthology.orchestra.run.smithi203.stderr:restrict ::: KOD does nothing without LIMITED. 2021-01-06T21:59:55.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:55 ntpd[8278]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T21:59:55.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:55 ntpd[8278]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T21:59:55.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:55 ntpd[8278]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T21:59:55.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:55 ntpd[8278]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T21:59:56.840 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:56 ntpd[8278]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T21:59:56.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:56 ntpd[8278]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T21:59:56.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:56 ntpd[8278]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T21:59:56.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:56 ntpd[8278]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T21:59:57.840 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:57 ntpd[8278]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T21:59:57.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:57 ntpd[8278]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T21:59:57.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:57 ntpd[8278]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T21:59:57.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:57 ntpd[8278]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T21:59:58.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:58 ntpd[8278]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T21:59:58.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:58 ntpd[8278]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T21:59:58.842 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:58 ntpd[8278]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T21:59:58.842 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:58 ntpd[8278]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T21:59:58.842 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:58 ntpd[8278]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T21:59:58.842 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:58 ntpd[8278]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T21:59:58.843 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:58 ntpd[8278]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T21:59:58.843 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:58 ntpd[8278]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T21:59:59.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:59 ntpd[8278]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T21:59:59.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:59 ntpd[8278]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T21:59:59.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:59 ntpd[8278]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T21:59:59.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:59 ntpd[8278]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T21:59:59.842 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:59 ntpd[8278]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T21:59:59.842 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:59 ntpd[8278]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T21:59:59.842 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:59 ntpd[8278]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T21:59:59.842 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 21:59:59 ntpd[8278]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:00.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:00 ntpd[8278]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T22:00:00.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:00 ntpd[8278]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T22:00:00.842 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:00 ntpd[8278]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T22:00:00.842 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:00 ntpd[8278]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:00.842 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:00 ntpd[8278]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T22:00:00.842 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:00 ntpd[8278]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T22:00:00.843 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:00 ntpd[8278]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T22:00:00.844 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:00 ntpd[8278]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:01.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:01 ntpd[8278]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T22:00:01.841 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:01 ntpd[8278]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T22:00:01.842 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:01 ntpd[8278]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T22:00:01.842 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:01 ntpd[8278]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:01.842 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:01 ntpd[8278]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T22:00:01.842 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:01 ntpd[8278]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T22:00:01.843 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:01 ntpd[8278]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T22:00:01.843 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:01 ntpd[8278]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:01.843 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:01 ntpd[8278]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-01-06T22:00:01.843 INFO:teuthology.orchestra.run.smithi203.stderr: 6 Jan 22:00:01 ntpd[8278]: can't open /var/log/ntpstats/loopstats.20210106: Permission denied 2021-01-06T22:00:01.844 INFO:teuthology.orchestra.run.smithi203.stdout: 6 Jan 22:00:01 ntpd[8278]: ntpd: time slew -0.010577 s 2021-01-06T22:00:01.844 INFO:teuthology.orchestra.run.smithi203.stdout:ntpd: time slew -0.010577s 2021-01-06T22:00:01.970 INFO:teuthology.orchestra.run.smithi203.stdout: remote refid st t when poll reach delay offset jitter 2021-01-06T22:00:01.970 INFO:teuthology.orchestra.run.smithi203.stdout:============================================================================== 2021-01-06T22:00:01.971 INFO:teuthology.orchestra.run.smithi203.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-01-06T22:00:01.971 INFO:teuthology.orchestra.run.smithi203.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-01-06T22:00:01.971 INFO:teuthology.orchestra.run.smithi203.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-01-06T22:00:01.972 INFO:teuthology.orchestra.run.smithi203.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-01-06T22:00:01.972 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-01-06T22:00:01.973 DEBUG:teuthology.orchestra.run.smithi091:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-01-06T22:00:02.035 INFO:teuthology.orchestra.run.smithi091.stdout: 6 Jan 22:00:02 ntpd[8242]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-01-06T22:00:02.036 INFO:teuthology.orchestra.run.smithi091.stdout: 6 Jan 22:00:02 ntpd[8242]: Command line: ntpd -gq 2021-01-06T22:00:02.037 INFO:teuthology.orchestra.run.smithi091.stdout: 6 Jan 22:00:02 ntpd[8242]: proto: precision = 0.123 usec (-23) 2021-01-06T22:00:02.037 INFO:teuthology.orchestra.run.smithi091.stdout: 6 Jan 22:00:02 ntpd[8242]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-01-06T22:00:02.038 INFO:teuthology.orchestra.run.smithi091.stdout: 6 Jan 22:00:02 ntpd[8242]: restrict ::: KOD does nothing without LIMITED. 2021-01-06T22:00:02.038 INFO:teuthology.orchestra.run.smithi091.stdout: 6 Jan 22:00:02 ntpd[8242]: Listen and drop on 0 v6wildcard [::]:123 2021-01-06T22:00:02.038 INFO:teuthology.orchestra.run.smithi091.stdout: 6 Jan 22:00:02 ntpd[8242]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-01-06T22:00:02.039 INFO:teuthology.orchestra.run.smithi091.stdout: 6 Jan 22:00:02 ntpd[8242]: Listen normally on 2 lo 127.0.0.1:123 2021-01-06T22:00:02.039 INFO:teuthology.orchestra.run.smithi091.stdout: 6 Jan 22:00:02 ntpd[8242]: Listen normally on 3 enp3s0f1 172.21.15.91:123 2021-01-06T22:00:02.039 INFO:teuthology.orchestra.run.smithi091.stdout: 6 Jan 22:00:02 ntpd[8242]: Listen normally on 4 lo [::1]:123 2021-01-06T22:00:02.040 INFO:teuthology.orchestra.run.smithi091.stdout: 6 Jan 22:00:02 ntpd[8242]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d3d5%5]:123 2021-01-06T22:00:02.040 INFO:teuthology.orchestra.run.smithi091.stdout: 6 Jan 22:00:02 ntpd[8242]: Listening on routing socket on fd #22 for interface updates 2021-01-06T22:00:02.040 INFO:teuthology.orchestra.run.smithi091.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-01-06T22:00:02.041 INFO:teuthology.orchestra.run.smithi091.stderr:restrict ::: KOD does nothing without LIMITED. 2021-01-06T22:00:04.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:04 ntpd[8242]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T22:00:04.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:04 ntpd[8242]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T22:00:04.037 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:04 ntpd[8242]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T22:00:04.037 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:04 ntpd[8242]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:04.037 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:04 ntpd[8242]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T22:00:04.037 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:04 ntpd[8242]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T22:00:04.038 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:04 ntpd[8242]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T22:00:04.038 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:04 ntpd[8242]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:05.035 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:05 ntpd[8242]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T22:00:05.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:05 ntpd[8242]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T22:00:05.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:05 ntpd[8242]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T22:00:05.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:05 ntpd[8242]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:06.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:06 ntpd[8242]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T22:00:06.037 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:06 ntpd[8242]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T22:00:06.037 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:06 ntpd[8242]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T22:00:06.037 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:06 ntpd[8242]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:06.037 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:06 ntpd[8242]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T22:00:06.037 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:06 ntpd[8242]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T22:00:06.038 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:06 ntpd[8242]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T22:00:06.038 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:06 ntpd[8242]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:06.038 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:06 ntpd[8242]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T22:00:06.038 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:06 ntpd[8242]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T22:00:06.039 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:06 ntpd[8242]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T22:00:06.039 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:06 ntpd[8242]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:07.035 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:07 ntpd[8242]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T22:00:07.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:07 ntpd[8242]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T22:00:07.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:07 ntpd[8242]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T22:00:07.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:07 ntpd[8242]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:08.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:08 ntpd[8242]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T22:00:08.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:08 ntpd[8242]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T22:00:08.037 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:08 ntpd[8242]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T22:00:08.037 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:08 ntpd[8242]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:08.037 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:08 ntpd[8242]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T22:00:08.037 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:08 ntpd[8242]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T22:00:08.037 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:08 ntpd[8242]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T22:00:08.038 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:08 ntpd[8242]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:08.038 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:08 ntpd[8242]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T22:00:08.038 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:08 ntpd[8242]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T22:00:08.038 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:08 ntpd[8242]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T22:00:08.039 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:08 ntpd[8242]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:09.035 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:09 ntpd[8242]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T22:00:09.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:09 ntpd[8242]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T22:00:09.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:09 ntpd[8242]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T22:00:09.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:09 ntpd[8242]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:10.035 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:10 ntpd[8242]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-01-06T22:00:10.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:10 ntpd[8242]: can't open /var/log/ntpstats/rawstats.20210106: Permission denied 2021-01-06T22:00:10.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:10 ntpd[8242]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-01-06T22:00:10.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:10 ntpd[8242]: can't open /var/log/ntpstats/peerstats.20210106: Permission denied 2021-01-06T22:00:10.036 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:10 ntpd[8242]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-01-06T22:00:10.037 INFO:teuthology.orchestra.run.smithi091.stderr: 6 Jan 22:00:10 ntpd[8242]: can't open /var/log/ntpstats/loopstats.20210106: Permission denied 2021-01-06T22:00:10.037 INFO:teuthology.orchestra.run.smithi091.stdout: 6 Jan 22:00:10 ntpd[8242]: ntpd: time slew -0.003565 s 2021-01-06T22:00:10.037 INFO:teuthology.orchestra.run.smithi091.stdout:ntpd: time slew -0.003565s 2021-01-06T22:00:10.079 INFO:teuthology.orchestra.run.smithi091.stdout: remote refid st t when poll reach delay offset jitter 2021-01-06T22:00:10.079 INFO:teuthology.orchestra.run.smithi091.stdout:============================================================================== 2021-01-06T22:00:10.080 INFO:teuthology.orchestra.run.smithi091.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-01-06T22:00:10.080 INFO:teuthology.orchestra.run.smithi091.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-01-06T22:00:10.080 INFO:teuthology.orchestra.run.smithi091.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-01-06T22:00:10.080 INFO:teuthology.orchestra.run.smithi091.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-01-06T22:00:10.081 INFO:teuthology.run_tasks:Running task install... 2021-01-06T22:00:10.095 DEBUG:teuthology.task.install:project ceph 2021-01-06T22:00:10.096 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': 'd0ed162b51928c50f20cee111f8292828eda755e'}} 2021-01-06T22:00:10.096 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': 'd0ed162b51928c50f20cee111f8292828eda755e'} 2021-01-06T22:00:10.097 INFO:teuthology.task.install:Using flavor: basic 2021-01-06T22:00:10.103 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2021-01-06T22:00:10.104 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2021-01-06T22:00:10.104 DEBUG:teuthology.orchestra.run.smithi203:> sudo lsb_release -is 2021-01-06T22:00:10.156 INFO:teuthology.orchestra.run.smithi203.stdout:Ubuntu 2021-01-06T22:00:10.161 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-01-06T22:00:10.162 DEBUG:teuthology.orchestra.run.smithi091:> sudo lsb_release -is 2021-01-06T22:00:10.163 DEBUG:teuthology.orchestra.run.smithi203:> sudo apt-key list | grep Ceph 2021-01-06T22:00:10.214 INFO:teuthology.orchestra.run.smithi091.stdout:Ubuntu 2021-01-06T22:00:10.219 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-01-06T22:00:10.220 DEBUG:teuthology.orchestra.run.smithi091:> sudo apt-key list | grep Ceph 2021-01-06T22:00:10.242 INFO:teuthology.orchestra.run.smithi203.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-01-06T22:00:10.297 INFO:teuthology.orchestra.run.smithi203.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-01-06T22:00:10.297 INFO:teuthology.orchestra.run.smithi203.stdout:uid [ unknown] Ceph.com (release key) 2021-01-06T22:00:10.298 DEBUG:teuthology.orchestra.run.smithi203:> sudo lsb_release -is 2021-01-06T22:00:10.314 INFO:teuthology.orchestra.run.smithi091.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-01-06T22:00:10.350 INFO:teuthology.orchestra.run.smithi203.stdout:Ubuntu 2021-01-06T22:00:10.355 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-01-06T22:00:10.355 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2021-01-06T22:00:10.355 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2021-01-06T22:00:10.356 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=d0ed162b51928c50f20cee111f8292828eda755e 2021-01-06T22:00:10.368 INFO:teuthology.orchestra.run.smithi091.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-01-06T22:00:10.369 INFO:teuthology.orchestra.run.smithi091.stdout:uid [ unknown] Ceph.com (release key) 2021-01-06T22:00:10.369 DEBUG:teuthology.orchestra.run.smithi091:> sudo lsb_release -is 2021-01-06T22:00:10.421 INFO:teuthology.orchestra.run.smithi091.stdout:Ubuntu 2021-01-06T22:00:10.427 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-01-06T22:00:10.427 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2021-01-06T22:00:10.427 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2021-01-06T22:00:10.428 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=d0ed162b51928c50f20cee111f8292828eda755e 2021-01-06T22:00:10.521 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default/ 2021-01-06T22:00:10.522 INFO:teuthology.task.install.deb:Package version is 16.0.0-8719-gd0ed162b-1bionic 2021-01-06T22:00:10.582 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default/ 2021-01-06T22:00:10.582 INFO:teuthology.task.install.deb:Package version is 16.0.0-8719-gd0ed162b-1bionic 2021-01-06T22:00:10.881 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:00:10.882 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-01-06T22:00:10.897 DEBUG:teuthology.orchestra.run.smithi203:> sudo apt-get update 2021-01-06T22:00:10.929 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:00:10.930 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-01-06T22:00:10.945 DEBUG:teuthology.orchestra.run.smithi091:> sudo apt-get update 2021-01-06T22:00:11.089 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-01-06T22:00:11.863 INFO:teuthology.orchestra.run.smithi091.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-01-06T22:00:11.863 INFO:teuthology.orchestra.run.smithi091.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-01-06T22:00:11.863 INFO:teuthology.orchestra.run.smithi091.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic InRelease 2021-01-06T22:00:11.864 INFO:teuthology.orchestra.run.smithi091.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-01-06T22:00:11.864 INFO:teuthology.orchestra.run.smithi091.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-01-06T22:00:11.864 INFO:teuthology.orchestra.run.smithi091.stdout:Hit:6 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-01-06T22:00:11.864 INFO:teuthology.orchestra.run.smithi091.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic Release.gpg 2021-01-06T22:00:11.864 INFO:teuthology.orchestra.run.smithi091.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main i386 Packages [2,954 B] 2021-01-06T22:00:11.865 INFO:teuthology.orchestra.run.smithi091.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.2 kB] 2021-01-06T22:00:11.865 INFO:teuthology.orchestra.run.smithi091.stdout:Fetched 26.9 kB in 1s (48.3 kB/s) 2021-01-06T22:00:11.865 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-01-06T22:00:11.865 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-01-06T22:00:11.865 INFO:teuthology.orchestra.run.smithi203.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic InRelease 2021-01-06T22:00:11.866 INFO:teuthology.orchestra.run.smithi203.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-01-06T22:00:11.866 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:6 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-01-06T22:00:11.866 INFO:teuthology.orchestra.run.smithi203.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic Release.gpg 2021-01-06T22:00:11.866 INFO:teuthology.orchestra.run.smithi203.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.2 kB] 2021-01-06T22:00:11.867 INFO:teuthology.orchestra.run.smithi203.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main i386 Packages [2,954 B] 2021-01-06T22:00:11.867 INFO:teuthology.orchestra.run.smithi203.stdout:Fetched 26.9 kB in 1s (49.5 kB/s) 2021-01-06T22:00:12.942 INFO:teuthology.orchestra.run.smithi203.stdout:Reading package lists... 2021-01-06T22:00:12.985 INFO:teuthology.orchestra.run.smithi091.stdout:Reading package lists... 2021-01-06T22:00:12.989 DEBUG:teuthology.orchestra.run.smithi203:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.0.0-8719-gd0ed162b-1bionic cephadm=16.0.0-8719-gd0ed162b-1bionic ceph-mds=16.0.0-8719-gd0ed162b-1bionic ceph-mgr=16.0.0-8719-gd0ed162b-1bionic ceph-common=16.0.0-8719-gd0ed162b-1bionic ceph-fuse=16.0.0-8719-gd0ed162b-1bionic ceph-test=16.0.0-8719-gd0ed162b-1bionic radosgw=16.0.0-8719-gd0ed162b-1bionic python3-rados=16.0.0-8719-gd0ed162b-1bionic python3-rgw=16.0.0-8719-gd0ed162b-1bionic python3-cephfs=16.0.0-8719-gd0ed162b-1bionic python3-rbd=16.0.0-8719-gd0ed162b-1bionic libcephfs2=16.0.0-8719-gd0ed162b-1bionic libcephfs-dev=16.0.0-8719-gd0ed162b-1bionic librados2=16.0.0-8719-gd0ed162b-1bionic librbd1=16.0.0-8719-gd0ed162b-1bionic rbd-fuse=16.0.0-8719-gd0ed162b-1bionic python3-cephfs=16.0.0-8719-gd0ed162b-1bionic cephfs-shell=16.0.0-8719-gd0ed162b-1bionic 2021-01-06T22:00:13.031 DEBUG:teuthology.orchestra.run.smithi091:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.0.0-8719-gd0ed162b-1bionic cephadm=16.0.0-8719-gd0ed162b-1bionic ceph-mds=16.0.0-8719-gd0ed162b-1bionic ceph-mgr=16.0.0-8719-gd0ed162b-1bionic ceph-common=16.0.0-8719-gd0ed162b-1bionic ceph-fuse=16.0.0-8719-gd0ed162b-1bionic ceph-test=16.0.0-8719-gd0ed162b-1bionic radosgw=16.0.0-8719-gd0ed162b-1bionic python3-rados=16.0.0-8719-gd0ed162b-1bionic python3-rgw=16.0.0-8719-gd0ed162b-1bionic python3-cephfs=16.0.0-8719-gd0ed162b-1bionic python3-rbd=16.0.0-8719-gd0ed162b-1bionic libcephfs2=16.0.0-8719-gd0ed162b-1bionic libcephfs-dev=16.0.0-8719-gd0ed162b-1bionic librados2=16.0.0-8719-gd0ed162b-1bionic librbd1=16.0.0-8719-gd0ed162b-1bionic rbd-fuse=16.0.0-8719-gd0ed162b-1bionic python3-cephfs=16.0.0-8719-gd0ed162b-1bionic cephfs-shell=16.0.0-8719-gd0ed162b-1bionic 2021-01-06T22:00:13.071 INFO:teuthology.orchestra.run.smithi203.stdout:Reading package lists... 2021-01-06T22:00:13.113 INFO:teuthology.orchestra.run.smithi091.stdout:Reading package lists... 2021-01-06T22:00:13.329 INFO:teuthology.orchestra.run.smithi203.stdout:Building dependency tree... 2021-01-06T22:00:13.330 INFO:teuthology.orchestra.run.smithi203.stdout:Reading state information... 2021-01-06T22:00:13.353 INFO:teuthology.orchestra.run.smithi091.stdout:Building dependency tree... 2021-01-06T22:00:13.353 INFO:teuthology.orchestra.run.smithi091.stdout:Reading state information... 2021-01-06T22:00:13.499 INFO:teuthology.orchestra.run.smithi203.stdout:The following additional packages will be installed: 2021-01-06T22:00:13.500 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2021-01-06T22:00:13.501 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2021-01-06T22:00:13.501 INFO:teuthology.orchestra.run.smithi203.stdout: libjaeger libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2021-01-06T22:00:13.501 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2021-01-06T22:00:13.501 INFO:teuthology.orchestra.run.smithi203.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2021-01-06T22:00:13.502 INFO:teuthology.orchestra.run.smithi203.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-01-06T22:00:13.502 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-01-06T22:00:13.502 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2021-01-06T22:00:13.502 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2021-01-06T22:00:13.503 INFO:teuthology.orchestra.run.smithi203.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2021-01-06T22:00:13.503 INFO:teuthology.orchestra.run.smithi203.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2021-01-06T22:00:13.503 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2021-01-06T22:00:13.503 INFO:teuthology.orchestra.run.smithi203.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2021-01-06T22:00:13.503 INFO:teuthology.orchestra.run.smithi203.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2021-01-06T22:00:13.504 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2021-01-06T22:00:13.504 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2021-01-06T22:00:13.504 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2021-01-06T22:00:13.504 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-01-06T22:00:13.504 INFO:teuthology.orchestra.run.smithi203.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-01-06T22:00:13.505 INFO:teuthology.orchestra.run.smithi203.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-01-06T22:00:13.505 INFO:teuthology.orchestra.run.smithi203.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-01-06T22:00:13.505 INFO:teuthology.orchestra.run.smithi203.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-01-06T22:00:13.505 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-01-06T22:00:13.506 INFO:teuthology.orchestra.run.smithi203.stdout:Suggested packages: 2021-01-06T22:00:13.506 INFO:teuthology.orchestra.run.smithi203.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-01-06T22:00:13.506 INFO:teuthology.orchestra.run.smithi203.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-01-06T22:00:13.506 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-01-06T22:00:13.507 INFO:teuthology.orchestra.run.smithi203.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-01-06T22:00:13.507 INFO:teuthology.orchestra.run.smithi203.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-01-06T22:00:13.507 INFO:teuthology.orchestra.run.smithi203.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-01-06T22:00:13.507 INFO:teuthology.orchestra.run.smithi203.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-01-06T22:00:13.507 INFO:teuthology.orchestra.run.smithi203.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-01-06T22:00:13.508 INFO:teuthology.orchestra.run.smithi203.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-01-06T22:00:13.508 INFO:teuthology.orchestra.run.smithi203.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-01-06T22:00:13.508 INFO:teuthology.orchestra.run.smithi203.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-01-06T22:00:13.508 INFO:teuthology.orchestra.run.smithi203.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-01-06T22:00:13.508 INFO:teuthology.orchestra.run.smithi203.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-01-06T22:00:13.509 INFO:teuthology.orchestra.run.smithi203.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-01-06T22:00:13.509 INFO:teuthology.orchestra.run.smithi203.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-01-06T22:00:13.520 INFO:teuthology.orchestra.run.smithi091.stdout:The following additional packages will be installed: 2021-01-06T22:00:13.520 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2021-01-06T22:00:13.521 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2021-01-06T22:00:13.521 INFO:teuthology.orchestra.run.smithi091.stdout: libjaeger libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2021-01-06T22:00:13.522 INFO:teuthology.orchestra.run.smithi091.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2021-01-06T22:00:13.522 INFO:teuthology.orchestra.run.smithi091.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2021-01-06T22:00:13.522 INFO:teuthology.orchestra.run.smithi091.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-01-06T22:00:13.522 INFO:teuthology.orchestra.run.smithi091.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-01-06T22:00:13.523 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2021-01-06T22:00:13.523 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2021-01-06T22:00:13.523 INFO:teuthology.orchestra.run.smithi091.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2021-01-06T22:00:13.523 INFO:teuthology.orchestra.run.smithi091.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2021-01-06T22:00:13.523 INFO:teuthology.orchestra.run.smithi091.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2021-01-06T22:00:13.524 INFO:teuthology.orchestra.run.smithi091.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2021-01-06T22:00:13.524 INFO:teuthology.orchestra.run.smithi091.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2021-01-06T22:00:13.524 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2021-01-06T22:00:13.524 INFO:teuthology.orchestra.run.smithi091.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2021-01-06T22:00:13.524 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2021-01-06T22:00:13.525 INFO:teuthology.orchestra.run.smithi091.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-01-06T22:00:13.525 INFO:teuthology.orchestra.run.smithi091.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-01-06T22:00:13.525 INFO:teuthology.orchestra.run.smithi091.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-01-06T22:00:13.525 INFO:teuthology.orchestra.run.smithi091.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-01-06T22:00:13.525 INFO:teuthology.orchestra.run.smithi091.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-01-06T22:00:13.526 INFO:teuthology.orchestra.run.smithi091.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-01-06T22:00:13.526 INFO:teuthology.orchestra.run.smithi091.stdout:Suggested packages: 2021-01-06T22:00:13.526 INFO:teuthology.orchestra.run.smithi091.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-01-06T22:00:13.526 INFO:teuthology.orchestra.run.smithi091.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-01-06T22:00:13.527 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-01-06T22:00:13.527 INFO:teuthology.orchestra.run.smithi091.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-01-06T22:00:13.527 INFO:teuthology.orchestra.run.smithi091.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-01-06T22:00:13.527 INFO:teuthology.orchestra.run.smithi091.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-01-06T22:00:13.527 INFO:teuthology.orchestra.run.smithi091.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-01-06T22:00:13.528 INFO:teuthology.orchestra.run.smithi091.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-01-06T22:00:13.528 INFO:teuthology.orchestra.run.smithi091.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-01-06T22:00:13.528 INFO:teuthology.orchestra.run.smithi091.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-01-06T22:00:13.528 INFO:teuthology.orchestra.run.smithi091.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-01-06T22:00:13.528 INFO:teuthology.orchestra.run.smithi091.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-01-06T22:00:13.529 INFO:teuthology.orchestra.run.smithi091.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-01-06T22:00:13.529 INFO:teuthology.orchestra.run.smithi091.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-01-06T22:00:13.529 INFO:teuthology.orchestra.run.smithi091.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-01-06T22:00:13.607 INFO:teuthology.orchestra.run.smithi203.stdout:The following NEW packages will be installed: 2021-01-06T22:00:13.608 INFO:teuthology.orchestra.run.smithi203.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-01-06T22:00:13.608 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2021-01-06T22:00:13.608 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm 2021-01-06T22:00:13.609 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell jq libcephfs-dev libcephfs2 libjaeger libjq1 libleveldb1v5 2021-01-06T22:00:13.610 INFO:teuthology.orchestra.run.smithi203.stdout: liblttng-ust0 liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 2021-01-06T22:00:13.610 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2 libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-01-06T22:00:13.610 INFO:teuthology.orchestra.run.smithi203.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-01-06T22:00:13.610 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-01-06T22:00:13.611 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2021-01-06T22:00:13.611 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi-backend python3-cherrypy3 python3-cmd2 python3-colorama 2021-01-06T22:00:13.611 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography python3-cycler python3-dateutil python3-decorator 2021-01-06T22:00:13.611 INFO:teuthology.orchestra.run.smithi203.stdout: python3-fasteners python3-html5lib python3-httplib2 python3-jinja2 2021-01-06T22:00:13.611 INFO:teuthology.orchestra.run.smithi203.stdout: python3-joblib python3-json-pointer python3-jsonpatch python3-jwt 2021-01-06T22:00:13.612 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2021-01-06T22:00:13.612 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-01-06T22:00:13.612 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-01-06T22:00:13.612 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-01-06T22:00:13.612 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-01-06T22:00:13.613 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2021-01-06T22:00:13.613 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-01-06T22:00:13.613 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-01-06T22:00:13.613 INFO:teuthology.orchestra.run.smithi203.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-01-06T22:00:13.613 INFO:teuthology.orchestra.run.smithi203.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-01-06T22:00:13.614 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-01-06T22:00:13.614 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-01-06T22:00:13.614 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-01-06T22:00:13.615 INFO:teuthology.orchestra.run.smithi203.stdout:The following packages will be upgraded: 2021-01-06T22:00:13.615 INFO:teuthology.orchestra.run.smithi203.stdout: librados2 librbd1 2021-01-06T22:00:13.632 INFO:teuthology.orchestra.run.smithi091.stdout:The following NEW packages will be installed: 2021-01-06T22:00:13.632 INFO:teuthology.orchestra.run.smithi091.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-01-06T22:00:13.633 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2021-01-06T22:00:13.633 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm 2021-01-06T22:00:13.634 INFO:teuthology.orchestra.run.smithi091.stdout: cephfs-shell jq libcephfs-dev libcephfs2 libjaeger libjq1 libleveldb1v5 2021-01-06T22:00:13.634 INFO:teuthology.orchestra.run.smithi091.stdout: liblttng-ust0 liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 2021-01-06T22:00:13.634 INFO:teuthology.orchestra.run.smithi091.stdout: librgw2 libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-01-06T22:00:13.635 INFO:teuthology.orchestra.run.smithi091.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-01-06T22:00:13.635 INFO:teuthology.orchestra.run.smithi091.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-01-06T22:00:13.635 INFO:teuthology.orchestra.run.smithi091.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2021-01-06T22:00:13.635 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cffi-backend python3-cherrypy3 python3-cmd2 python3-colorama 2021-01-06T22:00:13.635 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cryptography python3-cycler python3-dateutil python3-decorator 2021-01-06T22:00:13.636 INFO:teuthology.orchestra.run.smithi091.stdout: python3-fasteners python3-html5lib python3-httplib2 python3-jinja2 2021-01-06T22:00:13.636 INFO:teuthology.orchestra.run.smithi091.stdout: python3-joblib python3-json-pointer python3-jsonpatch python3-jwt 2021-01-06T22:00:13.636 INFO:teuthology.orchestra.run.smithi091.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2021-01-06T22:00:13.636 INFO:teuthology.orchestra.run.smithi091.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-01-06T22:00:13.636 INFO:teuthology.orchestra.run.smithi091.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-01-06T22:00:13.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-01-06T22:00:13.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-01-06T22:00:13.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2021-01-06T22:00:13.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-01-06T22:00:13.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-01-06T22:00:13.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-01-06T22:00:13.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-01-06T22:00:13.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-01-06T22:00:13.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-01-06T22:00:13.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-01-06T22:00:13.639 INFO:teuthology.orchestra.run.smithi091.stdout:The following packages will be upgraded: 2021-01-06T22:00:13.639 INFO:teuthology.orchestra.run.smithi091.stdout: librados2 librbd1 2021-01-06T22:00:13.816 INFO:teuthology.orchestra.run.smithi203.stdout:2 upgraded, 115 newly installed, 0 to remove and 9 not upgraded. 2021-01-06T22:00:13.817 INFO:teuthology.orchestra.run.smithi203.stdout:Need to get 169 MB of archives. 2021-01-06T22:00:13.817 INFO:teuthology.orchestra.run.smithi203.stdout:After this operation, 794 MB of additional disk space will be used. 2021-01-06T22:00:13.817 INFO:teuthology.orchestra.run.smithi203.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-01-06T22:00:13.828 INFO:teuthology.orchestra.run.smithi091.stdout:2 upgraded, 115 newly installed, 0 to remove and 9 not upgraded. 2021-01-06T22:00:13.828 INFO:teuthology.orchestra.run.smithi091.stdout:Need to get 169 MB of archives. 2021-01-06T22:00:13.828 INFO:teuthology.orchestra.run.smithi091.stdout:After this operation, 794 MB of additional disk space will be used. 2021-01-06T22:00:13.828 INFO:teuthology.orchestra.run.smithi091.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-01-06T22:00:13.940 INFO:teuthology.orchestra.run.smithi203.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 libjaeger amd64 16.0.0-8719-gd0ed162b-1bionic [3,728 B] 2021-01-06T22:00:13.969 INFO:teuthology.orchestra.run.smithi091.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 libjaeger amd64 16.0.0-8719-gd0ed162b-1bionic [3,728 B] 2021-01-06T22:00:14.077 INFO:teuthology.orchestra.run.smithi203.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2021-01-06T22:00:14.085 INFO:teuthology.orchestra.run.smithi091.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2021-01-06T22:00:14.350 INFO:teuthology.orchestra.run.smithi203.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2021-01-06T22:00:14.351 INFO:teuthology.orchestra.run.smithi203.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.0.0-8719-gd0ed162b-1bionic [2,250 kB] 2021-01-06T22:00:14.351 INFO:teuthology.orchestra.run.smithi203.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-01-06T22:00:14.351 INFO:teuthology.orchestra.run.smithi203.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-01-06T22:00:14.352 INFO:teuthology.orchestra.run.smithi203.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-01-06T22:00:14.352 INFO:teuthology.orchestra.run.smithi203.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-01-06T22:00:14.352 INFO:teuthology.orchestra.run.smithi203.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-01-06T22:00:14.352 INFO:teuthology.orchestra.run.smithi203.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-01-06T22:00:14.352 INFO:teuthology.orchestra.run.smithi203.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-01-06T22:00:14.353 INFO:teuthology.orchestra.run.smithi091.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2021-01-06T22:00:14.353 INFO:teuthology.orchestra.run.smithi091.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-01-06T22:00:14.353 INFO:teuthology.orchestra.run.smithi091.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-01-06T22:00:14.353 INFO:teuthology.orchestra.run.smithi091.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.0.0-8719-gd0ed162b-1bionic [2,250 kB] 2021-01-06T22:00:14.354 INFO:teuthology.orchestra.run.smithi091.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-01-06T22:00:14.354 INFO:teuthology.orchestra.run.smithi091.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-01-06T22:00:14.354 INFO:teuthology.orchestra.run.smithi091.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-01-06T22:00:14.361 INFO:teuthology.orchestra.run.smithi091.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-01-06T22:00:14.364 INFO:teuthology.orchestra.run.smithi091.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-01-06T22:00:14.364 INFO:teuthology.orchestra.run.smithi203.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-01-06T22:00:14.372 INFO:teuthology.orchestra.run.smithi203.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-01-06T22:00:14.380 INFO:teuthology.orchestra.run.smithi203.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-01-06T22:00:14.389 INFO:teuthology.orchestra.run.smithi091.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-01-06T22:00:14.391 INFO:teuthology.orchestra.run.smithi203.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-01-06T22:00:14.398 INFO:teuthology.orchestra.run.smithi091.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-01-06T22:00:14.400 INFO:teuthology.orchestra.run.smithi203.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-01-06T22:00:14.407 INFO:teuthology.orchestra.run.smithi091.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-01-06T22:00:14.418 INFO:teuthology.orchestra.run.smithi091.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-01-06T22:00:14.421 INFO:teuthology.orchestra.run.smithi203.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-01-06T22:00:14.425 INFO:teuthology.orchestra.run.smithi203.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-01-06T22:00:14.425 INFO:teuthology.orchestra.run.smithi091.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-01-06T22:00:14.449 INFO:teuthology.orchestra.run.smithi091.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-01-06T22:00:14.453 INFO:teuthology.orchestra.run.smithi091.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-01-06T22:00:14.489 INFO:teuthology.orchestra.run.smithi203.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-01-06T22:00:14.498 INFO:teuthology.orchestra.run.smithi203.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-01-06T22:00:14.498 INFO:teuthology.orchestra.run.smithi203.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-01-06T22:00:14.499 INFO:teuthology.orchestra.run.smithi203.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-01-06T22:00:14.500 INFO:teuthology.orchestra.run.smithi203.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-01-06T22:00:14.500 INFO:teuthology.orchestra.run.smithi091.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-01-06T22:00:14.502 INFO:teuthology.orchestra.run.smithi203.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-01-06T22:00:14.502 INFO:teuthology.orchestra.run.smithi203.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-01-06T22:00:14.510 INFO:teuthology.orchestra.run.smithi091.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-01-06T22:00:14.511 INFO:teuthology.orchestra.run.smithi091.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-01-06T22:00:14.512 INFO:teuthology.orchestra.run.smithi091.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-01-06T22:00:14.513 INFO:teuthology.orchestra.run.smithi091.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-01-06T22:00:14.516 INFO:teuthology.orchestra.run.smithi091.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-01-06T22:00:14.570 INFO:teuthology.orchestra.run.smithi203.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-01-06T22:00:14.572 INFO:teuthology.orchestra.run.smithi203.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-01-06T22:00:14.575 INFO:teuthology.orchestra.run.smithi203.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-01-06T22:00:14.581 INFO:teuthology.orchestra.run.smithi091.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-01-06T22:00:14.619 INFO:teuthology.orchestra.run.smithi091.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-01-06T22:00:14.620 INFO:teuthology.orchestra.run.smithi091.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-01-06T22:00:14.620 INFO:teuthology.orchestra.run.smithi091.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-01-06T22:00:14.651 INFO:teuthology.orchestra.run.smithi203.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-01-06T22:00:14.652 INFO:teuthology.orchestra.run.smithi203.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-01-06T22:00:14.663 INFO:teuthology.orchestra.run.smithi091.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-01-06T22:00:14.663 INFO:teuthology.orchestra.run.smithi091.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-01-06T22:00:14.673 INFO:teuthology.orchestra.run.smithi203.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-01-06T22:00:14.673 INFO:teuthology.orchestra.run.smithi203.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-01-06T22:00:14.674 INFO:teuthology.orchestra.run.smithi203.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-01-06T22:00:14.678 INFO:teuthology.orchestra.run.smithi203.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-01-06T22:00:14.678 INFO:teuthology.orchestra.run.smithi203.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-01-06T22:00:14.679 INFO:teuthology.orchestra.run.smithi091.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-01-06T22:00:14.679 INFO:teuthology.orchestra.run.smithi091.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-01-06T22:00:14.680 INFO:teuthology.orchestra.run.smithi091.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-01-06T22:00:14.680 INFO:teuthology.orchestra.run.smithi091.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-01-06T22:00:14.732 INFO:teuthology.orchestra.run.smithi203.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-01-06T22:00:14.737 INFO:teuthology.orchestra.run.smithi203.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-01-06T22:00:14.737 INFO:teuthology.orchestra.run.smithi203.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-01-06T22:00:14.743 INFO:teuthology.orchestra.run.smithi091.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-01-06T22:00:14.749 INFO:teuthology.orchestra.run.smithi091.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-01-06T22:00:14.752 INFO:teuthology.orchestra.run.smithi091.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-01-06T22:00:14.752 INFO:teuthology.orchestra.run.smithi091.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-01-06T22:00:14.814 INFO:teuthology.orchestra.run.smithi203.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-01-06T22:00:14.824 INFO:teuthology.orchestra.run.smithi091.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-01-06T22:00:14.880 INFO:teuthology.orchestra.run.smithi203.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-01-06T22:00:14.880 INFO:teuthology.orchestra.run.smithi203.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-01-06T22:00:14.909 INFO:teuthology.orchestra.run.smithi091.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-01-06T22:00:14.909 INFO:teuthology.orchestra.run.smithi091.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-01-06T22:00:15.021 INFO:teuthology.orchestra.run.smithi091.stdout:Get:43 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.0.0-8719-gd0ed162b-1bionic [3,140 kB] 2021-01-06T22:00:15.171 INFO:teuthology.orchestra.run.smithi203.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-01-06T22:00:15.190 INFO:teuthology.orchestra.run.smithi091.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-01-06T22:00:15.214 INFO:teuthology.orchestra.run.smithi203.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-01-06T22:00:15.215 INFO:teuthology.orchestra.run.smithi203.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-01-06T22:00:15.240 INFO:teuthology.orchestra.run.smithi091.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-01-06T22:00:15.243 INFO:teuthology.orchestra.run.smithi091.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-01-06T22:00:15.247 INFO:teuthology.orchestra.run.smithi203.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-01-06T22:00:15.248 INFO:teuthology.orchestra.run.smithi203.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-01-06T22:00:15.248 INFO:teuthology.orchestra.run.smithi203.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2021-01-06T22:00:15.248 INFO:teuthology.orchestra.run.smithi203.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-01-06T22:00:15.249 INFO:teuthology.orchestra.run.smithi203.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-01-06T22:00:15.249 INFO:teuthology.orchestra.run.smithi203.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-01-06T22:00:15.253 INFO:teuthology.orchestra.run.smithi203.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-01-06T22:00:15.257 INFO:teuthology.orchestra.run.smithi203.stdout:Get:53 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.0.0-8719-gd0ed162b-1bionic [3,140 kB] 2021-01-06T22:00:15.272 INFO:teuthology.orchestra.run.smithi091.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-01-06T22:00:15.272 INFO:teuthology.orchestra.run.smithi091.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-01-06T22:00:15.273 INFO:teuthology.orchestra.run.smithi091.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2021-01-06T22:00:15.273 INFO:teuthology.orchestra.run.smithi091.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-01-06T22:00:15.274 INFO:teuthology.orchestra.run.smithi091.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-01-06T22:00:15.274 INFO:teuthology.orchestra.run.smithi091.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-01-06T22:00:15.274 INFO:teuthology.orchestra.run.smithi091.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-01-06T22:00:15.331 INFO:teuthology.orchestra.run.smithi203.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-01-06T22:00:15.332 INFO:teuthology.orchestra.run.smithi203.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-01-06T22:00:15.334 INFO:teuthology.orchestra.run.smithi203.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-01-06T22:00:15.335 INFO:teuthology.orchestra.run.smithi203.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-01-06T22:00:15.337 INFO:teuthology.orchestra.run.smithi203.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-01-06T22:00:15.337 INFO:teuthology.orchestra.run.smithi203.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-01-06T22:00:15.338 INFO:teuthology.orchestra.run.smithi203.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-01-06T22:00:15.339 INFO:teuthology.orchestra.run.smithi203.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-01-06T22:00:15.339 INFO:teuthology.orchestra.run.smithi203.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-01-06T22:00:15.351 INFO:teuthology.orchestra.run.smithi091.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-01-06T22:00:15.353 INFO:teuthology.orchestra.run.smithi091.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-01-06T22:00:15.417 INFO:teuthology.orchestra.run.smithi203.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-01-06T22:00:15.426 INFO:teuthology.orchestra.run.smithi203.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-01-06T22:00:15.429 INFO:teuthology.orchestra.run.smithi203.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-01-06T22:00:15.433 INFO:teuthology.orchestra.run.smithi091.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-01-06T22:00:15.434 INFO:teuthology.orchestra.run.smithi091.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-01-06T22:00:15.444 INFO:teuthology.orchestra.run.smithi091.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-01-06T22:00:15.445 INFO:teuthology.orchestra.run.smithi091.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-01-06T22:00:15.446 INFO:teuthology.orchestra.run.smithi091.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-01-06T22:00:15.449 INFO:teuthology.orchestra.run.smithi091.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-01-06T22:00:15.450 INFO:teuthology.orchestra.run.smithi091.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-01-06T22:00:15.451 INFO:teuthology.orchestra.run.smithi091.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-01-06T22:00:15.498 INFO:teuthology.orchestra.run.smithi203.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2021-01-06T22:00:15.499 INFO:teuthology.orchestra.run.smithi203.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-01-06T22:00:15.504 INFO:teuthology.orchestra.run.smithi203.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-01-06T22:00:15.505 INFO:teuthology.orchestra.run.smithi203.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-01-06T22:00:15.505 INFO:teuthology.orchestra.run.smithi203.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-01-06T22:00:15.506 INFO:teuthology.orchestra.run.smithi203.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-01-06T22:00:15.507 INFO:teuthology.orchestra.run.smithi203.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-01-06T22:00:15.514 INFO:teuthology.orchestra.run.smithi091.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-01-06T22:00:15.521 INFO:teuthology.orchestra.run.smithi091.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-01-06T22:00:15.580 INFO:teuthology.orchestra.run.smithi203.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-01-06T22:00:15.580 INFO:teuthology.orchestra.run.smithi203.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-01-06T22:00:15.581 INFO:teuthology.orchestra.run.smithi203.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-01-06T22:00:15.595 INFO:teuthology.orchestra.run.smithi091.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2021-01-06T22:00:15.595 INFO:teuthology.orchestra.run.smithi091.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-01-06T22:00:15.600 INFO:teuthology.orchestra.run.smithi091.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-01-06T22:00:15.601 INFO:teuthology.orchestra.run.smithi091.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-01-06T22:00:15.602 INFO:teuthology.orchestra.run.smithi091.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-01-06T22:00:15.602 INFO:teuthology.orchestra.run.smithi091.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-01-06T22:00:15.603 INFO:teuthology.orchestra.run.smithi091.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-01-06T22:00:15.606 INFO:teuthology.orchestra.run.smithi091.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-01-06T22:00:15.661 INFO:teuthology.orchestra.run.smithi203.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-01-06T22:00:15.663 INFO:teuthology.orchestra.run.smithi203.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.3 [1,097 kB] 2021-01-06T22:00:15.677 INFO:teuthology.orchestra.run.smithi091.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-01-06T22:00:15.678 INFO:teuthology.orchestra.run.smithi091.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-01-06T22:00:15.682 INFO:teuthology.orchestra.run.smithi203.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-01-06T22:00:15.683 INFO:teuthology.orchestra.run.smithi203.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-01-06T22:00:15.732 INFO:teuthology.orchestra.run.smithi091.stdout:Get:76 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.0.0-8719-gd0ed162b-1bionic [480 kB] 2021-01-06T22:00:15.759 INFO:teuthology.orchestra.run.smithi091.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-01-06T22:00:15.760 INFO:teuthology.orchestra.run.smithi091.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.3 [1,097 kB] 2021-01-06T22:00:15.777 INFO:teuthology.orchestra.run.smithi091.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-01-06T22:00:15.778 INFO:teuthology.orchestra.run.smithi091.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-01-06T22:00:15.779 INFO:teuthology.orchestra.run.smithi203.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-01-06T22:00:15.780 INFO:teuthology.orchestra.run.smithi203.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-01-06T22:00:15.780 INFO:teuthology.orchestra.run.smithi203.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2021-01-06T22:00:15.809 INFO:teuthology.orchestra.run.smithi091.stdout:Get:81 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.0.0-8719-gd0ed162b-1bionic [337 kB] 2021-01-06T22:00:15.830 INFO:teuthology.orchestra.run.smithi203.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-01-06T22:00:15.831 INFO:teuthology.orchestra.run.smithi203.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-01-06T22:00:15.832 INFO:teuthology.orchestra.run.smithi203.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-01-06T22:00:15.833 INFO:teuthology.orchestra.run.smithi203.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-01-06T22:00:15.836 INFO:teuthology.orchestra.run.smithi203.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-01-06T22:00:15.837 INFO:teuthology.orchestra.run.smithi203.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-01-06T22:00:15.865 INFO:teuthology.orchestra.run.smithi091.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-01-06T22:00:15.866 INFO:teuthology.orchestra.run.smithi091.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-01-06T22:00:15.867 INFO:teuthology.orchestra.run.smithi091.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2021-01-06T22:00:15.871 INFO:teuthology.orchestra.run.smithi091.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-01-06T22:00:15.871 INFO:teuthology.orchestra.run.smithi091.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-01-06T22:00:15.872 INFO:teuthology.orchestra.run.smithi091.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-01-06T22:00:15.883 INFO:teuthology.orchestra.run.smithi091.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.0.0-8719-gd0ed162b-1bionic [20.6 kB] 2021-01-06T22:00:15.884 INFO:teuthology.orchestra.run.smithi091.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.0.0-8719-gd0ed162b-1bionic [175 kB] 2021-01-06T22:00:15.889 INFO:teuthology.orchestra.run.smithi091.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.0.0-8719-gd0ed162b-1bionic [27.8 kB] 2021-01-06T22:00:15.893 INFO:teuthology.orchestra.run.smithi091.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.0.0-8719-gd0ed162b-1bionic [330 kB] 2021-01-06T22:00:15.921 INFO:teuthology.orchestra.run.smithi091.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-01-06T22:00:15.924 INFO:teuthology.orchestra.run.smithi091.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-01-06T22:00:15.925 INFO:teuthology.orchestra.run.smithi091.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-01-06T22:00:15.964 INFO:teuthology.orchestra.run.smithi091.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.0.0-8719-gd0ed162b-1bionic [3,232 kB] 2021-01-06T22:00:16.402 INFO:teuthology.orchestra.run.smithi203.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.0.0-8719-gd0ed162b-1bionic [480 kB] 2021-01-06T22:00:16.488 INFO:teuthology.orchestra.run.smithi091.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.0.0-8719-gd0ed162b-1bionic [99.3 kB] 2021-01-06T22:00:16.491 INFO:teuthology.orchestra.run.smithi091.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.0.0-8719-gd0ed162b-1bionic [414 kB] 2021-01-06T22:00:16.515 INFO:teuthology.orchestra.run.smithi203.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.0.0-8719-gd0ed162b-1bionic [337 kB] 2021-01-06T22:00:16.552 INFO:teuthology.orchestra.run.smithi091.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.0.0-8719-gd0ed162b-1bionic [20.7 MB] 2021-01-06T22:00:16.634 INFO:teuthology.orchestra.run.smithi203.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.0.0-8719-gd0ed162b-1bionic [20.6 kB] 2021-01-06T22:00:16.635 INFO:teuthology.orchestra.run.smithi203.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.0.0-8719-gd0ed162b-1bionic [175 kB] 2021-01-06T22:00:16.695 INFO:teuthology.orchestra.run.smithi203.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.0.0-8719-gd0ed162b-1bionic [27.8 kB] 2021-01-06T22:00:16.696 INFO:teuthology.orchestra.run.smithi203.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.0.0-8719-gd0ed162b-1bionic [330 kB] 2021-01-06T22:00:16.760 INFO:teuthology.orchestra.run.smithi203.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.0.0-8719-gd0ed162b-1bionic [3,232 kB] 2021-01-06T22:00:17.659 INFO:teuthology.orchestra.run.smithi203.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.0.0-8719-gd0ed162b-1bionic [99.3 kB] 2021-01-06T22:00:17.663 INFO:teuthology.orchestra.run.smithi203.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.0.0-8719-gd0ed162b-1bionic [414 kB] 2021-01-06T22:00:17.793 INFO:teuthology.orchestra.run.smithi203.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.0.0-8719-gd0ed162b-1bionic [20.7 MB] 2021-01-06T22:00:19.442 INFO:teuthology.orchestra.run.smithi091.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.0.0-8719-gd0ed162b-1bionic [5,605 kB] 2021-01-06T22:00:20.173 INFO:teuthology.orchestra.run.smithi091.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.0.0-8719-gd0ed162b-1bionic [165 kB] 2021-01-06T22:00:20.230 INFO:teuthology.orchestra.run.smithi091.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.0.0-8719-gd0ed162b-1bionic [1,357 kB] 2021-01-06T22:00:20.389 INFO:teuthology.orchestra.run.smithi091.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.0.0-8719-gd0ed162b-1bionic [6,645 kB] 2021-01-06T22:00:21.275 INFO:teuthology.orchestra.run.smithi091.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.0.0-8719-gd0ed162b-1bionic [24.8 MB] 2021-01-06T22:00:23.089 INFO:teuthology.orchestra.run.smithi203.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.0.0-8719-gd0ed162b-1bionic [5,605 kB] 2021-01-06T22:00:23.320 INFO:teuthology.orchestra.run.smithi091.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.0.0-8719-gd0ed162b-1bionic [3,812 B] 2021-01-06T22:00:23.320 INFO:teuthology.orchestra.run.smithi091.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.0.0-8719-gd0ed162b-1bionic [563 kB] 2021-01-06T22:00:23.346 INFO:teuthology.orchestra.run.smithi091.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.0.0-8719-gd0ed162b-1bionic [1,945 kB] 2021-01-06T22:00:23.441 INFO:teuthology.orchestra.run.smithi091.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.0.0-8719-gd0ed162b-1bionic [57.9 kB] 2021-01-06T22:00:23.443 INFO:teuthology.orchestra.run.smithi091.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.0.0-8719-gd0ed162b-1bionic [59.3 kB] 2021-01-06T22:00:23.445 INFO:teuthology.orchestra.run.smithi091.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.0.0-8719-gd0ed162b-1bionic [2,372 kB] 2021-01-06T22:00:23.561 INFO:teuthology.orchestra.run.smithi091.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.0.0-8719-gd0ed162b-1bionic [7,885 kB] 2021-01-06T22:00:23.653 INFO:teuthology.orchestra.run.smithi203.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.0.0-8719-gd0ed162b-1bionic [165 kB] 2021-01-06T22:00:23.675 INFO:teuthology.orchestra.run.smithi203.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.0.0-8719-gd0ed162b-1bionic [1,357 kB] 2021-01-06T22:00:23.807 INFO:teuthology.orchestra.run.smithi203.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.0.0-8719-gd0ed162b-1bionic [6,645 kB] 2021-01-06T22:00:23.953 INFO:teuthology.orchestra.run.smithi091.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.0.0-8719-gd0ed162b-1bionic [3,932 B] 2021-01-06T22:00:23.953 INFO:teuthology.orchestra.run.smithi091.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 16.0.0-8719-gd0ed162b-1bionic [22.6 kB] 2021-01-06T22:00:23.954 INFO:teuthology.orchestra.run.smithi091.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.0.0-8719-gd0ed162b-1bionic [50.6 MB] 2021-01-06T22:00:24.445 INFO:teuthology.orchestra.run.smithi203.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.0.0-8719-gd0ed162b-1bionic [24.8 MB] 2021-01-06T22:00:26.595 INFO:teuthology.orchestra.run.smithi203.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.0.0-8719-gd0ed162b-1bionic [3,812 B] 2021-01-06T22:00:26.595 INFO:teuthology.orchestra.run.smithi203.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.0.0-8719-gd0ed162b-1bionic [563 kB] 2021-01-06T22:00:26.600 INFO:teuthology.orchestra.run.smithi091.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 16.0.0-8719-gd0ed162b-1bionic [17.0 kB] 2021-01-06T22:00:26.601 INFO:teuthology.orchestra.run.smithi091.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.0.0-8719-gd0ed162b-1bionic [19.8 kB] 2021-01-06T22:00:26.601 INFO:teuthology.orchestra.run.smithi091.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.0.0-8719-gd0ed162b-1bionic [9,901 kB] 2021-01-06T22:00:26.646 INFO:teuthology.orchestra.run.smithi203.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.0.0-8719-gd0ed162b-1bionic [1,945 kB] 2021-01-06T22:00:26.825 INFO:teuthology.orchestra.run.smithi203.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.0.0-8719-gd0ed162b-1bionic [57.9 kB] 2021-01-06T22:00:26.826 INFO:teuthology.orchestra.run.smithi203.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.0.0-8719-gd0ed162b-1bionic [59.3 kB] 2021-01-06T22:00:26.826 INFO:teuthology.orchestra.run.smithi203.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.0.0-8719-gd0ed162b-1bionic [2,372 kB] 2021-01-06T22:00:27.029 INFO:teuthology.orchestra.run.smithi203.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.0.0-8719-gd0ed162b-1bionic [7,885 kB] 2021-01-06T22:00:27.114 INFO:teuthology.orchestra.run.smithi091.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.0.0-8719-gd0ed162b-1bionic [77.7 kB] 2021-01-06T22:00:27.604 INFO:teuthology.orchestra.run.smithi203.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.0.0-8719-gd0ed162b-1bionic [3,932 B] 2021-01-06T22:00:27.605 INFO:teuthology.orchestra.run.smithi203.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 16.0.0-8719-gd0ed162b-1bionic [22.6 kB] 2021-01-06T22:00:27.606 INFO:teuthology.orchestra.run.smithi203.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.0.0-8719-gd0ed162b-1bionic [50.6 MB] 2021-01-06T22:00:27.821 INFO:teuthology.orchestra.run.smithi091.stdout:Fetched 169 MB in 13s (12.6 MB/s) 2021-01-06T22:00:27.948 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-pyparsing. 2021-01-06T22:00:27.999 INFO:teuthology.orchestra.run.smithi091.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137611 files and directories currently installed.) 2021-01-06T22:00:28.003 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-01-06T22:00:28.038 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-01-06T22:00:28.329 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-pyperclip. 2021-01-06T22:00:28.342 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2021-01-06T22:00:28.379 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2021-01-06T22:00:28.625 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-cmd2. 2021-01-06T22:00:28.638 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2021-01-06T22:00:28.682 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2021-01-06T22:00:28.993 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-uritemplate. 2021-01-06T22:00:29.009 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-01-06T22:00:29.052 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-01-06T22:00:29.262 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libjaeger. 2021-01-06T22:00:29.275 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../004-libjaeger_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:29.320 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libjaeger (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:29.640 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-01-06T22:00:29.655 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../005-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-01-06T22:00:29.740 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-01-06T22:00:30.116 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../006-librbd1_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:30.244 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking librbd1 (16.0.0-8719-gd0ed162b-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2021-01-06T22:00:30.605 INFO:teuthology.orchestra.run.smithi203.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 16.0.0-8719-gd0ed162b-1bionic [17.0 kB] 2021-01-06T22:00:30.606 INFO:teuthology.orchestra.run.smithi203.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.0.0-8719-gd0ed162b-1bionic [19.8 kB] 2021-01-06T22:00:30.606 INFO:teuthology.orchestra.run.smithi203.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.0.0-8719-gd0ed162b-1bionic [9,901 kB] 2021-01-06T22:00:30.817 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../007-librados2_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:30.940 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking librados2 (16.0.0-8719-gd0ed162b-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2021-01-06T22:00:31.139 INFO:teuthology.orchestra.run.smithi203.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20210105.221014/d0ed162b51928c50f20cee111f8292828eda755e/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.0.0-8719-gd0ed162b-1bionic [77.7 kB] 2021-01-06T22:00:31.577 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libcephfs2. 2021-01-06T22:00:31.591 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../008-libcephfs2_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:31.635 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libcephfs2 (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:31.842 INFO:teuthology.orchestra.run.smithi203.stdout:Fetched 169 MB in 18s (9,677 kB/s) 2021-01-06T22:00:31.933 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-rados. 2021-01-06T22:00:31.947 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../009-python3-rados_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:31.965 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pyparsing. 2021-01-06T22:00:32.013 INFO:teuthology.orchestra.run.smithi203.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137611 files and directories currently installed.) 2021-01-06T22:00:32.017 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-01-06T22:00:32.025 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-rados (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:32.055 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-01-06T22:00:32.350 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pyperclip. 2021-01-06T22:00:32.363 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-01-06T22:00:32.364 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2021-01-06T22:00:32.377 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../010-python3-ceph-argparse_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:00:32.408 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2021-01-06T22:00:32.414 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-ceph-argparse (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:32.694 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-cmd2. 2021-01-06T22:00:32.708 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2021-01-06T22:00:32.727 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-cephfs. 2021-01-06T22:00:32.740 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../011-python3-cephfs_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:32.753 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2021-01-06T22:00:32.777 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-cephfs (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:33.022 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-uritemplate. 2021-01-06T22:00:33.036 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-01-06T22:00:33.072 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-01-06T22:00:33.096 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-ceph-common. 2021-01-06T22:00:33.111 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../012-python3-ceph-common_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:00:33.155 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-ceph-common (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:33.275 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libjaeger. 2021-01-06T22:00:33.289 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../004-libjaeger_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:33.333 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libjaeger (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:33.475 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-prettytable. 2021-01-06T22:00:33.488 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../013-python3-prettytable_0.7.2-3_all.deb ... 2021-01-06T22:00:33.526 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-01-06T22:00:33.652 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-01-06T22:00:33.667 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../005-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-01-06T22:00:33.754 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-01-06T22:00:33.806 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-rbd. 2021-01-06T22:00:33.820 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../014-python3-rbd_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:33.857 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-rbd (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:34.087 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../006-librbd1_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:34.198 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking librbd1 (16.0.0-8719-gd0ed162b-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2021-01-06T22:00:34.221 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-01-06T22:00:34.235 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../015-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-01-06T22:00:34.279 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-01-06T22:00:34.615 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-01-06T22:00:34.628 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../016-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-01-06T22:00:34.673 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-01-06T22:00:34.732 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../007-librados2_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:34.861 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking librados2 (16.0.0-8719-gd0ed162b-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2021-01-06T22:00:34.993 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package librgw2. 2021-01-06T22:00:35.009 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../017-librgw2_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:35.051 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking librgw2 (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:35.498 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libcephfs2. 2021-01-06T22:00:35.512 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../008-libcephfs2_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:35.548 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libcephfs2 (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:35.620 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-rgw. 2021-01-06T22:00:35.633 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../018-python3-rgw_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:35.670 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-rgw (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:35.836 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-rados. 2021-01-06T22:00:35.851 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../009-python3-rados_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:35.885 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-rados (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:35.981 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-01-06T22:00:35.997 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../019-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-01-06T22:00:36.039 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-01-06T22:00:36.200 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-01-06T22:00:36.215 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../010-python3-ceph-argparse_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:00:36.284 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-ceph-argparse (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:36.342 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-01-06T22:00:36.358 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../020-libleveldb1v5_1.20-2_amd64.deb ... 2021-01-06T22:00:36.400 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-01-06T22:00:36.562 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-cephfs. 2021-01-06T22:00:36.576 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../011-python3-cephfs_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:36.620 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-cephfs (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:36.770 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package liboath0. 2021-01-06T22:00:36.784 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../021-liboath0_2.6.1-1_amd64.deb ... 2021-01-06T22:00:36.828 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-01-06T22:00:36.940 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-ceph-common. 2021-01-06T22:00:36.953 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../012-python3-ceph-common_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:00:36.990 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-ceph-common (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:37.173 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libradosstriper1. 2021-01-06T22:00:37.187 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../022-libradosstriper1_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:37.231 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libradosstriper1 (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:37.252 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-prettytable. 2021-01-06T22:00:37.265 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../013-python3-prettytable_0.7.2-3_all.deb ... 2021-01-06T22:00:37.294 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-01-06T22:00:37.534 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-rbd. 2021-01-06T22:00:37.548 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../014-python3-rbd_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:37.592 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-rbd (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:37.684 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package ceph-common. 2021-01-06T22:00:37.697 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../023-ceph-common_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:37.735 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking ceph-common (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:37.872 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-01-06T22:00:37.887 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../015-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-01-06T22:00:37.922 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-01-06T22:00:38.241 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-01-06T22:00:38.255 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../016-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-01-06T22:00:38.300 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-01-06T22:00:38.636 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package librgw2. 2021-01-06T22:00:38.652 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../017-librgw2_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:38.694 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking librgw2 (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:39.221 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-rgw. 2021-01-06T22:00:39.234 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../018-python3-rgw_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:39.263 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-rgw (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:39.558 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-01-06T22:00:39.572 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../019-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-01-06T22:00:39.606 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package ceph-base. 2021-01-06T22:00:39.616 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-01-06T22:00:39.619 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../024-ceph-base_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:39.673 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking ceph-base (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:39.902 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-01-06T22:00:39.915 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../020-libleveldb1v5_1.20-2_amd64.deb ... 2021-01-06T22:00:39.952 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-01-06T22:00:40.238 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package liboath0. 2021-01-06T22:00:40.251 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../021-liboath0_2.6.1-1_amd64.deb ... 2021-01-06T22:00:40.288 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-01-06T22:00:40.532 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libradosstriper1. 2021-01-06T22:00:40.544 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../022-libradosstriper1_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:40.582 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libradosstriper1 (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:40.636 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-dateutil. 2021-01-06T22:00:40.651 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../025-python3-dateutil_2.6.1-1_all.deb ... 2021-01-06T22:00:40.694 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-01-06T22:00:40.988 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-asn1crypto. 2021-01-06T22:00:41.004 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../026-python3-asn1crypto_0.24.0-1_all.deb ... 2021-01-06T22:00:41.035 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-common. 2021-01-06T22:00:41.047 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-01-06T22:00:41.049 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../023-ceph-common_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:41.093 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-common (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:41.341 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-cffi-backend. 2021-01-06T22:00:41.355 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../027-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-01-06T22:00:41.392 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-01-06T22:00:41.644 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-cryptography. 2021-01-06T22:00:41.657 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../028-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-01-06T22:00:41.685 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-01-06T22:00:42.041 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-openssl. 2021-01-06T22:00:42.055 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../029-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-01-06T22:00:42.099 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-01-06T22:00:42.385 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-01-06T22:00:42.400 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../030-ceph-mgr-modules-core_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:00:42.443 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking ceph-mgr-modules-core (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:42.748 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-bcrypt. 2021-01-06T22:00:42.761 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../031-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-01-06T22:00:42.790 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-01-06T22:00:42.832 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-base. 2021-01-06T22:00:42.845 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../024-ceph-base_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:42.882 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-base (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:43.034 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-cherrypy3. 2021-01-06T22:00:43.048 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../032-python3-cherrypy3_8.9.1-2_all.deb ... 2021-01-06T22:00:43.084 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-01-06T22:00:43.412 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-jwt. 2021-01-06T22:00:43.426 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../033-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-01-06T22:00:43.470 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-01-06T22:00:43.745 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-dateutil. 2021-01-06T22:00:43.759 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../025-python3-dateutil_2.6.1-1_all.deb ... 2021-01-06T22:00:43.803 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-01-06T22:00:44.073 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-asn1crypto. 2021-01-06T22:00:44.086 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../026-python3-asn1crypto_0.24.0-1_all.deb ... 2021-01-06T22:00:44.114 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-01-06T22:00:44.367 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-cffi-backend. 2021-01-06T22:00:44.382 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../027-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-01-06T22:00:44.417 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-01-06T22:00:44.582 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-logutils. 2021-01-06T22:00:44.595 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../034-python3-logutils_0.3.3-5_all.deb ... 2021-01-06T22:00:44.640 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-01-06T22:00:44.686 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-cryptography. 2021-01-06T22:00:44.702 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../028-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-01-06T22:00:44.745 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-01-06T22:00:44.935 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-markupsafe. 2021-01-06T22:00:44.951 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../035-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-01-06T22:00:44.993 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-01-06T22:00:45.109 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-openssl. 2021-01-06T22:00:45.125 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../029-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-01-06T22:00:45.184 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-01-06T22:00:45.304 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-mako. 2021-01-06T22:00:45.318 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../036-python3-mako_1.0.7+ds1-1_all.deb ... 2021-01-06T22:00:45.362 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-01-06T22:00:45.445 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-01-06T22:00:45.461 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../030-ceph-mgr-modules-core_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:00:45.503 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mgr-modules-core (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:45.657 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-simplegeneric. 2021-01-06T22:00:45.876 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-bcrypt. 2021-01-06T22:00:45.878 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../037-python3-simplegeneric_0.8.1-1_all.deb ... 2021-01-06T22:00:45.878 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-01-06T22:00:45.890 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../031-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-01-06T22:00:45.932 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-01-06T22:00:46.043 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-singledispatch. 2021-01-06T22:00:46.058 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../038-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-01-06T22:00:46.101 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-01-06T22:00:46.235 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-cherrypy3. 2021-01-06T22:00:46.248 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../032-python3-cherrypy3_8.9.1-2_all.deb ... 2021-01-06T22:00:46.285 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-01-06T22:00:46.396 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-webob. 2021-01-06T22:00:46.410 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../039-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-01-06T22:00:46.454 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-01-06T22:00:46.622 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-jwt. 2021-01-06T22:00:46.637 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../033-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-01-06T22:00:46.680 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-01-06T22:00:46.782 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-bs4. 2021-01-06T22:00:46.797 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../040-python3-bs4_4.6.0-1_all.deb ... 2021-01-06T22:00:46.848 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-01-06T22:00:47.008 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-logutils. 2021-01-06T22:00:47.023 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../034-python3-logutils_0.3.3-5_all.deb ... 2021-01-06T22:00:47.074 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-01-06T22:00:47.176 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-waitress. 2021-01-06T22:00:47.192 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../041-python3-waitress_1.0.1-1_all.deb ... 2021-01-06T22:00:47.234 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-01-06T22:00:47.377 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-markupsafe. 2021-01-06T22:00:47.392 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../035-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-01-06T22:00:47.436 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-01-06T22:00:47.554 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-tempita. 2021-01-06T22:00:47.570 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-2_all.deb ... 2021-01-06T22:00:47.612 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-01-06T22:00:47.747 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-mako. 2021-01-06T22:00:47.762 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../036-python3-mako_1.0.7+ds1-1_all.deb ... 2021-01-06T22:00:47.805 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-01-06T22:00:47.925 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-paste. 2021-01-06T22:00:47.939 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../043-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-01-06T22:00:47.983 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-01-06T22:00:48.116 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-simplegeneric. 2021-01-06T22:00:48.131 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../037-python3-simplegeneric_0.8.1-1_all.deb ... 2021-01-06T22:00:48.175 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-01-06T22:00:48.286 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-01-06T22:00:48.301 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-01-06T22:00:48.344 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-01-06T22:00:48.477 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-singledispatch. 2021-01-06T22:00:48.491 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../038-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-01-06T22:00:48.536 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-01-06T22:00:48.663 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-pastedeploy. 2021-01-06T22:00:48.678 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../045-python3-pastedeploy_1.5.2-4_all.deb ... 2021-01-06T22:00:48.722 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-01-06T22:00:48.830 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-webob. 2021-01-06T22:00:48.844 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../039-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-01-06T22:00:48.888 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-01-06T22:00:49.208 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-bs4. 2021-01-06T22:00:49.223 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../040-python3-bs4_4.6.0-1_all.deb ... 2021-01-06T22:00:49.266 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-01-06T22:00:49.333 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-webtest. 2021-01-06T22:00:49.347 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../046-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-01-06T22:00:49.391 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-01-06T22:00:49.769 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-pecan. 2021-01-06T22:00:49.783 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../047-python3-pecan_1.2.1-2_all.deb ... 2021-01-06T22:00:49.830 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-01-06T22:00:49.894 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-waitress. 2021-01-06T22:00:49.908 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../041-python3-waitress_1.0.1-1_all.deb ... 2021-01-06T22:00:49.952 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-01-06T22:00:50.216 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-werkzeug. 2021-01-06T22:00:50.228 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../048-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-01-06T22:00:50.265 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-01-06T22:00:50.289 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-tempita. 2021-01-06T22:00:50.303 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-2_all.deb ... 2021-01-06T22:00:50.347 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-01-06T22:00:50.635 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package ceph-mgr. 2021-01-06T22:00:50.651 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-paste. 2021-01-06T22:00:50.651 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../049-ceph-mgr_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:50.664 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../043-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-01-06T22:00:50.693 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking ceph-mgr (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:50.709 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-01-06T22:00:51.037 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-01-06T22:00:51.051 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-01-06T22:00:51.095 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-01-06T22:00:51.172 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package ceph-mon. 2021-01-06T22:00:51.186 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../050-ceph-mon_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:51.230 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking ceph-mon (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:51.390 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pastedeploy. 2021-01-06T22:00:51.404 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../045-python3-pastedeploy_1.5.2-4_all.deb ... 2021-01-06T22:00:51.448 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-01-06T22:00:51.751 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-webtest. 2021-01-06T22:00:51.764 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../046-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-01-06T22:00:51.809 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-01-06T22:00:52.150 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package ceph-osd. 2021-01-06T22:00:52.153 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pecan. 2021-01-06T22:00:52.163 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../051-ceph-osd_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:52.168 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../047-python3-pecan_1.2.1-2_all.deb ... 2021-01-06T22:00:52.200 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking ceph-osd (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:52.212 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-01-06T22:00:52.617 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-werkzeug. 2021-01-06T22:00:52.633 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../048-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-01-06T22:00:52.675 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-01-06T22:00:53.053 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mgr. 2021-01-06T22:00:53.067 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../049-ceph-mgr_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:53.111 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mgr (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:53.581 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mon. 2021-01-06T22:00:53.595 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../050-ceph-mon_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:53.639 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mon (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:54.222 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package ceph. 2021-01-06T22:00:54.235 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../052-ceph_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:54.280 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking ceph (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:54.601 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-osd. 2021-01-06T22:00:54.615 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../051-ceph-osd_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:54.633 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package ceph-fuse. 2021-01-06T22:00:54.647 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../053-ceph-fuse_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:54.659 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-osd (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:54.691 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking ceph-fuse (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:55.103 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package ceph-mds. 2021-01-06T22:00:55.116 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../054-ceph-mds_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:55.153 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking ceph-mds (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:55.640 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package cephadm. 2021-01-06T22:00:55.654 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../055-cephadm_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:55.698 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking cephadm (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:55.984 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-jinja2. 2021-01-06T22:00:56.000 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../056-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-01-06T22:00:56.059 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-01-06T22:00:56.387 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-01-06T22:00:56.400 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:00:56.429 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking ceph-mgr-cephadm (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:56.706 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph. 2021-01-06T22:00:56.720 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../052-ceph_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:56.731 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-repoze.lru. 2021-01-06T22:00:56.747 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../058-python3-repoze.lru_0.7-1_all.deb ... 2021-01-06T22:00:56.764 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:56.790 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-01-06T22:00:57.076 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-routes. 2021-01-06T22:00:57.084 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-fuse. 2021-01-06T22:00:57.092 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../059-python3-routes_2.4.1-1_all.deb ... 2021-01-06T22:00:57.098 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../053-ceph-fuse_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:57.134 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-01-06T22:00:57.142 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-fuse (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:57.454 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-01-06T22:00:57.467 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../060-ceph-mgr-dashboard_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:00:57.512 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking ceph-mgr-dashboard (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:57.545 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mds. 2021-01-06T22:00:57.561 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../054-ceph-mds_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:57.605 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mds (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:58.082 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package cephadm. 2021-01-06T22:00:58.096 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../055-cephadm_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:00:58.140 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking cephadm (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:58.312 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-numpy. 2021-01-06T22:00:58.327 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-01-06T22:00:58.370 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-01-06T22:00:58.485 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-jinja2. 2021-01-06T22:00:58.499 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../056-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-01-06T22:00:58.543 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-01-06T22:00:58.863 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-01-06T22:00:58.878 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:00:58.921 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mgr-cephadm (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:00:58.967 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-decorator. 2021-01-06T22:00:58.982 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2021-01-06T22:00:59.025 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-01-06T22:00:59.224 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-repoze.lru. 2021-01-06T22:00:59.240 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../058-python3-repoze.lru_0.7-1_all.deb ... 2021-01-06T22:00:59.291 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-01-06T22:00:59.379 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-scipy. 2021-01-06T22:00:59.394 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-01-06T22:00:59.445 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-01-06T22:00:59.585 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-routes. 2021-01-06T22:00:59.599 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../059-python3-routes_2.4.1-1_all.deb ... 2021-01-06T22:00:59.644 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-01-06T22:01:00.046 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-01-06T22:01:00.059 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../060-ceph-mgr-dashboard_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:01:00.096 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mgr-dashboard (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:00.712 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-01-06T22:01:00.729 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-01-06T22:01:00.762 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-01-06T22:01:00.912 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-numpy. 2021-01-06T22:01:00.930 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-01-06T22:01:00.979 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-01-06T22:01:01.233 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-joblib. 2021-01-06T22:01:01.246 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2021-01-06T22:01:01.282 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-joblib (0.11-1) ... 2021-01-06T22:01:01.568 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-decorator. 2021-01-06T22:01:01.585 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2021-01-06T22:01:01.626 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-01-06T22:01:01.652 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-sklearn. 2021-01-06T22:01:01.666 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2021-01-06T22:01:01.711 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-01-06T22:01:01.929 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-scipy. 2021-01-06T22:01:01.945 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-01-06T22:01:01.996 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-01-06T22:01:02.276 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-01-06T22:01:02.290 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:01:02.334 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking ceph-mgr-diskprediction-local (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:03.338 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-01-06T22:01:03.354 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-01-06T22:01:03.396 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-01-06T22:01:03.429 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-monotonic. 2021-01-06T22:01:03.443 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2021-01-06T22:01:03.487 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-01-06T22:01:03.800 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-fasteners. 2021-01-06T22:01:03.814 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2021-01-06T22:01:03.833 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-joblib. 2021-01-06T22:01:03.850 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2021-01-06T22:01:03.857 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-01-06T22:01:03.908 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-joblib (0.11-1) ... 2021-01-06T22:01:04.168 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-httplib2. 2021-01-06T22:01:04.181 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2021-01-06T22:01:04.218 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2021-01-06T22:01:04.253 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-sklearn. 2021-01-06T22:01:04.267 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2021-01-06T22:01:04.311 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-01-06T22:01:04.546 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-pyasn1. 2021-01-06T22:01:04.559 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2021-01-06T22:01:04.604 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-01-06T22:01:04.937 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-01-06T22:01:04.941 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-01-06T22:01:04.951 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:01:04.955 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-01-06T22:01:04.995 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mgr-diskprediction-local (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:04.999 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-01-06T22:01:05.319 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-rsa. 2021-01-06T22:01:05.335 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2021-01-06T22:01:05.377 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-01-06T22:01:05.672 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-01-06T22:01:05.688 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-01-06T22:01:05.730 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-01-06T22:01:06.133 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-monotonic. 2021-01-06T22:01:06.147 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2021-01-06T22:01:06.151 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-oauth2client. 2021-01-06T22:01:06.168 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2021-01-06T22:01:06.191 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-01-06T22:01:06.234 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-01-06T22:01:06.502 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-fasteners. 2021-01-06T22:01:06.516 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2021-01-06T22:01:06.560 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-01-06T22:01:06.587 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-setuptools. 2021-01-06T22:01:06.602 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2_all.deb ... 2021-01-06T22:01:06.645 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-01-06T22:01:06.905 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-httplib2. 2021-01-06T22:01:06.920 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2021-01-06T22:01:06.964 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2021-01-06T22:01:07.041 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-websocket. 2021-01-06T22:01:07.057 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-01-06T22:01:07.099 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-01-06T22:01:07.267 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pyasn1. 2021-01-06T22:01:07.280 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2021-01-06T22:01:07.325 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-01-06T22:01:07.426 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-kubernetes. 2021-01-06T22:01:07.435 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-01-06T22:01:07.477 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-01-06T22:01:07.654 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-01-06T22:01:07.668 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-01-06T22:01:07.712 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-01-06T22:01:07.910 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-01-06T22:01:07.926 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:01:07.976 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking ceph-mgr-k8sevents (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:08.040 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-rsa. 2021-01-06T22:01:08.055 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2021-01-06T22:01:08.098 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-01-06T22:01:08.329 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-json-pointer. 2021-01-06T22:01:08.343 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../080-python3-json-pointer_1.10-1_all.deb ... 2021-01-06T22:01:08.427 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-01-06T22:01:08.440 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-01-06T22:01:08.472 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-01-06T22:01:08.485 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-01-06T22:01:08.802 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-jsonpatch. 2021-01-06T22:01:08.817 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../081-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-01-06T22:01:08.863 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-01-06T22:01:08.931 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-oauth2client. 2021-01-06T22:01:08.947 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2021-01-06T22:01:08.990 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-01-06T22:01:09.171 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-01-06T22:01:09.186 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../082-ceph-mgr-rook_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:01:09.229 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking ceph-mgr-rook (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:09.335 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-setuptools. 2021-01-06T22:01:09.350 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2_all.deb ... 2021-01-06T22:01:09.393 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-01-06T22:01:09.550 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libonig4:amd64. 2021-01-06T22:01:09.567 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../083-libonig4_6.7.0-1_amd64.deb ... 2021-01-06T22:01:09.625 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-01-06T22:01:09.797 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-websocket. 2021-01-06T22:01:09.813 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-01-06T22:01:09.855 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-01-06T22:01:10.021 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libjq1:amd64. 2021-01-06T22:01:10.035 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../084-libjq1_1.5+dfsg-2_amd64.deb ... 2021-01-06T22:01:10.079 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-01-06T22:01:10.167 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-kubernetes. 2021-01-06T22:01:10.182 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-01-06T22:01:10.225 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-01-06T22:01:10.348 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package jq. 2021-01-06T22:01:10.362 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../085-jq_1.5+dfsg-2_amd64.deb ... 2021-01-06T22:01:10.398 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-01-06T22:01:10.648 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-01-06T22:01:10.651 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package socat. 2021-01-06T22:01:10.665 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:01:10.667 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../086-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-01-06T22:01:10.706 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mgr-k8sevents (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:10.709 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-01-06T22:01:11.026 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-json-pointer. 2021-01-06T22:01:11.040 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../080-python3-json-pointer_1.10-1_all.deb ... 2021-01-06T22:01:11.096 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-01-06T22:01:11.111 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../087-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-01-06T22:01:11.154 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-01-06T22:01:11.170 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-01-06T22:01:11.449 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package xmlstarlet. 2021-01-06T22:01:11.466 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2_amd64.deb ... 2021-01-06T22:01:11.507 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-01-06T22:01:11.524 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-jsonpatch. 2021-01-06T22:01:11.539 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../081-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-01-06T22:01:11.585 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-01-06T22:01:11.870 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package ceph-test. 2021-01-06T22:01:11.885 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-01-06T22:01:11.886 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../089-ceph-test_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:01:11.900 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../082-ceph-mgr-rook_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:01:11.929 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking ceph-test (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:11.952 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mgr-rook (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:12.290 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libonig4:amd64. 2021-01-06T22:01:12.307 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../083-libonig4_6.7.0-1_amd64.deb ... 2021-01-06T22:01:12.348 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-01-06T22:01:12.712 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libjq1:amd64. 2021-01-06T22:01:12.727 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../084-libjq1_1.5+dfsg-2_amd64.deb ... 2021-01-06T22:01:12.770 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-01-06T22:01:13.056 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package jq. 2021-01-06T22:01:13.069 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../085-jq_1.5+dfsg-2_amd64.deb ... 2021-01-06T22:01:13.114 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-01-06T22:01:13.384 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package socat. 2021-01-06T22:01:13.657 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../086-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-01-06T22:01:13.657 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-01-06T22:01:13.804 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-01-06T22:01:13.821 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../087-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-01-06T22:01:13.863 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-01-06T22:01:14.141 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package xmlstarlet. 2021-01-06T22:01:14.154 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2_amd64.deb ... 2021-01-06T22:01:14.207 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-01-06T22:01:14.596 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-test. 2021-01-06T22:01:14.610 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../089-ceph-test_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:01:14.655 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-test (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:15.708 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-colorama. 2021-01-06T22:01:15.720 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../090-python3-colorama_0.3.7-1_all.deb ... 2021-01-06T22:01:15.766 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-colorama (0.3.7-1) ... 2021-01-06T22:01:16.019 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package cephfs-shell. 2021-01-06T22:01:16.034 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../091-cephfs-shell_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:01:16.077 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking cephfs-shell (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:16.330 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libcephfs-dev. 2021-01-06T22:01:16.347 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../092-libcephfs-dev_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:01:16.408 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libcephfs-dev (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:16.682 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package nvme-cli. 2021-01-06T22:01:16.697 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../093-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-01-06T22:01:16.741 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-01-06T22:01:17.086 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python-attr. 2021-01-06T22:01:17.100 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../094-python-attr_17.4.0-2_all.deb ... 2021-01-06T22:01:17.145 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python-attr (17.4.0-2) ... 2021-01-06T22:01:17.448 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python-funcsigs. 2021-01-06T22:01:17.463 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../095-python-funcsigs_1.0.2-4_all.deb ... 2021-01-06T22:01:17.505 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-01-06T22:01:17.775 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python-pluggy. 2021-01-06T22:01:17.793 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../096-python-pluggy_0.6.0-1_all.deb ... 2021-01-06T22:01:17.833 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-01-06T22:01:18.170 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python-py. 2021-01-06T22:01:18.186 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../097-python-py_1.5.2-1_all.deb ... 2021-01-06T22:01:18.228 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python-py (1.5.2-1) ... 2021-01-06T22:01:18.334 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-colorama. 2021-01-06T22:01:18.350 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../090-python3-colorama_0.3.7-1_all.deb ... 2021-01-06T22:01:18.393 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-colorama (0.3.7-1) ... 2021-01-06T22:01:18.548 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python-pytest. 2021-01-06T22:01:18.565 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../098-python-pytest_3.3.2-2_all.deb ... 2021-01-06T22:01:18.606 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-01-06T22:01:18.654 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package cephfs-shell. 2021-01-06T22:01:18.670 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../091-cephfs-shell_16.0.0-8719-gd0ed162b-1bionic_all.deb ... 2021-01-06T22:01:18.712 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking cephfs-shell (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:18.954 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-attr. 2021-01-06T22:01:18.968 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../099-python3-attr_17.4.0-2_all.deb ... 2021-01-06T22:01:18.974 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libcephfs-dev. 2021-01-06T22:01:18.987 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../092-libcephfs-dev_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:01:19.012 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-01-06T22:01:19.032 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libcephfs-dev (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:19.306 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-cycler. 2021-01-06T22:01:19.323 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../100-python3-cycler_0.10.0-1_all.deb ... 2021-01-06T22:01:19.327 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package nvme-cli. 2021-01-06T22:01:19.344 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../093-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-01-06T22:01:19.356 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-01-06T22:01:19.385 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-01-06T22:01:19.706 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-webencodings. 2021-01-06T22:01:19.732 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python-attr. 2021-01-06T22:01:19.747 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../094-python-attr_17.4.0-2_all.deb ... 2021-01-06T22:01:19.760 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../101-python3-webencodings_0.5-2_all.deb ... 2021-01-06T22:01:19.790 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python-attr (17.4.0-2) ... 2021-01-06T22:01:20.093 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python-funcsigs. 2021-01-06T22:01:20.107 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../095-python-funcsigs_1.0.2-4_all.deb ... 2021-01-06T22:01:20.151 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-01-06T22:01:20.167 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-01-06T22:01:20.470 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-html5lib. 2021-01-06T22:01:20.484 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../102-python3-html5lib_0.999999999-1_all.deb ... 2021-01-06T22:01:20.529 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-01-06T22:01:20.873 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-01-06T22:01:20.888 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../103-python3-lxml_4.2.1-1ubuntu0.3_amd64.deb ... 2021-01-06T22:01:20.930 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python-pluggy. 2021-01-06T22:01:20.931 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-01-06T22:01:20.944 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../096-python-pluggy_0.6.0-1_all.deb ... 2021-01-06T22:01:20.988 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-01-06T22:01:21.292 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python-py. 2021-01-06T22:01:21.307 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../097-python-py_1.5.2-1_all.deb ... 2021-01-06T22:01:21.350 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python-py (1.5.2-1) ... 2021-01-06T22:01:21.377 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-tz. 2021-01-06T22:01:21.391 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../104-python3-tz_2018.3-2_all.deb ... 2021-01-06T22:01:21.435 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-tz (2018.3-2) ... 2021-01-06T22:01:21.671 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python-pytest. 2021-01-06T22:01:21.687 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../098-python-pytest_3.3.2-2_all.deb ... 2021-01-06T22:01:21.729 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-01-06T22:01:21.771 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-matplotlib. 2021-01-06T22:01:21.785 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../105-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-01-06T22:01:21.837 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-01-06T22:01:22.077 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-attr. 2021-01-06T22:01:22.093 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../099-python3-attr_17.4.0-2_all.deb ... 2021-01-06T22:01:22.136 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-01-06T22:01:22.447 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-cycler. 2021-01-06T22:01:22.464 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../100-python3-cycler_0.10.0-1_all.deb ... 2021-01-06T22:01:22.502 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-olefile. 2021-01-06T22:01:22.505 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-01-06T22:01:22.518 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../106-python3-olefile_0.45.1-1_all.deb ... 2021-01-06T22:01:22.560 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-01-06T22:01:22.808 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-webencodings. 2021-01-06T22:01:22.823 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../101-python3-webencodings_0.5-2_all.deb ... 2021-01-06T22:01:22.863 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-pastescript. 2021-01-06T22:01:22.866 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-01-06T22:01:22.878 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-2_all.deb ... 2021-01-06T22:01:22.922 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-01-06T22:01:23.194 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-html5lib. 2021-01-06T22:01:23.208 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../102-python3-html5lib_0.999999999-1_all.deb ... 2021-01-06T22:01:23.252 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-01-06T22:01:23.325 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-pil:amd64. 2021-01-06T22:01:23.339 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../108-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2021-01-06T22:01:23.392 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2021-01-06T22:01:23.640 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-01-06T22:01:23.655 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../103-python3-lxml_4.2.1-1ubuntu0.3_amd64.deb ... 2021-01-06T22:01:23.698 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-01-06T22:01:23.763 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-pluggy. 2021-01-06T22:01:23.778 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../109-python3-pluggy_0.6.0-1_all.deb ... 2021-01-06T22:01:23.821 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-01-06T22:01:24.132 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-py. 2021-01-06T22:01:24.143 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-tz. 2021-01-06T22:01:24.147 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../110-python3-py_1.5.2-1_all.deb ... 2021-01-06T22:01:24.158 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../104-python3-tz_2018.3-2_all.deb ... 2021-01-06T22:01:24.190 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-py (1.5.2-1) ... 2021-01-06T22:01:24.201 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-tz (2018.3-2) ... 2021-01-06T22:01:24.510 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-pyinotify. 2021-01-06T22:01:24.524 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1_all.deb ... 2021-01-06T22:01:24.538 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-matplotlib. 2021-01-06T22:01:24.552 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../105-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-01-06T22:01:24.568 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-01-06T22:01:24.604 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-01-06T22:01:24.996 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-pytest. 2021-01-06T22:01:25.013 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../112-python3-pytest_3.3.2-2_all.deb ... 2021-01-06T22:01:25.054 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-01-06T22:01:25.260 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-olefile. 2021-01-06T22:01:25.277 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../106-python3-olefile_0.45.1-1_all.deb ... 2021-01-06T22:01:25.318 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-01-06T22:01:25.383 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-simplejson. 2021-01-06T22:01:25.400 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../113-python3-simplejson_3.13.2-1_amd64.deb ... 2021-01-06T22:01:25.441 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-01-06T22:01:25.655 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pastescript. 2021-01-06T22:01:25.672 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-2_all.deb ... 2021-01-06T22:01:25.713 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-01-06T22:01:25.802 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package python3-tk:amd64. 2021-01-06T22:01:25.818 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../114-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-01-06T22:01:25.869 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-01-06T22:01:26.084 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pil:amd64. 2021-01-06T22:01:26.101 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../108-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2021-01-06T22:01:26.142 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2021-01-06T22:01:26.307 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package radosgw. 2021-01-06T22:01:26.324 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../115-radosgw_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:01:26.365 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking radosgw (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:26.539 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pluggy. 2021-01-06T22:01:26.556 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../109-python3-pluggy_0.6.0-1_all.deb ... 2021-01-06T22:01:26.597 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-01-06T22:01:26.942 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-py. 2021-01-06T22:01:26.956 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../110-python3-py_1.5.2-1_all.deb ... 2021-01-06T22:01:27.008 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-py (1.5.2-1) ... 2021-01-06T22:01:27.353 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pyinotify. 2021-01-06T22:01:27.368 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1_all.deb ... 2021-01-06T22:01:27.420 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-01-06T22:01:27.427 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package rbd-fuse. 2021-01-06T22:01:27.444 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../116-rbd-fuse_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:01:27.485 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking rbd-fuse (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:27.715 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pytest. 2021-01-06T22:01:27.732 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../112-python3-pytest_3.3.2-2_all.deb ... 2021-01-06T22:01:27.773 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-01-06T22:01:27.798 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-01-06T22:01:28.107 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-01-06T22:01:28.118 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-simplejson. 2021-01-06T22:01:28.135 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../113-python3-simplejson_3.13.2-1_amd64.deb ... 2021-01-06T22:01:28.177 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-01-06T22:01:28.477 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-01-06T22:01:28.514 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-tk:amd64. 2021-01-06T22:01:28.527 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../114-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-01-06T22:01:28.572 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-01-06T22:01:28.612 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-01-06T22:01:29.085 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package radosgw. 2021-01-06T22:01:29.099 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../115-radosgw_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:01:29.144 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking radosgw (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:29.239 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2021-01-06T22:01:29.624 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-01-06T22:01:29.908 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-webencodings (0.5-2) ... 2021-01-06T22:01:30.164 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package rbd-fuse. 2021-01-06T22:01:30.181 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../116-rbd-fuse_16.0.0-8719-gd0ed162b-1bionic_amd64.deb ... 2021-01-06T22:01:30.201 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python-attr (17.4.0-2) ... 2021-01-06T22:01:30.222 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking rbd-fuse (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:30.402 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-01-06T22:01:30.532 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-01-06T22:01:30.545 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-py (1.5.2-1) ... 2021-01-06T22:01:30.854 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-01-06T22:01:30.871 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-01-06T22:01:31.229 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-01-06T22:01:31.304 INFO:teuthology.orchestra.run.smithi091.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-01-06T22:01:31.354 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-01-06T22:01:31.616 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-01-06T22:01:31.917 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-ceph-argparse (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:31.949 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2021-01-06T22:01:32.218 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-01-06T22:01:32.359 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-01-06T22:01:32.519 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-01-06T22:01:32.662 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-01-06T22:01:32.668 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-webencodings (0.5-2) ... 2021-01-06T22:01:32.961 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python-attr (17.4.0-2) ... 2021-01-06T22:01:33.163 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-01-06T22:01:33.298 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-py (1.5.2-1) ... 2021-01-06T22:01:33.615 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-01-06T22:01:33.650 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-01-06T22:01:33.933 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-01-06T22:01:34.017 INFO:teuthology.orchestra.run.smithi203.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-01-06T22:01:34.234 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python-py (1.5.2-1) ... 2021-01-06T22:01:34.294 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-01-06T22:01:34.460 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-01-06T22:01:34.621 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-ceph-argparse (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:34.795 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-01-06T22:01:34.893 INFO:teuthology.orchestra.run.smithi091.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-01-06T22:01:34.930 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-01-06T22:01:35.129 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-01-06T22:01:35.239 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-01-06T22:01:35.280 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-01-06T22:01:35.357 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-01-06T22:01:35.482 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-colorama (0.3.7-1) ... 2021-01-06T22:01:35.783 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-01-06T22:01:36.067 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-01-06T22:01:36.302 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-01-06T22:01:36.377 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-01-06T22:01:36.620 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-01-06T22:01:36.669 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-01-06T22:01:36.922 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python-py (1.5.2-1) ... 2021-01-06T22:01:37.046 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-01-06T22:01:37.158 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-01-06T22:01:37.380 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-01-06T22:01:37.510 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-01-06T22:01:37.617 INFO:teuthology.orchestra.run.smithi203.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-01-06T22:01:37.665 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up cephadm (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:37.895 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-01-06T22:01:38.021 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-01-06T22:01:38.222 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-colorama (0.3.7-1) ... 2021-01-06T22:01:38.316 INFO:teuthology.orchestra.run.smithi091.stdout:Adding system user cephadm....done 2021-01-06T22:01:38.471 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-01-06T22:01:38.524 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-01-06T22:01:38.781 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-01-06T22:01:38.825 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-01-06T22:01:38.907 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-01-06T22:01:39.042 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-01-06T22:01:39.126 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-01-06T22:01:39.401 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-01-06T22:01:39.534 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-01-06T22:01:39.702 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-01-06T22:01:39.787 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-01-06T22:01:39.836 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-01-06T22:01:40.105 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-01-06T22:01:40.196 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-ceph-common (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:40.398 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up cephadm (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:40.548 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-monotonic (1.1-2) ... 2021-01-06T22:01:40.832 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-01-06T22:01:40.967 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-01-06T22:01:41.074 INFO:teuthology.orchestra.run.smithi203.stdout:Adding system user cephadm....done 2021-01-06T22:01:41.239 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-01-06T22:01:41.318 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-01-06T22:01:41.540 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-01-06T22:01:41.703 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-01-06T22:01:41.703 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-01-06T22:01:41.775 INFO:teuthology.orchestra.run.smithi091.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-01-06T22:01:41.808 INFO:teuthology.orchestra.run.smithi091.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-01-06T22:01:41.809 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-01-06T22:01:42.038 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-01-06T22:01:42.144 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-01-06T22:01:42.349 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2021-01-06T22:01:42.437 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-01-06T22:01:42.565 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-01-06T22:01:42.634 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-01-06T22:01:42.898 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-ceph-common (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:42.983 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-attr (17.4.0-2) ... 2021-01-06T22:01:43.225 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-monotonic (1.1-2) ... 2021-01-06T22:01:43.293 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-01-06T22:01:43.494 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libjaeger (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:43.559 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-01-06T22:01:43.620 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-01-06T22:01:43.687 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-01-06T22:01:44.054 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-01-06T22:01:44.062 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-01-06T22:01:44.144 INFO:teuthology.orchestra.run.smithi091.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-01-06T22:01:44.406 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up liboath0 (2.6.1-1) ... 2021-01-06T22:01:44.413 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-01-06T22:01:44.503 INFO:teuthology.orchestra.run.smithi203.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-01-06T22:01:44.527 INFO:teuthology.orchestra.run.smithi203.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-01-06T22:01:44.599 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-01-06T22:01:44.725 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-01-06T22:01:44.740 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-01-06T22:01:44.968 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-01-06T22:01:45.042 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2021-01-06T22:01:45.094 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-01-06T22:01:45.351 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-01-06T22:01:45.471 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-01-06T22:01:45.597 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-01-06T22:01:45.703 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-attr (17.4.0-2) ... 2021-01-06T22:01:45.898 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-tz (2018.3-2) ... 2021-01-06T22:01:46.021 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-01-06T22:01:46.207 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-01-06T22:01:46.238 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libjaeger (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:46.364 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-01-06T22:01:46.494 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-01-06T22:01:46.792 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-01-06T22:01:46.803 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-joblib (0.11-1) ... 2021-01-06T22:01:46.881 INFO:teuthology.orchestra.run.smithi203.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-01-06T22:01:47.143 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up liboath0 (2.6.1-1) ... 2021-01-06T22:01:47.188 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-routes (2.4.1-1) ... 2021-01-06T22:01:47.328 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-01-06T22:01:47.454 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-01-06T22:01:47.490 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-01-06T22:01:47.697 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-01-06T22:01:47.807 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-01-06T22:01:47.832 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-01-06T22:01:48.125 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2021-01-06T22:01:48.234 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-01-06T22:01:48.360 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-01-06T22:01:48.409 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-01-06T22:01:48.653 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-tz (2018.3-2) ... 2021-01-06T22:01:48.803 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-01-06T22:01:49.029 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-01-06T22:01:49.121 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up jq (1.5+dfsg-2) ... 2021-01-06T22:01:49.253 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-01-06T22:01:49.347 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-01-06T22:01:49.665 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-joblib (0.11-1) ... 2021-01-06T22:01:50.033 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-routes (2.4.1-1) ... 2021-01-06T22:01:50.351 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-01-06T22:01:50.652 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-01-06T22:01:51.016 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-01-06T22:01:51.334 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-01-06T22:01:51.420 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2021-01-06T22:01:51.652 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-01-06T22:01:51.780 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-01-06T22:01:51.795 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-01-06T22:01:52.141 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-01-06T22:01:52.171 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2021-01-06T22:01:52.459 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up jq (1.5+dfsg-2) ... 2021-01-06T22:01:52.464 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python-pytest (3.3.2-2) ... 2021-01-06T22:01:52.585 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-01-06T22:01:52.799 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-01-06T22:01:52.925 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-01-06T22:01:53.051 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up librados2 (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:53.185 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-01-06T22:01:53.495 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libcephfs2 (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:53.621 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libcephfs-dev (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:53.747 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-rados (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:53.747 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-01-06T22:01:53.958 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-01-06T22:01:54.057 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-01-06T22:01:54.300 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-01-06T22:01:54.392 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-01-06T22:01:54.518 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-01-06T22:01:54.610 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-01-06T22:01:54.920 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2021-01-06T22:01:54.961 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-01-06T22:01:55.229 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python-pytest (3.3.2-2) ... 2021-01-06T22:01:55.481 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-01-06T22:01:55.607 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-01-06T22:01:55.733 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up librados2 (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:55.832 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-01-06T22:01:55.859 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-01-06T22:01:56.169 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libcephfs2 (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:56.270 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libcephfs-dev (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:56.379 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-rados (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:56.565 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-01-06T22:01:56.645 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-01-06T22:01:56.884 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-01-06T22:01:56.938 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libradosstriper1 (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:57.065 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-cephfs (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:57.185 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-01-06T22:01:57.348 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up librgw2 (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:57.478 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-01-06T22:01:57.491 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up ceph-fuse (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:57.709 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-01-06T22:01:57.710 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-01-06T22:01:58.010 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up librbd1 (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:58.144 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up cephfs-shell (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:58.278 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-01-06T22:01:58.314 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-01-06T22:01:58.377 INFO:teuthology.orchestra.run.smithi091.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-01-06T22:01:58.406 INFO:teuthology.orchestra.run.smithi091.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-01-06T22:01:58.715 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-01-06T22:01:59.130 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-01-06T22:01:59.384 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up ceph-mgr-modules-core (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:59.405 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libradosstriper1 (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:59.506 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-cephfs (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:59.510 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-01-06T22:01:59.791 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up librgw2 (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:59.828 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-rgw (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:59.917 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-fuse (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:01:59.954 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up rbd-fuse (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:00.080 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up python3-rbd (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:00.133 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-01-06T22:02:00.134 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-01-06T22:02:00.206 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up ceph-common (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:00.386 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up librbd1 (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:00.487 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up cephfs-shell (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:00.588 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-01-06T22:02:00.661 INFO:teuthology.orchestra.run.smithi203.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-01-06T22:02:00.693 INFO:teuthology.orchestra.run.smithi203.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-01-06T22:02:00.810 INFO:teuthology.orchestra.run.smithi091.stdout:Adding group ceph....done 2021-01-06T22:02:01.000 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-01-06T22:02:01.179 INFO:teuthology.orchestra.run.smithi091.stdout:Adding system user ceph....done 2021-01-06T22:02:01.363 INFO:teuthology.orchestra.run.smithi091.stdout:Setting system user ceph properties....done 2021-01-06T22:02:01.526 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-01-06T22:02:01.662 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mgr-modules-core (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:01.762 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-01-06T22:02:01.776 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-01-06T22:02:02.047 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-rgw (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:02.101 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up radosgw (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:02.147 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up rbd-fuse (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:02.257 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-rbd (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:02.337 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-01-06T22:02:02.337 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-01-06T22:02:02.375 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-common (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:02.744 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up ceph-test (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:02.837 INFO:teuthology.orchestra.run.smithi203.stdout:Adding group ceph....done 2021-01-06T22:02:02.870 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up ceph-base (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:03.204 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-01-06T22:02:03.208 INFO:teuthology.orchestra.run.smithi203.stdout:Adding system user ceph....done 2021-01-06T22:02:03.383 INFO:teuthology.orchestra.run.smithi203.stdout:Setting system user ceph properties....done 2021-01-06T22:02:03.490 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up ceph-mgr (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:03.541 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-01-06T22:02:03.706 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-01-06T22:02:03.706 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-01-06T22:02:03.777 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-01-06T22:02:04.007 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up ceph-mgr-diskprediction-local (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:04.081 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up radosgw (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:04.126 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up ceph-osd (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:04.342 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-01-06T22:02:04.343 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-01-06T22:02:04.570 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-01-06T22:02:04.570 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-01-06T22:02:04.733 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-test (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:04.852 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up ceph-mgr-cephadm (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:04.867 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-base (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:04.969 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up ceph-mgr-rook (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:05.113 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up ceph-mds (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:05.196 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-01-06T22:02:05.330 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-01-06T22:02:05.331 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-01-06T22:02:05.485 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mgr (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:05.613 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up ceph-mon (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:05.686 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-01-06T22:02:05.686 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-01-06T22:02:05.829 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-01-06T22:02:05.829 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-01-06T22:02:05.962 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mgr-diskprediction-local (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:06.079 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-osd (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:06.099 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up ceph-mgr-k8sevents (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:06.203 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up ceph-mgr-dashboard (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:06.318 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up ceph (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:06.419 INFO:teuthology.orchestra.run.smithi091.stdout:Processing triggers for systemd (237-3ubuntu10.43) ... 2021-01-06T22:02:06.493 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-01-06T22:02:06.493 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-01-06T22:02:06.645 INFO:teuthology.orchestra.run.smithi091.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-01-06T22:02:06.783 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mgr-cephadm (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:06.926 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mgr-rook (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:07.070 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mds (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:07.277 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-01-06T22:02:07.278 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-01-06T22:02:07.562 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mon (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:07.755 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-01-06T22:02:07.755 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-01-06T22:02:08.021 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mgr-k8sevents (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:08.122 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mgr-dashboard (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:08.232 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph (16.0.0-8719-gd0ed162b-1bionic) ... 2021-01-06T22:02:08.351 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for systemd (237-3ubuntu10.43) ... 2021-01-06T22:02:08.568 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-01-06T22:02:08.672 INFO:teuthology.orchestra.run.smithi091.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-01-06T22:02:08.715 INFO:teuthology.orchestra.run.smithi091.stdout:ureadahead will be reprofiled on next reboot 2021-01-06T22:02:08.773 INFO:teuthology.orchestra.run.smithi091.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.4) ... 2021-01-06T22:02:09.204 INFO:teuthology.orchestra.run.smithi091.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-01-06T22:02:09.206 DEBUG:teuthology.orchestra.run.smithi091:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2021-01-06T22:02:09.288 INFO:teuthology.orchestra.run.smithi091.stdout:Reading package lists... 2021-01-06T22:02:09.550 INFO:teuthology.orchestra.run.smithi091.stdout:Building dependency tree... 2021-01-06T22:02:09.551 INFO:teuthology.orchestra.run.smithi091.stdout:Reading state information... 2021-01-06T22:02:09.733 INFO:teuthology.orchestra.run.smithi091.stdout:libtool-bin is already the newest version (2.4.6-2). 2021-01-06T22:02:09.733 INFO:teuthology.orchestra.run.smithi091.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2021-01-06T22:02:09.734 INFO:teuthology.orchestra.run.smithi091.stdout:iproute2 is already the newest version (4.15.0-2ubuntu1.2). 2021-01-06T22:02:09.734 INFO:teuthology.orchestra.run.smithi091.stdout:util-linux is already the newest version (2.31.1-0.4ubuntu3.7). 2021-01-06T22:02:09.734 INFO:teuthology.orchestra.run.smithi091.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2021-01-06T22:02:09.734 INFO:teuthology.orchestra.run.smithi091.stdout:The following additional packages will be installed: 2021-01-06T22:02:09.734 INFO:teuthology.orchestra.run.smithi091.stdout: iputils-arping libbison-dev libfl-dev libfl2 libmbim-glib4 libmbim-proxy 2021-01-06T22:02:09.735 INFO:teuthology.orchestra.run.smithi091.stdout: libmm-glib0 libndp0 libnm0 libpolkit-agent-1-0 libpolkit-backend-1-0 2021-01-06T22:02:09.735 INFO:teuthology.orchestra.run.smithi091.stdout: libqmi-glib5 libqmi-proxy libteamdctl0 modemmanager network-manager-pptp 2021-01-06T22:02:09.736 INFO:teuthology.orchestra.run.smithi091.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2021-01-06T22:02:09.736 INFO:teuthology.orchestra.run.smithi091.stdout: zlib1g-dev 2021-01-06T22:02:09.736 INFO:teuthology.orchestra.run.smithi091.stdout:Suggested packages: 2021-01-06T22:02:09.736 INFO:teuthology.orchestra.run.smithi091.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils comgt 2021-01-06T22:02:09.736 INFO:teuthology.orchestra.run.smithi091.stdout: wvdial wpagui libengine-pkcs11-openssl 2021-01-06T22:02:09.769 INFO:teuthology.orchestra.run.smithi091.stdout:The following NEW packages will be installed: 2021-01-06T22:02:09.770 INFO:teuthology.orchestra.run.smithi091.stdout: bison dump flex indent iputils-arping libaio-dev libbison-dev libelf-dev 2021-01-06T22:02:09.770 INFO:teuthology.orchestra.run.smithi091.stdout: libfl-dev libfl2 libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 2021-01-06T22:02:09.770 INFO:teuthology.orchestra.run.smithi091.stdout: libpolkit-agent-1-0 libpolkit-backend-1-0 libqmi-glib5 libqmi-proxy 2021-01-06T22:02:09.771 INFO:teuthology.orchestra.run.smithi091.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2021-01-06T22:02:09.771 INFO:teuthology.orchestra.run.smithi091.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2021-01-06T22:02:09.772 INFO:teuthology.orchestra.run.smithi091.stdout: zlib1g-dev 2021-01-06T22:02:09.974 INFO:teuthology.orchestra.run.smithi091.stdout:0 upgraded, 31 newly installed, 0 to remove and 9 not upgraded. 2021-01-06T22:02:09.975 INFO:teuthology.orchestra.run.smithi091.stdout:Need to get 7,862 kB of archives. 2021-01-06T22:02:09.975 INFO:teuthology.orchestra.run.smithi091.stdout:After this operation, 32.0 MB of additional disk space will be used. 2021-01-06T22:02:09.975 INFO:teuthology.orchestra.run.smithi091.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2021-01-06T22:02:10.430 INFO:teuthology.orchestra.run.smithi091.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2021-01-06T22:02:10.514 INFO:teuthology.orchestra.run.smithi091.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2021-01-06T22:02:10.574 INFO:teuthology.orchestra.run.smithi091.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2021-01-06T22:02:10.595 INFO:teuthology.orchestra.run.smithi091.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2021-01-06T22:02:10.602 INFO:teuthology.orchestra.run.smithi091.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 iputils-arping amd64 3:20161105-1ubuntu3 [29.7 kB] 2021-01-06T22:02:10.604 INFO:teuthology.orchestra.run.smithi091.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2021-01-06T22:02:10.605 INFO:teuthology.orchestra.run.smithi091.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2021-01-06T22:02:10.620 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-01-06T22:02:10.621 INFO:teuthology.orchestra.run.smithi091.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2021-01-06T22:02:10.626 INFO:teuthology.orchestra.run.smithi091.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2021-01-06T22:02:10.626 INFO:teuthology.orchestra.run.smithi091.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2021-01-06T22:02:10.627 INFO:teuthology.orchestra.run.smithi091.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-glib4 amd64 1.18.0-1~ubuntu18.04.1 [91.4 kB] 2021-01-06T22:02:10.655 INFO:teuthology.orchestra.run.smithi203.stdout:ureadahead will be reprofiled on next reboot 2021-01-06T22:02:10.680 INFO:teuthology.orchestra.run.smithi091.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-proxy amd64 1.18.0-1~ubuntu18.04.1 [5,572 B] 2021-01-06T22:02:10.680 INFO:teuthology.orchestra.run.smithi091.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmm-glib0 amd64 1.10.0-1~ubuntu18.04.2 [179 kB] 2021-01-06T22:02:10.705 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.4) ... 2021-01-06T22:02:10.761 INFO:teuthology.orchestra.run.smithi091.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 libndp0 amd64 1.6-1 [10.7 kB] 2021-01-06T22:02:10.762 INFO:teuthology.orchestra.run.smithi091.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libnm0 amd64 1.10.6-2ubuntu1.4 [298 kB] 2021-01-06T22:02:10.774 INFO:teuthology.orchestra.run.smithi091.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-agent-1-0 amd64 0.105-20ubuntu0.18.04.5 [14.9 kB] 2021-01-06T22:02:10.774 INFO:teuthology.orchestra.run.smithi091.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-backend-1-0 amd64 0.105-20ubuntu0.18.04.5 [36.3 kB] 2021-01-06T22:02:10.776 INFO:teuthology.orchestra.run.smithi091.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-glib5 amd64 1.22.0-1.2~ubuntu18.04.1 [494 kB] 2021-01-06T22:02:10.797 INFO:teuthology.orchestra.run.smithi091.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-proxy amd64 1.22.0-1.2~ubuntu18.04.1 [5,632 B] 2021-01-06T22:02:10.798 INFO:teuthology.orchestra.run.smithi091.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.7 [1,566 kB] 2021-01-06T22:02:10.887 INFO:teuthology.orchestra.run.smithi091.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 libteamdctl0 amd64 1.26-1 [11.6 kB] 2021-01-06T22:02:10.888 INFO:teuthology.orchestra.run.smithi091.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 modemmanager amd64 1.10.0-1~ubuntu18.04.2 [728 kB] 2021-01-06T22:02:10.927 INFO:teuthology.orchestra.run.smithi091.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 wpasupplicant amd64 2:2.6-15ubuntu2.6 [955 kB] 2021-01-06T22:02:10.960 INFO:teuthology.orchestra.run.smithi091.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 policykit-1 amd64 0.105-20ubuntu0.18.04.5 [53.5 kB] 2021-01-06T22:02:10.962 INFO:teuthology.orchestra.run.smithi091.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 network-manager amd64 1.10.6-2ubuntu1.4 [1,505 kB] 2021-01-06T22:02:10.981 INFO:teuthology.orchestra.run.smithi091.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 ppp amd64 2.4.7-2+2ubuntu1.3 [328 kB] 2021-01-06T22:02:10.988 INFO:teuthology.orchestra.run.smithi091.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 pptp-linux amd64 1.9.0+ds-2 [41.8 kB] 2021-01-06T22:02:10.988 INFO:teuthology.orchestra.run.smithi091.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 network-manager-pptp amd64 1.2.6-1 [30.0 kB] 2021-01-06T22:02:10.989 INFO:teuthology.orchestra.run.smithi091.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch-data all 20170806-2 [30.7 kB] 2021-01-06T22:02:11.005 INFO:teuthology.orchestra.run.smithi091.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu1 [53.6 kB] 2021-01-06T22:02:11.147 INFO:teuthology.orchestra.run.smithi203.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-01-06T22:02:11.149 DEBUG:teuthology.orchestra.run.smithi203:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2021-01-06T22:02:11.228 INFO:teuthology.orchestra.run.smithi203.stdout:Reading package lists... 2021-01-06T22:02:11.482 INFO:teuthology.orchestra.run.smithi091.stdout:Fetched 7,862 kB in 1s (6,462 kB/s) 2021-01-06T22:02:11.487 INFO:teuthology.orchestra.run.smithi203.stdout:Building dependency tree... 2021-01-06T22:02:11.488 INFO:teuthology.orchestra.run.smithi203.stdout:Reading state information... 2021-01-06T22:02:11.612 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package flex. 2021-01-06T22:02:11.664 INFO:teuthology.orchestra.run.smithi091.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 145523 files and directories currently installed.) 2021-01-06T22:02:11.668 INFO:teuthology.orchestra.run.smithi203.stdout:libtool-bin is already the newest version (2.4.6-2). 2021-01-06T22:02:11.668 INFO:teuthology.orchestra.run.smithi203.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2021-01-06T22:02:11.668 INFO:teuthology.orchestra.run.smithi203.stdout:iproute2 is already the newest version (4.15.0-2ubuntu1.2). 2021-01-06T22:02:11.669 INFO:teuthology.orchestra.run.smithi203.stdout:util-linux is already the newest version (2.31.1-0.4ubuntu3.7). 2021-01-06T22:02:11.669 INFO:teuthology.orchestra.run.smithi203.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2021-01-06T22:02:11.669 INFO:teuthology.orchestra.run.smithi203.stdout:The following additional packages will be installed: 2021-01-06T22:02:11.670 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2021-01-06T22:02:11.670 INFO:teuthology.orchestra.run.smithi203.stdout: iputils-arping libbison-dev libfl-dev libfl2 libmbim-glib4 libmbim-proxy 2021-01-06T22:02:11.670 INFO:teuthology.orchestra.run.smithi203.stdout: libmm-glib0 libndp0 libnm0 libpolkit-agent-1-0 libpolkit-backend-1-0 2021-01-06T22:02:11.671 INFO:teuthology.orchestra.run.smithi203.stdout: libqmi-glib5 libqmi-proxy libteamdctl0 modemmanager network-manager-pptp 2021-01-06T22:02:11.671 INFO:teuthology.orchestra.run.smithi203.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2021-01-06T22:02:11.671 INFO:teuthology.orchestra.run.smithi203.stdout: zlib1g-dev 2021-01-06T22:02:11.672 INFO:teuthology.orchestra.run.smithi203.stdout:Suggested packages: 2021-01-06T22:02:11.673 INFO:teuthology.orchestra.run.smithi203.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils comgt 2021-01-06T22:02:11.673 INFO:teuthology.orchestra.run.smithi203.stdout: wvdial wpagui libengine-pkcs11-openssl 2021-01-06T22:02:11.706 INFO:teuthology.orchestra.run.smithi203.stdout:The following NEW packages will be installed: 2021-01-06T22:02:11.707 INFO:teuthology.orchestra.run.smithi203.stdout: bison dump flex indent iputils-arping libaio-dev libbison-dev libelf-dev 2021-01-06T22:02:11.707 INFO:teuthology.orchestra.run.smithi203.stdout: libfl-dev libfl2 libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 2021-01-06T22:02:11.708 INFO:teuthology.orchestra.run.smithi203.stdout: libpolkit-agent-1-0 libpolkit-backend-1-0 libqmi-glib5 libqmi-proxy 2021-01-06T22:02:11.708 INFO:teuthology.orchestra.run.smithi203.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2021-01-06T22:02:11.708 INFO:teuthology.orchestra.run.smithi203.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2021-01-06T22:02:11.709 INFO:teuthology.orchestra.run.smithi203.stdout: zlib1g-dev 2021-01-06T22:02:11.711 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking flex (2.6.4-6) ... 2021-01-06T22:02:11.895 INFO:teuthology.orchestra.run.smithi203.stdout:0 upgraded, 31 newly installed, 0 to remove and 9 not upgraded. 2021-01-06T22:02:11.895 INFO:teuthology.orchestra.run.smithi203.stdout:Need to get 7,862 kB of archives. 2021-01-06T22:02:11.896 INFO:teuthology.orchestra.run.smithi203.stdout:After this operation, 32.0 MB of additional disk space will be used. 2021-01-06T22:02:11.896 INFO:teuthology.orchestra.run.smithi203.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2021-01-06T22:02:12.090 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libbison-dev:amd64. 2021-01-06T22:02:12.105 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2021-01-06T22:02:12.148 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2021-01-06T22:02:12.354 INFO:teuthology.orchestra.run.smithi203.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2021-01-06T22:02:12.436 INFO:teuthology.orchestra.run.smithi203.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2021-01-06T22:02:12.468 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package bison. 2021-01-06T22:02:12.482 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2021-01-06T22:02:12.486 INFO:teuthology.orchestra.run.smithi203.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2021-01-06T22:02:12.504 INFO:teuthology.orchestra.run.smithi203.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2021-01-06T22:02:12.517 INFO:teuthology.orchestra.run.smithi203.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 iputils-arping amd64 3:20161105-1ubuntu3 [29.7 kB] 2021-01-06T22:02:12.519 INFO:teuthology.orchestra.run.smithi203.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2021-01-06T22:02:12.520 INFO:teuthology.orchestra.run.smithi203.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2021-01-06T22:02:12.522 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2021-01-06T22:02:12.536 INFO:teuthology.orchestra.run.smithi203.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2021-01-06T22:02:12.540 INFO:teuthology.orchestra.run.smithi203.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2021-01-06T22:02:12.541 INFO:teuthology.orchestra.run.smithi203.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2021-01-06T22:02:12.542 INFO:teuthology.orchestra.run.smithi203.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-glib4 amd64 1.18.0-1~ubuntu18.04.1 [91.4 kB] 2021-01-06T22:02:12.601 INFO:teuthology.orchestra.run.smithi203.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-proxy amd64 1.18.0-1~ubuntu18.04.1 [5,572 B] 2021-01-06T22:02:12.602 INFO:teuthology.orchestra.run.smithi203.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmm-glib0 amd64 1.10.0-1~ubuntu18.04.2 [179 kB] 2021-01-06T22:02:12.611 INFO:teuthology.orchestra.run.smithi203.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 libndp0 amd64 1.6-1 [10.7 kB] 2021-01-06T22:02:12.612 INFO:teuthology.orchestra.run.smithi203.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libnm0 amd64 1.10.6-2ubuntu1.4 [298 kB] 2021-01-06T22:02:12.683 INFO:teuthology.orchestra.run.smithi203.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-agent-1-0 amd64 0.105-20ubuntu0.18.04.5 [14.9 kB] 2021-01-06T22:02:12.684 INFO:teuthology.orchestra.run.smithi203.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-backend-1-0 amd64 0.105-20ubuntu0.18.04.5 [36.3 kB] 2021-01-06T22:02:12.686 INFO:teuthology.orchestra.run.smithi203.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-glib5 amd64 1.22.0-1.2~ubuntu18.04.1 [494 kB] 2021-01-06T22:02:12.705 INFO:teuthology.orchestra.run.smithi203.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-proxy amd64 1.22.0-1.2~ubuntu18.04.1 [5,632 B] 2021-01-06T22:02:12.706 INFO:teuthology.orchestra.run.smithi203.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.7 [1,566 kB] 2021-01-06T22:02:12.772 INFO:teuthology.orchestra.run.smithi203.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 libteamdctl0 amd64 1.26-1 [11.6 kB] 2021-01-06T22:02:12.772 INFO:teuthology.orchestra.run.smithi203.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 modemmanager amd64 1.10.0-1~ubuntu18.04.2 [728 kB] 2021-01-06T22:02:12.802 INFO:teuthology.orchestra.run.smithi203.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 wpasupplicant amd64 2:2.6-15ubuntu2.6 [955 kB] 2021-01-06T22:02:12.831 INFO:teuthology.orchestra.run.smithi203.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 policykit-1 amd64 0.105-20ubuntu0.18.04.5 [53.5 kB] 2021-01-06T22:02:12.832 INFO:teuthology.orchestra.run.smithi203.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 network-manager amd64 1.10.6-2ubuntu1.4 [1,505 kB] 2021-01-06T22:02:12.871 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package dump. 2021-01-06T22:02:12.874 INFO:teuthology.orchestra.run.smithi203.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 ppp amd64 2.4.7-2+2ubuntu1.3 [328 kB] 2021-01-06T22:02:12.879 INFO:teuthology.orchestra.run.smithi203.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 pptp-linux amd64 1.9.0+ds-2 [41.8 kB] 2021-01-06T22:02:12.880 INFO:teuthology.orchestra.run.smithi203.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 network-manager-pptp amd64 1.2.6-1 [30.0 kB] 2021-01-06T22:02:12.880 INFO:teuthology.orchestra.run.smithi203.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch-data all 20170806-2 [30.7 kB] 2021-01-06T22:02:12.885 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2021-01-06T22:02:12.921 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking dump (0.4b46-3) ... 2021-01-06T22:02:12.927 INFO:teuthology.orchestra.run.smithi203.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu1 [53.6 kB] 2021-01-06T22:02:13.216 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package indent. 2021-01-06T22:02:13.233 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2021-01-06T22:02:13.274 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking indent (2.2.11-5) ... 2021-01-06T22:02:13.413 INFO:teuthology.orchestra.run.smithi203.stdout:Fetched 7,862 kB in 1s (6,547 kB/s) 2021-01-06T22:02:13.544 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package iputils-arping. 2021-01-06T22:02:13.545 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package flex. 2021-01-06T22:02:13.559 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../05-iputils-arping_3%3a20161105-1ubuntu3_amd64.deb ... 2021-01-06T22:02:13.594 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking iputils-arping (3:20161105-1ubuntu3) ... 2021-01-06T22:02:13.600 INFO:teuthology.orchestra.run.smithi203.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 145523 files and directories currently installed.) 2021-01-06T22:02:13.604 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2021-01-06T22:02:13.644 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking flex (2.6.4-6) ... 2021-01-06T22:02:13.821 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libaio-dev:amd64. 2021-01-06T22:02:13.842 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../06-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2021-01-06T22:02:13.871 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2021-01-06T22:02:13.991 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libbison-dev:amd64. 2021-01-06T22:02:14.005 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2021-01-06T22:02:14.040 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2021-01-06T22:02:14.099 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2021-01-06T22:02:14.115 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../07-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2021-01-06T22:02:14.158 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2021-01-06T22:02:14.343 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package bison. 2021-01-06T22:02:14.359 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2021-01-06T22:02:14.398 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2021-01-06T22:02:14.470 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libelf-dev:amd64. 2021-01-06T22:02:14.485 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../08-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2021-01-06T22:02:14.519 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2021-01-06T22:02:14.788 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libfl2:amd64. 2021-01-06T22:02:14.799 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../09-libfl2_2.6.4-6_amd64.deb ... 2021-01-06T22:02:14.813 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package dump. 2021-01-06T22:02:14.827 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2021-01-06T22:02:14.864 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2021-01-06T22:02:14.872 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking dump (0.4b46-3) ... 2021-01-06T22:02:15.084 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libfl-dev:amd64. 2021-01-06T22:02:15.099 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../10-libfl-dev_2.6.4-6_amd64.deb ... 2021-01-06T22:02:15.117 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package indent. 2021-01-06T22:02:15.133 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2021-01-06T22:02:15.141 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2021-01-06T22:02:15.167 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking indent (2.2.11-5) ... 2021-01-06T22:02:15.412 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package iputils-arping. 2021-01-06T22:02:15.427 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../05-iputils-arping_3%3a20161105-1ubuntu3_amd64.deb ... 2021-01-06T22:02:15.462 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking iputils-arping (3:20161105-1ubuntu3) ... 2021-01-06T22:02:15.486 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2021-01-06T22:02:15.501 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../11-libmbim-glib4_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2021-01-06T22:02:15.544 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2021-01-06T22:02:15.715 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libaio-dev:amd64. 2021-01-06T22:02:15.730 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../06-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2021-01-06T22:02:15.773 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2021-01-06T22:02:15.788 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libmbim-proxy. 2021-01-06T22:02:15.806 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../12-libmbim-proxy_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2021-01-06T22:02:15.839 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2021-01-06T22:02:15.993 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2021-01-06T22:02:16.009 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../07-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2021-01-06T22:02:16.043 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2021-01-06T22:02:16.192 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libmm-glib0:amd64. 2021-01-06T22:02:16.208 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../13-libmm-glib0_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2021-01-06T22:02:16.250 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2021-01-06T22:02:16.313 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libelf-dev:amd64. 2021-01-06T22:02:16.329 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../08-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2021-01-06T22:02:16.371 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2021-01-06T22:02:16.595 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libndp0:amd64. 2021-01-06T22:02:16.611 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../14-libndp0_1.6-1_amd64.deb ... 2021-01-06T22:02:16.653 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libndp0:amd64 (1.6-1) ... 2021-01-06T22:02:16.684 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libfl2:amd64. 2021-01-06T22:02:16.700 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../09-libfl2_2.6.4-6_amd64.deb ... 2021-01-06T22:02:16.767 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2021-01-06T22:02:16.931 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libnm0:amd64. 2021-01-06T22:02:16.945 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../15-libnm0_1.10.6-2ubuntu1.4_amd64.deb ... 2021-01-06T22:02:16.981 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2021-01-06T22:02:17.045 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libfl-dev:amd64. 2021-01-06T22:02:17.061 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../10-libfl-dev_2.6.4-6_amd64.deb ... 2021-01-06T22:02:17.112 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2021-01-06T22:02:17.293 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libpolkit-agent-1-0:amd64. 2021-01-06T22:02:17.309 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../16-libpolkit-agent-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2021-01-06T22:02:17.342 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2021-01-06T22:02:17.415 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2021-01-06T22:02:17.430 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../11-libmbim-glib4_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2021-01-06T22:02:17.474 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2021-01-06T22:02:17.670 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libpolkit-backend-1-0:amd64. 2021-01-06T22:02:17.685 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../17-libpolkit-backend-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2021-01-06T22:02:17.718 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libmbim-proxy. 2021-01-06T22:02:17.728 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2021-01-06T22:02:17.735 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../12-libmbim-proxy_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2021-01-06T22:02:17.768 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2021-01-06T22:02:18.048 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2021-01-06T22:02:18.054 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libmm-glib0:amd64. 2021-01-06T22:02:18.062 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../18-libqmi-glib5_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2021-01-06T22:02:18.068 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../13-libmm-glib0_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2021-01-06T22:02:18.098 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2021-01-06T22:02:18.104 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2021-01-06T22:02:18.393 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libqmi-proxy. 2021-01-06T22:02:18.408 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../19-libqmi-proxy_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2021-01-06T22:02:18.442 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2021-01-06T22:02:18.449 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libndp0:amd64. 2021-01-06T22:02:18.465 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../14-libndp0_1.6-1_amd64.deb ... 2021-01-06T22:02:18.516 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libndp0:amd64 (1.6-1) ... 2021-01-06T22:02:18.695 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libssl-dev:amd64. 2021-01-06T22:02:18.709 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../20-libssl-dev_1.1.1-1ubuntu2.1~18.04.7_amd64.deb ... 2021-01-06T22:02:18.754 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.7) ... 2021-01-06T22:02:18.811 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libnm0:amd64. 2021-01-06T22:02:18.826 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../15-libnm0_1.10.6-2ubuntu1.4_amd64.deb ... 2021-01-06T22:02:18.861 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2021-01-06T22:02:19.172 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libpolkit-agent-1-0:amd64. 2021-01-06T22:02:19.186 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../16-libpolkit-agent-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2021-01-06T22:02:19.222 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2021-01-06T22:02:19.233 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package libteamdctl0:amd64. 2021-01-06T22:02:19.250 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../21-libteamdctl0_1.26-1_amd64.deb ... 2021-01-06T22:02:19.291 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking libteamdctl0:amd64 (1.26-1) ... 2021-01-06T22:02:19.542 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libpolkit-backend-1-0:amd64. 2021-01-06T22:02:19.556 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../17-libpolkit-backend-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2021-01-06T22:02:19.601 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2021-01-06T22:02:19.645 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package modemmanager. 2021-01-06T22:02:19.659 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../22-modemmanager_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2021-01-06T22:02:19.704 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking modemmanager (1.10.0-1~ubuntu18.04.2) ... 2021-01-06T22:02:19.912 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2021-01-06T22:02:19.926 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../18-libqmi-glib5_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2021-01-06T22:02:19.963 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2021-01-06T22:02:20.240 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libqmi-proxy. 2021-01-06T22:02:20.256 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../19-libqmi-proxy_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2021-01-06T22:02:20.290 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2021-01-06T22:02:20.313 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package wpasupplicant. 2021-01-06T22:02:20.329 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../23-wpasupplicant_2%3a2.6-15ubuntu2.6_amd64.deb ... 2021-01-06T22:02:20.521 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking wpasupplicant (2:2.6-15ubuntu2.6) ... 2021-01-06T22:02:20.535 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libssl-dev:amd64. 2021-01-06T22:02:20.549 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../20-libssl-dev_1.1.1-1ubuntu2.1~18.04.7_amd64.deb ... 2021-01-06T22:02:20.593 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.7) ... 2021-01-06T22:02:21.039 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package policykit-1. 2021-01-06T22:02:21.071 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libteamdctl0:amd64. 2021-01-06T22:02:21.071 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../24-policykit-1_0.105-20ubuntu0.18.04.5_amd64.deb ... 2021-01-06T22:02:21.081 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../21-libteamdctl0_1.26-1_amd64.deb ... 2021-01-06T22:02:21.109 INFO:teuthology.orchestra.run.smithi091.stdout:Unit polkit.service does not exist, proceeding anyway. 2021-01-06T22:02:21.109 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /run/systemd/system/polkit.service -> /dev/null. 2021-01-06T22:02:21.123 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libteamdctl0:amd64 (1.26-1) ... 2021-01-06T22:02:21.214 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking policykit-1 (0.105-20ubuntu0.18.04.5) ... 2021-01-06T22:02:21.437 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package modemmanager. 2021-01-06T22:02:21.452 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../22-modemmanager_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2021-01-06T22:02:21.496 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking modemmanager (1.10.0-1~ubuntu18.04.2) ... 2021-01-06T22:02:22.088 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package network-manager. 2021-01-06T22:02:22.188 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../25-network-manager_1.10.6-2ubuntu1.4_amd64.deb ... 2021-01-06T22:02:22.188 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking network-manager (1.10.6-2ubuntu1.4) ... 2021-01-06T22:02:22.648 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package wpasupplicant. 2021-01-06T22:02:23.391 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package ppp. 2021-01-06T22:02:23.391 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../26-ppp_2.4.7-2+2ubuntu1.3_amd64.deb ... 2021-01-06T22:02:23.391 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking ppp (2.4.7-2+2ubuntu1.3) ... 2021-01-06T22:02:23.391 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../23-wpasupplicant_2%3a2.6-15ubuntu2.6_amd64.deb ... 2021-01-06T22:02:23.392 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking wpasupplicant (2:2.6-15ubuntu2.6) ... 2021-01-06T22:02:23.392 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package policykit-1. 2021-01-06T22:02:23.393 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../24-policykit-1_0.105-20ubuntu0.18.04.5_amd64.deb ... 2021-01-06T22:02:23.448 INFO:teuthology.orchestra.run.smithi203.stdout:Unit polkit.service does not exist, proceeding anyway. 2021-01-06T22:02:23.448 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /run/systemd/system/polkit.service -> /dev/null. 2021-01-06T22:02:23.474 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package pptp-linux. 2021-01-06T22:02:23.489 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../27-pptp-linux_1.9.0+ds-2_amd64.deb ... 2021-01-06T22:02:23.532 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking pptp-linux (1.9.0+ds-2) ... 2021-01-06T22:02:23.553 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking policykit-1 (0.105-20ubuntu0.18.04.5) ... 2021-01-06T22:02:23.919 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package network-manager-pptp. 2021-01-06T22:02:23.933 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../28-network-manager-pptp_1.2.6-1_amd64.deb ... 2021-01-06T22:02:23.979 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package network-manager. 2021-01-06T22:02:23.987 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking network-manager-pptp (1.2.6-1) ... 2021-01-06T22:02:23.994 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../25-network-manager_1.10.6-2ubuntu1.4_amd64.deb ... 2021-01-06T22:02:24.065 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking network-manager (1.10.6-2ubuntu1.4) ... 2021-01-06T22:02:24.222 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package usb-modeswitch-data. 2021-01-06T22:02:24.238 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../29-usb-modeswitch-data_20170806-2_all.deb ... 2021-01-06T22:02:24.280 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking usb-modeswitch-data (20170806-2) ... 2021-01-06T22:02:24.692 INFO:teuthology.orchestra.run.smithi091.stdout:Selecting previously unselected package usb-modeswitch. 2021-01-06T22:02:24.706 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing to unpack .../30-usb-modeswitch_2.5.2+repack0-2ubuntu1_amd64.deb ... 2021-01-06T22:02:24.719 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ppp. 2021-01-06T22:02:24.733 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../26-ppp_2.4.7-2+2ubuntu1.3_amd64.deb ... 2021-01-06T22:02:24.759 INFO:teuthology.orchestra.run.smithi091.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2021-01-06T22:02:25.054 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2021-01-06T22:02:25.076 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ppp (2.4.7-2+2ubuntu1.3) ... 2021-01-06T22:02:25.172 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up ppp (2.4.7-2+2ubuntu1.3) ... 2021-01-06T22:02:25.390 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package pptp-linux. 2021-01-06T22:02:25.405 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../27-pptp-linux_1.9.0+ds-2_amd64.deb ... 2021-01-06T22:02:25.449 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking pptp-linux (1.9.0+ds-2) ... 2021-01-06T22:02:25.877 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package network-manager-pptp. 2021-01-06T22:02:25.897 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../28-network-manager-pptp_1.2.6-1_amd64.deb ... 2021-01-06T22:02:25.943 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking network-manager-pptp (1.2.6-1) ... 2021-01-06T22:02:26.028 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2021-01-06T22:02:26.188 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up flex (2.6.4-6) ... 2021-01-06T22:02:26.205 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package usb-modeswitch-data. 2021-01-06T22:02:26.222 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../29-usb-modeswitch-data_20170806-2_all.deb ... 2021-01-06T22:02:26.263 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking usb-modeswitch-data (20170806-2) ... 2021-01-06T22:02:26.432 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2021-01-06T22:02:26.566 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up dump (0.4b46-3) ... 2021-01-06T22:02:26.641 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package usb-modeswitch. 2021-01-06T22:02:26.657 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../30-usb-modeswitch_2.5.2+repack0-2ubuntu1_amd64.deb ... 2021-01-06T22:02:26.672 INFO:teuthology.orchestra.run.smithi091.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2021-01-06T22:02:26.708 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2021-01-06T22:02:26.743 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2021-01-06T22:02:26.869 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.7) ... 2021-01-06T22:02:26.997 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2021-01-06T22:02:27.003 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up pptp-linux (1.9.0+ds-2) ... 2021-01-06T22:02:27.171 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up wpasupplicant (2:2.6-15ubuntu2.6) ... 2021-01-06T22:02:27.222 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ppp (2.4.7-2+2ubuntu1.3) ... 2021-01-06T22:02:27.543 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2021-01-06T22:02:27.543 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2021-01-06T22:02:27.849 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2021-01-06T22:02:27.975 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up usb-modeswitch-data (20170806-2) ... 2021-01-06T22:02:28.079 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2021-01-06T22:02:28.118 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2021-01-06T22:02:28.212 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up flex (2.6.4-6) ... 2021-01-06T22:02:28.294 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2021-01-06T22:02:28.363 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2021-01-06T22:02:28.420 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2021-01-06T22:02:28.489 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up dump (0.4b46-3) ... 2021-01-06T22:02:28.546 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up indent (2.2.11-5) ... 2021-01-06T22:02:28.594 INFO:teuthology.orchestra.run.smithi203.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2021-01-06T22:02:28.666 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2021-01-06T22:02:28.672 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2021-01-06T22:02:28.793 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.7) ... 2021-01-06T22:02:28.801 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2021-01-06T22:02:28.918 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up pptp-linux (1.9.0+ds-2) ... 2021-01-06T22:02:28.927 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2021-01-06T22:02:29.026 INFO:teuthology.orchestra.run.smithi091.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2021-01-06T22:02:29.086 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up wpasupplicant (2:2.6-15ubuntu2.6) ... 2021-01-06T22:02:29.096 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2021-01-06T22:02:29.222 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libndp0:amd64 (1.6-1) ... 2021-01-06T22:02:29.348 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up iputils-arping (3:20161105-1ubuntu3) ... 2021-01-06T22:02:29.475 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libteamdctl0:amd64 (1.26-1) ... 2021-01-06T22:02:29.483 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2021-01-06T22:02:29.483 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2021-01-06T22:02:29.600 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2021-01-06T22:02:29.732 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up policykit-1 (0.105-20ubuntu0.18.04.5) ... 2021-01-06T22:02:29.781 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2021-01-06T22:02:29.907 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up usb-modeswitch-data (20170806-2) ... 2021-01-06T22:02:30.041 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2021-01-06T22:02:30.087 INFO:teuthology.orchestra.run.smithi091.stdout:Removed /run/systemd/system/polkit.service. 2021-01-06T22:02:30.218 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2021-01-06T22:02:30.307 INFO:teuthology.orchestra.run.smithi091.stdout:polkit.service is a disabled or a static unit not running, not starting it. 2021-01-06T22:02:30.345 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2021-01-06T22:02:30.453 INFO:teuthology.orchestra.run.smithi091.stdout:polkit.service is a disabled or a static unit, not starting it. 2021-01-06T22:02:30.478 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up indent (2.2.11-5) ... 2021-01-06T22:02:30.495 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2021-01-06T22:02:30.604 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2021-01-06T22:02:30.638 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2021-01-06T22:02:30.739 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2021-01-06T22:02:30.764 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2021-01-06T22:02:30.864 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2021-01-06T22:02:30.890 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2021-01-06T22:02:30.954 INFO:teuthology.orchestra.run.smithi203.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2021-01-06T22:02:31.016 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up network-manager (1.10.6-2ubuntu1.4) ... 2021-01-06T22:02:31.024 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2021-01-06T22:02:31.176 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libndp0:amd64 (1.6-1) ... 2021-01-06T22:02:31.304 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up iputils-arping (3:20161105-1ubuntu3) ... 2021-01-06T22:02:31.430 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libteamdctl0:amd64 (1.26-1) ... 2021-01-06T22:02:31.557 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2021-01-06T22:02:31.662 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:31.662 INFO:teuthology.orchestra.run.smithi091.stdout:The following network interfaces were found in /etc/network/interfaces 2021-01-06T22:02:31.662 INFO:teuthology.orchestra.run.smithi091.stdout:which means they are currently configured by ifupdown: 2021-01-06T22:02:31.663 INFO:teuthology.orchestra.run.smithi091.stdout:- enp3s0f1 2021-01-06T22:02:31.663 INFO:teuthology.orchestra.run.smithi091.stdout:If you want to manage those interfaces with NetworkManager instead 2021-01-06T22:02:31.663 INFO:teuthology.orchestra.run.smithi091.stdout:remove their configuration from /etc/network/interfaces. 2021-01-06T22:02:31.663 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:31.682 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up policykit-1 (0.105-20ubuntu0.18.04.5) ... 2021-01-06T22:02:31.798 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2021-01-06T22:02:32.018 INFO:teuthology.orchestra.run.smithi203.stdout:Removed /run/systemd/system/polkit.service. 2021-01-06T22:02:32.039 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2021-01-06T22:02:32.247 INFO:teuthology.orchestra.run.smithi203.stdout:polkit.service is a disabled or a static unit not running, not starting it. 2021-01-06T22:02:32.282 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2021-01-06T22:02:32.387 INFO:teuthology.orchestra.run.smithi203.stdout:polkit.service is a disabled or a static unit, not starting it. 2021-01-06T22:02:32.471 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2021-01-06T22:02:32.597 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2021-01-06T22:02:32.723 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2021-01-06T22:02:32.838 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up network-manager-pptp (1.2.6-1) ... 2021-01-06T22:02:32.849 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2021-01-06T22:02:32.950 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up network-manager (1.10.6-2ubuntu1.4) ... 2021-01-06T22:02:33.014 INFO:teuthology.orchestra.run.smithi091.stdout:Setting up modemmanager (1.10.0-1~ubuntu18.04.2) ... 2021-01-06T22:02:33.274 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2021-01-06T22:02:33.274 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2021-01-06T22:02:33.502 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:33.503 INFO:teuthology.orchestra.run.smithi203.stdout:The following network interfaces were found in /etc/network/interfaces 2021-01-06T22:02:33.503 INFO:teuthology.orchestra.run.smithi203.stdout:which means they are currently configured by ifupdown: 2021-01-06T22:02:33.503 INFO:teuthology.orchestra.run.smithi203.stdout:- enp3s0f1 2021-01-06T22:02:33.503 INFO:teuthology.orchestra.run.smithi203.stdout:If you want to manage those interfaces with NetworkManager instead 2021-01-06T22:02:33.504 INFO:teuthology.orchestra.run.smithi203.stdout:remove their configuration from /etc/network/interfaces. 2021-01-06T22:02:33.504 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:33.629 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2021-01-06T22:02:33.642 INFO:teuthology.orchestra.run.smithi091.stdout:Processing triggers for systemd (237-3ubuntu10.43) ... 2021-01-06T22:02:33.851 INFO:teuthology.orchestra.run.smithi091.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-01-06T22:02:33.858 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2021-01-06T22:02:34.099 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2021-01-06T22:02:34.629 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up network-manager-pptp (1.2.6-1) ... 2021-01-06T22:02:34.780 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up modemmanager (1.10.0-1~ubuntu18.04.2) ... 2021-01-06T22:02:35.037 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2021-01-06T22:02:35.038 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2021-01-06T22:02:35.433 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for systemd (237-3ubuntu10.43) ... 2021-01-06T22:02:35.609 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-01-06T22:02:36.345 INFO:teuthology.orchestra.run.smithi091.stdout:Processing triggers for dbus (1.12.2-1ubuntu1.2) ... 2021-01-06T22:02:36.437 INFO:teuthology.orchestra.run.smithi091.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2021-01-06T22:02:36.822 INFO:teuthology.orchestra.run.smithi091.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-01-06T22:02:36.931 INFO:teuthology.orchestra.run.smithi091.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2021-01-06T22:02:37.309 INFO:teuthology.orchestra.run.smithi091.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.4) ... 2021-01-06T22:02:37.757 INFO:teuthology.orchestra.run.smithi091.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-01-06T22:02:37.759 DEBUG:teuthology.parallel:result is None 2021-01-06T22:02:38.019 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for dbus (1.12.2-1ubuntu1.2) ... 2021-01-06T22:02:38.103 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2021-01-06T22:02:38.471 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-01-06T22:02:38.580 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2021-01-06T22:02:38.957 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.4) ... 2021-01-06T22:02:39.437 INFO:teuthology.orchestra.run.smithi203.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-01-06T22:02:39.439 DEBUG:teuthology.parallel:result is None 2021-01-06T22:02:39.440 DEBUG:teuthology.orchestra.run.smithi203:> sudo lsb_release -is 2021-01-06T22:02:39.493 INFO:teuthology.orchestra.run.smithi203.stdout:Ubuntu 2021-01-06T22:02:39.497 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-01-06T22:02:39.498 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=d0ed162b51928c50f20cee111f8292828eda755e 2021-01-06T22:02:39.665 DEBUG:teuthology.orchestra.run.smithi203:> dpkg-query -W -f '${Version}' ceph 2021-01-06T22:02:39.690 INFO:teuthology.orchestra.run.smithi203.stdout:16.0.0-8719-gd0ed162b-1bionic 2021-01-06T22:02:39.691 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-8719-gd0ed162b-1bionic 2021-01-06T22:02:39.691 INFO:teuthology.task.install:The correct ceph version 16.0.0-8719-gd0ed162b-1bionic is installed. 2021-01-06T22:02:39.692 DEBUG:teuthology.orchestra.run.smithi091:> sudo lsb_release -is 2021-01-06T22:02:39.746 INFO:teuthology.orchestra.run.smithi091.stdout:Ubuntu 2021-01-06T22:02:39.751 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-01-06T22:02:39.752 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=d0ed162b51928c50f20cee111f8292828eda755e 2021-01-06T22:02:39.913 DEBUG:teuthology.orchestra.run.smithi091:> dpkg-query -W -f '${Version}' ceph 2021-01-06T22:02:39.938 INFO:teuthology.orchestra.run.smithi091.stdout:16.0.0-8719-gd0ed162b-1bionic 2021-01-06T22:02:39.938 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-8719-gd0ed162b-1bionic 2021-01-06T22:02:39.939 INFO:teuthology.task.install:The correct ceph version 16.0.0-8719-gd0ed162b-1bionic is installed. 2021-01-06T22:02:39.939 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2021-01-06T22:02:39.940 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:02:39.940 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-01-06T22:02:39.955 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:02:39.955 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-01-06T22:02:39.995 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2021-01-06T22:02:39.996 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:02:39.996 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2021-01-06T22:02:40.011 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-01-06T22:02:40.065 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:02:40.066 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/usr/bin/daemon-helper 2021-01-06T22:02:40.082 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-01-06T22:02:40.137 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2021-01-06T22:02:40.137 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:02:40.138 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2021-01-06T22:02:40.153 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-01-06T22:02:40.209 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:02:40.210 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/usr/bin/adjust-ulimits 2021-01-06T22:02:40.225 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-01-06T22:02:40.281 INFO:teuthology.run_tasks:Running task ceph... 2021-01-06T22:02:40.346 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2021-01-06T22:02:40.347 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 777 /var/log/ceph 2021-01-06T22:02:40.349 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /var/log/ceph 2021-01-06T22:02:40.364 INFO:tasks.ceph:Disabling ceph logrotate... 2021-01-06T22:02:40.364 DEBUG:teuthology.orchestra.run.smithi091:> sudo rm -f -- /etc/logrotate.d/ceph 2021-01-06T22:02:40.406 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/logrotate.d/ceph 2021-01-06T22:02:40.422 INFO:tasks.ceph:Creating extra log directories... 2021-01-06T22:02:40.423 DEBUG:teuthology.orchestra.run.smithi091:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-01-06T22:02:40.465 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-01-06T22:02:40.542 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2021-01-06T22:02:40.543 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2021-01-06T22:02:40.543 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2021-01-06T22:02:40.543 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:02:40.544 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-01-06T22:02:40.550 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-01-06T22:02:40.629 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:02:40.630 DEBUG:teuthology.orchestra.run.smithi091:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-01-06T22:02:40.636 DEBUG:teuthology.orchestra.run.smithi091:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-01-06T22:02:40.721 INFO:tasks.ceph:Creating ceph cluster ceph... 2021-01-06T22:02:40.723 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'global': {'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd mds', 'ms inject socket failures': 2500}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'SLOW_OPS', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2021-01-06T22:02:40.723 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2021-01-06_20:20:52-fs-wip-pdonnell-testing-20210105.221014-distro-basic-smithi/5760088', 'branch': 'wip-pdonnell-testing-20210105.221014', 'description': 'fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '5760088', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2021-01-06_20:20:52-fs-wip-pdonnell-testing-20210105.221014-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '18.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20210105.221014'}, 'ceph': {'cephfs': {'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'global': {'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd mds', 'ms inject socket failures': 2500}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'SLOW_OPS', 'slow request'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'd0ed162b51928c50f20cee111f8292828eda755e'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'sha1': 'd0ed162b51928c50f20cee111f8292828eda755e'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20210105.221014', 'sha1': 'd0ed162b51928c50f20cee111f8292828eda755e'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 69, 'repo': 'git://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'sha1': 'd0ed162b51928c50f20cee111f8292828eda755e', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20210105.221014', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20210105.221014/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': 'd0ed162b51928c50f20cee111f8292828eda755e', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': 'd0ed162b51928c50f20cee111f8292828eda755e'}}, {'ceph': None}, {'ceph-fuse': None}, {'mds_thrash': None}, {'workunit': {'clients': {'all': ['suites/ffsb.sh']}}}], 'teuthology_branch': 'master', 'timestamp': '2021-01-06_20:20:52', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.15477', 'targets': {'smithi203.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYX2aW5kSZE4K8H3f9mq0QhiOgFdcVZIPZTH/OdlVTmb/0CxzDjZk+mT1Cv8kBWgASGhIWzXz9DjE3qA5yVw06s3OdhwHfr6mCAIYHaT3suYc6wDjhn2n+t7vH6fzriQVnSrtlb8V42iRkLCRXk9nU2CwsoweRATMDsyo9uBCJ5Lhzi8r39asEJwib6EkVf29udO+FBNns0Xse5oesTZdSA+8miMv9oQ1/g8fF+hZciz8csOFvQlDbK1nx8Z0vpqR/PiRsUEglXBL4QWDNTiqRBnjSeMYMBWfo3QPvR1Bzjkx8n8f4ckNOYh6itU+jVhFOUD6+wm0gKxE5JiunGK4V', 'smithi091.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxD2AnNqm9fQ9ntMAHZi1hz7TkCqFG+G2H17VCIENzH2xNWbd+PIj9VLD01/nlr0QeUu7U77/ROiQkPQZjIt4QbBtEb8Y989DzZz5Lo4smSz4IDy+ud54pvq39hQPkDUaTGe834I87W8FVlOzpdEaNKWy0M6vP/UZXeI+2C8OKd+E05ocr7ycUJK094JTAhiL50/7okOhQtUz1Dha+/apxYfKleVBb6A6j1qc+gO61IeVIRuWMZJ809vOmgaC/puV86opCpqmzV680aQC3KAMhuz8M7EsKpKlA7eqlTRZCVRbFpkgbfsI4D1ml9jjIS+7SehwL4NcKDuDAv/ogDm0f'}} 2021-01-06T22:02:40.724 DEBUG:teuthology.orchestra.run.smithi091:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-01-06T22:02:40.761 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-01-06T22:02:40.770 DEBUG:teuthology.orchestra.run.smithi091:> sudo install -d -m0777 -- /var/run/ceph 2021-01-06T22:02:40.809 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m0777 -- /var/run/ceph 2021-01-06T22:02:40.827 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:02:40.828 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2021-01-06T22:02:40.875 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-01-06T22:02:40.876 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-01-06T22:02:40.922 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-01-06T22:02:40.923 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-01-06T22:02:40.923 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 342 Links: 1 2021-01-06T22:02:40.923 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-06T22:02:40.923 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2021-01-06 21:59:28.686637015 +0000 2021-01-06T22:02:40.923 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2021-01-06 21:59:28.610637017 +0000 2021-01-06T22:02:40.924 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2021-01-06 21:59:28.610637017 +0000 2021-01-06T22:02:40.924 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2021-01-06T22:02:40.932 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2021-01-06T22:02:40.932 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2021-01-06T22:02:40.932 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000363829 s, 1.4 MB/s 2021-01-06T22:02:40.937 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-01-06T22:02:40.986 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-01-06T22:02:40.987 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-01-06T22:02:40.987 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 350 Links: 1 2021-01-06T22:02:40.987 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-06T22:02:40.987 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2021-01-06 21:59:28.934637010 +0000 2021-01-06T22:02:40.987 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2021-01-06 21:59:28.830637012 +0000 2021-01-06T22:02:40.988 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2021-01-06 21:59:28.830637012 +0000 2021-01-06T22:02:40.988 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2021-01-06T22:02:40.995 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2021-01-06T22:02:40.995 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2021-01-06T22:02:40.995 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.00036193 s, 1.4 MB/s 2021-01-06T22:02:40.999 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-01-06T22:02:41.050 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-01-06T22:02:41.051 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-01-06T22:02:41.051 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 356 Links: 1 2021-01-06T22:02:41.051 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-06T22:02:41.052 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2021-01-06 21:59:29.166637005 +0000 2021-01-06T22:02:41.052 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2021-01-06 21:59:29.054637007 +0000 2021-01-06T22:02:41.052 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2021-01-06 21:59:29.054637007 +0000 2021-01-06T22:02:41.052 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2021-01-06T22:02:41.060 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2021-01-06T22:02:41.061 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2021-01-06T22:02:41.061 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.00039089 s, 1.3 MB/s 2021-01-06T22:02:41.065 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-01-06T22:02:41.114 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-01-06T22:02:41.114 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-01-06T22:02:41.115 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 362 Links: 1 2021-01-06T22:02:41.116 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-06T22:02:41.116 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2021-01-06 21:59:29.402636999 +0000 2021-01-06T22:02:41.116 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2021-01-06 21:59:29.326637001 +0000 2021-01-06T22:02:41.116 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2021-01-06 21:59:29.326637001 +0000 2021-01-06T22:02:41.116 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2021-01-06T22:02:41.124 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2021-01-06T22:02:41.124 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2021-01-06T22:02:41.124 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000445024 s, 1.2 MB/s 2021-01-06T22:02:41.128 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-01-06T22:02:41.129 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:02:41.129 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/scratch_devs of=/dev/stdout 2021-01-06T22:02:41.135 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-01-06T22:02:41.136 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-01-06T22:02:41.186 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-01-06T22:02:41.187 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-01-06T22:02:41.187 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 6h/6d Inode: 343 Links: 1 2021-01-06T22:02:41.187 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-06T22:02:41.187 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2021-01-06 21:59:28.645590931 +0000 2021-01-06T22:02:41.188 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2021-01-06 21:59:28.553590933 +0000 2021-01-06T22:02:41.188 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2021-01-06 21:59:28.553590933 +0000 2021-01-06T22:02:41.188 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: - 2021-01-06T22:02:41.196 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2021-01-06T22:02:41.196 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2021-01-06T22:02:41.197 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000357812 s, 1.4 MB/s 2021-01-06T22:02:41.201 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-01-06T22:02:41.249 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-01-06T22:02:41.250 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-01-06T22:02:41.250 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 6h/6d Inode: 350 Links: 1 2021-01-06T22:02:41.250 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-06T22:02:41.250 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2021-01-06 21:59:28.889590925 +0000 2021-01-06T22:02:41.250 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2021-01-06 21:59:28.801590927 +0000 2021-01-06T22:02:41.251 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2021-01-06 21:59:28.801590927 +0000 2021-01-06T22:02:41.251 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: - 2021-01-06T22:02:41.259 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2021-01-06T22:02:41.259 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2021-01-06T22:02:41.259 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000370524 s, 1.4 MB/s 2021-01-06T22:02:41.264 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-01-06T22:02:41.314 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-01-06T22:02:41.315 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-01-06T22:02:41.315 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 6h/6d Inode: 356 Links: 1 2021-01-06T22:02:41.316 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-06T22:02:41.316 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2021-01-06 21:59:29.133590920 +0000 2021-01-06T22:02:41.316 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2021-01-06 21:59:29.029590922 +0000 2021-01-06T22:02:41.317 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2021-01-06 21:59:29.029590922 +0000 2021-01-06T22:02:41.317 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: - 2021-01-06T22:02:41.323 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2021-01-06T22:02:41.324 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2021-01-06T22:02:41.324 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000341367 s, 1.5 MB/s 2021-01-06T22:02:41.328 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-01-06T22:02:41.377 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-01-06T22:02:41.378 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-01-06T22:02:41.378 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 6h/6d Inode: 362 Links: 1 2021-01-06T22:02:41.378 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-06T22:02:41.378 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2021-01-06 21:59:29.397590914 +0000 2021-01-06T22:02:41.379 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2021-01-06 21:59:29.277590917 +0000 2021-01-06T22:02:41.379 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2021-01-06 21:59:29.277590917 +0000 2021-01-06T22:02:41.380 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: - 2021-01-06T22:02:41.387 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2021-01-06T22:02:41.387 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2021-01-06T22:02:41.388 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000462026 s, 1.1 MB/s 2021-01-06T22:02:41.392 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-01-06T22:02:41.393 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi203.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi091.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2021-01-06T22:02:41.393 INFO:tasks.ceph:Generating config... 2021-01-06T22:02:41.396 INFO:tasks.ceph:[client] client mount timeout = 600 2021-01-06T22:02:41.397 INFO:tasks.ceph:[client] debug client = 20 2021-01-06T22:02:41.397 INFO:tasks.ceph:[client] debug ms = 1 2021-01-06T22:02:41.398 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2021-01-06T22:02:41.398 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2021-01-06T22:02:41.399 INFO:tasks.ceph:[global] mon client directed command retry = 5 2021-01-06T22:02:41.399 INFO:tasks.ceph:[global] ms inject delay max = 1 2021-01-06T22:02:41.399 INFO:tasks.ceph:[global] ms inject delay probability = 0.005 2021-01-06T22:02:41.399 INFO:tasks.ceph:[global] ms inject delay type = osd mds 2021-01-06T22:02:41.400 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2021-01-06T22:02:41.400 INFO:tasks.ceph:[mds] debug mds = 20 2021-01-06T22:02:41.400 INFO:tasks.ceph:[mds] debug ms = 1 2021-01-06T22:02:41.401 INFO:tasks.ceph:[mds] mds bal frag = True 2021-01-06T22:02:41.401 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2021-01-06T22:02:41.401 INFO:tasks.ceph:[mds] mds bal merge size = 5 2021-01-06T22:02:41.402 INFO:tasks.ceph:[mds] mds bal split bits = 3 2021-01-06T22:02:41.402 INFO:tasks.ceph:[mds] mds bal split size = 100 2021-01-06T22:02:41.402 INFO:tasks.ceph:[mds] mds debug frag = True 2021-01-06T22:02:41.403 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2021-01-06T22:02:41.403 INFO:tasks.ceph:[mds] mds op complaint time = 180 2021-01-06T22:02:41.403 INFO:tasks.ceph:[mds] mds verify scatter = True 2021-01-06T22:02:41.403 INFO:tasks.ceph:[mds] osd op complaint time = 180 2021-01-06T22:02:41.404 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2021-01-06T22:02:41.404 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2021-01-06T22:02:41.404 INFO:tasks.ceph:[mgr] debug mgr = 20 2021-01-06T22:02:41.405 INFO:tasks.ceph:[mgr] debug ms = 1 2021-01-06T22:02:41.405 INFO:tasks.ceph:[mon] debug mon = 20 2021-01-06T22:02:41.405 INFO:tasks.ceph:[mon] debug ms = 1 2021-01-06T22:02:41.405 INFO:tasks.ceph:[mon] debug paxos = 20 2021-01-06T22:02:41.406 INFO:tasks.ceph:[mon] mon op complaint time = 120 2021-01-06T22:02:41.406 INFO:tasks.ceph:[osd] bdev async discard = True 2021-01-06T22:02:41.406 INFO:tasks.ceph:[osd] bdev enable discard = True 2021-01-06T22:02:41.407 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2021-01-06T22:02:41.407 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2021-01-06T22:02:41.408 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2021-01-06T22:02:41.409 INFO:tasks.ceph:[osd] debug bluefs = 20 2021-01-06T22:02:41.409 INFO:tasks.ceph:[osd] debug bluestore = 20 2021-01-06T22:02:41.410 INFO:tasks.ceph:[osd] debug filestore = 20 2021-01-06T22:02:41.410 INFO:tasks.ceph:[osd] debug journal = 20 2021-01-06T22:02:41.410 INFO:tasks.ceph:[osd] debug ms = 20 2021-01-06T22:02:41.411 INFO:tasks.ceph:[osd] debug osd = 25 2021-01-06T22:02:41.411 INFO:tasks.ceph:[osd] debug rocksdb = 10 2021-01-06T22:02:41.412 INFO:tasks.ceph:[osd] filestore flush min = 0 2021-01-06T22:02:41.412 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2021-01-06T22:02:41.413 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2021-01-06T22:02:41.414 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2021-01-06T22:02:41.414 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2021-01-06T22:02:41.414 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2021-01-06T22:02:41.414 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2021-01-06T22:02:41.415 INFO:tasks.ceph:[osd] osd op complaint time = 180 2021-01-06T22:02:41.415 INFO:tasks.ceph:Setting up mon.a... 2021-01-06T22:02:41.416 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2021-01-06T22:02:41.448 INFO:teuthology.orchestra.run.smithi203.stdout:creating /etc/ceph/ceph.keyring 2021-01-06T22:02:41.450 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2021-01-06T22:02:41.524 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-01-06T22:02:41.584 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.203'), ('mon.b', '172.21.15.91'), ('mon.c', '[v2:172.21.15.91:3301,v1:172.21.15.91:6790]')] 2021-01-06T22:02:41.585 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.203,172.21.15.91,[v2:172.21.15.91:3301,v1:172.21.15.91:6790]', 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd mds', 'ms inject socket failures': 2500}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}} 2021-01-06T22:02:41.586 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:02:41.587 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2021-01-06T22:02:41.635 DEBUG:teuthology.orchestra.run.smithi203:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.203 --add b 172.21.15.91 --addv c '[v2:172.21.15.91:3301,v1:172.21.15.91:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2021-01-06T22:02:41.713 INFO:teuthology.orchestra.run.smithi203.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2021-01-06T22:02:41.713 INFO:teuthology.orchestra.run.smithi203.stdout:monmaptool: generated fsid f96dbde2-6d40-40dc-b34a-9fb254ce805f 2021-01-06T22:02:41.714 INFO:teuthology.orchestra.run.smithi203.stdout:epoch 0 2021-01-06T22:02:41.714 INFO:teuthology.orchestra.run.smithi203.stdout:fsid f96dbde2-6d40-40dc-b34a-9fb254ce805f 2021-01-06T22:02:41.714 INFO:teuthology.orchestra.run.smithi203.stdout:last_changed 2021-01-06T22:02:41.717515+0000 2021-01-06T22:02:41.715 INFO:teuthology.orchestra.run.smithi203.stdout:created 2021-01-06T22:02:41.717515+0000 2021-01-06T22:02:41.715 INFO:teuthology.orchestra.run.smithi203.stdout:min_mon_release 0 (unknown) 2021-01-06T22:02:41.715 INFO:teuthology.orchestra.run.smithi203.stdout:election_strategy: 1 2021-01-06T22:02:41.715 INFO:teuthology.orchestra.run.smithi203.stdout:0: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.a 2021-01-06T22:02:41.716 INFO:teuthology.orchestra.run.smithi203.stdout:1: [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] mon.b 2021-01-06T22:02:41.716 INFO:teuthology.orchestra.run.smithi203.stdout:2: [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] mon.c 2021-01-06T22:02:41.716 INFO:teuthology.orchestra.run.smithi203.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2021-01-06T22:02:41.721 DEBUG:teuthology.orchestra.run.smithi203:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2021-01-06T22:02:41.769 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f96dbde2-6d40-40dc-b34a-9fb254ce805f... 2021-01-06T22:02:41.771 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-01-06T22:02:41.777 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-01-06T22:02:41.814 INFO:teuthology.orchestra.run.smithi091.stdout:[global] 2021-01-06T22:02:41.815 INFO:teuthology.orchestra.run.smithi091.stdout: chdir = "" 2021-01-06T22:02:41.815 INFO:teuthology.orchestra.run.smithi091.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-01-06T22:02:41.815 INFO:teuthology.orchestra.run.smithi091.stdout: auth supported = cephx 2021-01-06T22:02:41.815 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.816 INFO:teuthology.orchestra.run.smithi091.stdout: filestore xattr use omap = true 2021-01-06T22:02:41.816 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.816 INFO:teuthology.orchestra.run.smithi091.stdout: mon clock drift allowed = 1.000 2021-01-06T22:02:41.816 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.816 INFO:teuthology.orchestra.run.smithi091.stdout: osd crush chooseleaf type = 0 2021-01-06T22:02:41.817 INFO:teuthology.orchestra.run.smithi091.stdout: auth debug = true 2021-01-06T22:02:41.817 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.817 INFO:teuthology.orchestra.run.smithi091.stdout: ms die on old message = true 2021-01-06T22:02:41.817 INFO:teuthology.orchestra.run.smithi091.stdout: ms die on bug = true 2021-01-06T22:02:41.817 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.818 INFO:teuthology.orchestra.run.smithi091.stdout: mon max pg per osd = 10000 # >= luminous 2021-01-06T22:02:41.818 INFO:teuthology.orchestra.run.smithi091.stdout: mon pg warn max object skew = 0 2021-01-06T22:02:41.818 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.818 INFO:teuthology.orchestra.run.smithi091.stdout: # disable pg_autoscaler by default for new pools 2021-01-06T22:02:41.819 INFO:teuthology.orchestra.run.smithi091.stdout: osd_pool_default_pg_autoscale_mode = off 2021-01-06T22:02:41.819 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.819 INFO:teuthology.orchestra.run.smithi091.stdout: osd pool default size = 2 2021-01-06T22:02:41.819 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.819 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd allow primary affinity = true 2021-01-06T22:02:41.820 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd allow pg remap = true 2021-01-06T22:02:41.820 INFO:teuthology.orchestra.run.smithi091.stdout: mon warn on legacy crush tunables = false 2021-01-06T22:02:41.820 INFO:teuthology.orchestra.run.smithi091.stdout: mon warn on crush straw calc version zero = false 2021-01-06T22:02:41.820 INFO:teuthology.orchestra.run.smithi091.stdout: mon warn on no sortbitwise = false 2021-01-06T22:02:41.820 INFO:teuthology.orchestra.run.smithi091.stdout: mon warn on osd down out interval zero = false 2021-01-06T22:02:41.821 INFO:teuthology.orchestra.run.smithi091.stdout: mon warn on too few osds = false 2021-01-06T22:02:41.821 INFO:teuthology.orchestra.run.smithi091.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-01-06T22:02:41.821 INFO:teuthology.orchestra.run.smithi091.stdout: mon_warn_on_pool_no_redundancy = false 2021-01-06T22:02:41.822 INFO:teuthology.orchestra.run.smithi091.stdout: mon_allow_pool_size_one = true 2021-01-06T22:02:41.822 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.822 INFO:teuthology.orchestra.run.smithi091.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-01-06T22:02:41.822 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.822 INFO:teuthology.orchestra.run.smithi091.stdout: osd default data pool replay window = 5 2021-01-06T22:02:41.823 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.823 INFO:teuthology.orchestra.run.smithi091.stdout: mon allow pool delete = true 2021-01-06T22:02:41.823 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.823 INFO:teuthology.orchestra.run.smithi091.stdout: mon cluster log file level = debug 2021-01-06T22:02:41.824 INFO:teuthology.orchestra.run.smithi091.stdout: debug asserts on shutdown = true 2021-01-06T22:02:41.824 INFO:teuthology.orchestra.run.smithi091.stdout: mon health detail to clog = false 2021-01-06T22:02:41.824 INFO:teuthology.orchestra.run.smithi091.stdout: mon host = "172.21.15.203,172.21.15.91,[v2:172.21.15.91:3301,v1:172.21.15.91:6790]" 2021-01-06T22:02:41.824 INFO:teuthology.orchestra.run.smithi091.stdout: mon client directed command retry = 5 2021-01-06T22:02:41.824 INFO:teuthology.orchestra.run.smithi091.stdout: ms inject delay max = 1 2021-01-06T22:02:41.825 INFO:teuthology.orchestra.run.smithi091.stdout: ms inject delay probability = 0.005 2021-01-06T22:02:41.825 INFO:teuthology.orchestra.run.smithi091.stdout: ms inject delay type = osd mds 2021-01-06T22:02:41.825 INFO:teuthology.orchestra.run.smithi091.stdout: ms inject socket failures = 2500 2021-01-06T22:02:41.825 INFO:teuthology.orchestra.run.smithi091.stdout: fsid = f96dbde2-6d40-40dc-b34a-9fb254ce805f 2021-01-06T22:02:41.825 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.826 INFO:teuthology.orchestra.run.smithi091.stdout:[osd] 2021-01-06T22:02:41.826 INFO:teuthology.orchestra.run.smithi091.stdout: osd journal size = 100 2021-01-06T22:02:41.826 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.826 INFO:teuthology.orchestra.run.smithi091.stdout: osd scrub load threshold = 5.0 2021-01-06T22:02:41.827 INFO:teuthology.orchestra.run.smithi091.stdout: osd scrub max interval = 600 2021-01-06T22:02:41.827 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.827 INFO:teuthology.orchestra.run.smithi091.stdout: osd recover clone overlap = true 2021-01-06T22:02:41.827 INFO:teuthology.orchestra.run.smithi091.stdout: osd recovery max chunk = 1048576 2021-01-06T22:02:41.827 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.828 INFO:teuthology.orchestra.run.smithi091.stdout: osd debug shutdown = true 2021-01-06T22:02:41.828 INFO:teuthology.orchestra.run.smithi091.stdout: osd debug op order = true 2021-01-06T22:02:41.828 INFO:teuthology.orchestra.run.smithi091.stdout: osd debug verify stray on activate = true 2021-01-06T22:02:41.828 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.829 INFO:teuthology.orchestra.run.smithi091.stdout: osd open classes on start = true 2021-01-06T22:02:41.829 INFO:teuthology.orchestra.run.smithi091.stdout: osd debug pg log writeout = true 2021-01-06T22:02:41.829 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.829 INFO:teuthology.orchestra.run.smithi091.stdout: osd deep scrub update digest min age = 30 2021-01-06T22:02:41.830 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.830 INFO:teuthology.orchestra.run.smithi091.stdout: osd map max advance = 10 2021-01-06T22:02:41.830 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.830 INFO:teuthology.orchestra.run.smithi091.stdout: journal zero on create = true 2021-01-06T22:02:41.830 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.831 INFO:teuthology.orchestra.run.smithi091.stdout: filestore ondisk finisher threads = 3 2021-01-06T22:02:41.831 INFO:teuthology.orchestra.run.smithi091.stdout: filestore apply finisher threads = 3 2021-01-06T22:02:41.831 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.831 INFO:teuthology.orchestra.run.smithi091.stdout: bdev debug aio = true 2021-01-06T22:02:41.831 INFO:teuthology.orchestra.run.smithi091.stdout: osd debug misdirected ops = true 2021-01-06T22:02:41.832 INFO:teuthology.orchestra.run.smithi091.stdout: bdev async discard = True 2021-01-06T22:02:41.832 INFO:teuthology.orchestra.run.smithi091.stdout: bdev enable discard = True 2021-01-06T22:02:41.832 INFO:teuthology.orchestra.run.smithi091.stdout: bluestore allocator = bitmap 2021-01-06T22:02:41.832 INFO:teuthology.orchestra.run.smithi091.stdout: bluestore block size = 96636764160 2021-01-06T22:02:41.832 INFO:teuthology.orchestra.run.smithi091.stdout: bluestore fsck on mount = True 2021-01-06T22:02:41.833 INFO:teuthology.orchestra.run.smithi091.stdout: debug bluefs = 20 2021-01-06T22:02:41.833 INFO:teuthology.orchestra.run.smithi091.stdout: debug bluestore = 20 2021-01-06T22:02:41.833 INFO:teuthology.orchestra.run.smithi091.stdout: debug filestore = 20 2021-01-06T22:02:41.833 INFO:teuthology.orchestra.run.smithi091.stdout: debug journal = 20 2021-01-06T22:02:41.834 INFO:teuthology.orchestra.run.smithi091.stdout: debug ms = 20 2021-01-06T22:02:41.834 INFO:teuthology.orchestra.run.smithi091.stdout: debug osd = 25 2021-01-06T22:02:41.834 INFO:teuthology.orchestra.run.smithi091.stdout: debug rocksdb = 10 2021-01-06T22:02:41.834 INFO:teuthology.orchestra.run.smithi091.stdout: filestore flush min = 0 2021-01-06T22:02:41.834 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd backfillfull_ratio = 0.85 2021-01-06T22:02:41.835 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd full ratio = 0.9 2021-01-06T22:02:41.835 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd nearfull ratio = 0.8 2021-01-06T22:02:41.835 INFO:teuthology.orchestra.run.smithi091.stdout: osd failsafe full ratio = 0.95 2021-01-06T22:02:41.835 INFO:teuthology.orchestra.run.smithi091.stdout: osd max markdown count = 1000 2021-01-06T22:02:41.835 INFO:teuthology.orchestra.run.smithi091.stdout: osd objectstore = bluestore 2021-01-06T22:02:41.836 INFO:teuthology.orchestra.run.smithi091.stdout: osd op complaint time = 180 2021-01-06T22:02:41.836 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.836 INFO:teuthology.orchestra.run.smithi091.stdout:[mgr] 2021-01-06T22:02:41.836 INFO:teuthology.orchestra.run.smithi091.stdout: debug ms = 1 2021-01-06T22:02:41.836 INFO:teuthology.orchestra.run.smithi091.stdout: debug mgr = 20 2021-01-06T22:02:41.837 INFO:teuthology.orchestra.run.smithi091.stdout: debug mon = 20 2021-01-06T22:02:41.837 INFO:teuthology.orchestra.run.smithi091.stdout: debug auth = 20 2021-01-06T22:02:41.837 INFO:teuthology.orchestra.run.smithi091.stdout: mon reweight min pgs per osd = 4 2021-01-06T22:02:41.837 INFO:teuthology.orchestra.run.smithi091.stdout: mon reweight min bytes per osd = 10 2021-01-06T22:02:41.837 INFO:teuthology.orchestra.run.smithi091.stdout: mgr/telemetry/nag = false 2021-01-06T22:02:41.837 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.838 INFO:teuthology.orchestra.run.smithi091.stdout:[mon] 2021-01-06T22:02:41.838 INFO:teuthology.orchestra.run.smithi091.stdout: debug ms = 1 2021-01-06T22:02:41.838 INFO:teuthology.orchestra.run.smithi091.stdout: debug mon = 20 2021-01-06T22:02:41.838 INFO:teuthology.orchestra.run.smithi091.stdout: debug paxos = 20 2021-01-06T22:02:41.839 INFO:teuthology.orchestra.run.smithi091.stdout: debug auth = 20 2021-01-06T22:02:41.839 INFO:teuthology.orchestra.run.smithi091.stdout: mon data avail warn = 5 2021-01-06T22:02:41.839 INFO:teuthology.orchestra.run.smithi091.stdout: mon mgr mkfs grace = 240 2021-01-06T22:02:41.839 INFO:teuthology.orchestra.run.smithi091.stdout: mon reweight min pgs per osd = 4 2021-01-06T22:02:41.839 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd reporter subtree level = osd 2021-01-06T22:02:41.840 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd prime pg temp = true 2021-01-06T22:02:41.840 INFO:teuthology.orchestra.run.smithi091.stdout: mon reweight min bytes per osd = 10 2021-01-06T22:02:41.840 INFO:teuthology.orchestra.run.smithi091.stdout: mon op complaint time = 120 2021-01-06T22:02:41.840 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-01-06T22:02:41.840 INFO:teuthology.orchestra.run.smithi091.stdout:[client] 2021-01-06T22:02:41.841 INFO:teuthology.orchestra.run.smithi091.stdout: rgw cache enabled = true 2021-01-06T22:02:41.841 INFO:teuthology.orchestra.run.smithi091.stdout: rgw enable ops log = true 2021-01-06T22:02:41.841 INFO:teuthology.orchestra.run.smithi091.stdout: rgw enable usage log = true 2021-01-06T22:02:41.841 INFO:teuthology.orchestra.run.smithi091.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-01-06T22:02:41.841 INFO:teuthology.orchestra.run.smithi091.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-01-06T22:02:41.842 INFO:teuthology.orchestra.run.smithi091.stdout: client mount timeout = 600 2021-01-06T22:02:41.842 INFO:teuthology.orchestra.run.smithi091.stdout: debug client = 20 2021-01-06T22:02:41.842 INFO:teuthology.orchestra.run.smithi091.stdout: debug ms = 1 2021-01-06T22:02:41.842 INFO:teuthology.orchestra.run.smithi091.stdout: rados mon op timeout = 15m 2021-01-06T22:02:41.842 INFO:teuthology.orchestra.run.smithi091.stdout: rados osd op timeout = 15m 2021-01-06T22:02:41.843 INFO:teuthology.orchestra.run.smithi091.stdout:[mon.a] 2021-01-06T22:02:41.843 INFO:teuthology.orchestra.run.smithi091.stdout:[mon.b] 2021-01-06T22:02:41.843 INFO:teuthology.orchestra.run.smithi091.stdout:[mon.c] 2021-01-06T22:02:41.843 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.a] 2021-01-06T22:02:41.844 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.c] 2021-01-06T22:02:41.844 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.e] 2021-01-06T22:02:41.844 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.b] 2021-01-06T22:02:41.845 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.d] 2021-01-06T22:02:41.845 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.f] 2021-01-06T22:02:41.845 INFO:teuthology.orchestra.run.smithi091.stdout:[mds] 2021-01-06T22:02:41.845 INFO:teuthology.orchestra.run.smithi091.stdout: debug mds = 20 2021-01-06T22:02:41.845 INFO:teuthology.orchestra.run.smithi091.stdout: debug ms = 1 2021-01-06T22:02:41.846 INFO:teuthology.orchestra.run.smithi091.stdout: mds bal frag = True 2021-01-06T22:02:41.846 INFO:teuthology.orchestra.run.smithi091.stdout: mds bal fragment size max = 10000 2021-01-06T22:02:41.846 INFO:teuthology.orchestra.run.smithi091.stdout: mds bal merge size = 5 2021-01-06T22:02:41.846 INFO:teuthology.orchestra.run.smithi091.stdout: mds bal split bits = 3 2021-01-06T22:02:41.846 INFO:teuthology.orchestra.run.smithi091.stdout: mds bal split size = 100 2021-01-06T22:02:41.847 INFO:teuthology.orchestra.run.smithi091.stdout: mds debug frag = True 2021-01-06T22:02:41.847 INFO:teuthology.orchestra.run.smithi091.stdout: mds debug scatterstat = True 2021-01-06T22:02:41.847 INFO:teuthology.orchestra.run.smithi091.stdout: mds op complaint time = 180 2021-01-06T22:02:41.847 INFO:teuthology.orchestra.run.smithi091.stdout: mds verify scatter = True 2021-01-06T22:02:41.848 INFO:teuthology.orchestra.run.smithi091.stdout: osd op complaint time = 180 2021-01-06T22:02:41.848 INFO:teuthology.orchestra.run.smithi091.stdout: rados mon op timeout = 15m 2021-01-06T22:02:41.848 INFO:teuthology.orchestra.run.smithi091.stdout: rados osd op timeout = 15m 2021-01-06T22:02:41.849 INFO:teuthology.orchestra.run.smithi203.stdout:[global] 2021-01-06T22:02:41.849 INFO:teuthology.orchestra.run.smithi203.stdout: chdir = "" 2021-01-06T22:02:41.849 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-01-06T22:02:41.850 INFO:teuthology.orchestra.run.smithi203.stdout: auth supported = cephx 2021-01-06T22:02:41.850 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.850 INFO:teuthology.orchestra.run.smithi203.stdout: filestore xattr use omap = true 2021-01-06T22:02:41.850 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.851 INFO:teuthology.orchestra.run.smithi203.stdout: mon clock drift allowed = 1.000 2021-01-06T22:02:41.851 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.851 INFO:teuthology.orchestra.run.smithi203.stdout: osd crush chooseleaf type = 0 2021-01-06T22:02:41.851 INFO:teuthology.orchestra.run.smithi203.stdout: auth debug = true 2021-01-06T22:02:41.851 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.852 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on old message = true 2021-01-06T22:02:41.852 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on bug = true 2021-01-06T22:02:41.852 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.852 INFO:teuthology.orchestra.run.smithi203.stdout: mon max pg per osd = 10000 # >= luminous 2021-01-06T22:02:41.852 INFO:teuthology.orchestra.run.smithi203.stdout: mon pg warn max object skew = 0 2021-01-06T22:02:41.853 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.853 INFO:teuthology.orchestra.run.smithi203.stdout: # disable pg_autoscaler by default for new pools 2021-01-06T22:02:41.853 INFO:teuthology.orchestra.run.smithi203.stdout: osd_pool_default_pg_autoscale_mode = off 2021-01-06T22:02:41.853 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.853 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default size = 2 2021-01-06T22:02:41.854 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.854 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow primary affinity = true 2021-01-06T22:02:41.854 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow pg remap = true 2021-01-06T22:02:41.854 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on legacy crush tunables = false 2021-01-06T22:02:41.854 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on crush straw calc version zero = false 2021-01-06T22:02:41.855 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on no sortbitwise = false 2021-01-06T22:02:41.855 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on osd down out interval zero = false 2021-01-06T22:02:41.855 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on too few osds = false 2021-01-06T22:02:41.855 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-01-06T22:02:41.855 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_no_redundancy = false 2021-01-06T22:02:41.856 INFO:teuthology.orchestra.run.smithi203.stdout: mon_allow_pool_size_one = true 2021-01-06T22:02:41.856 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.856 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-01-06T22:02:41.856 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.856 INFO:teuthology.orchestra.run.smithi203.stdout: osd default data pool replay window = 5 2021-01-06T22:02:41.857 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.857 INFO:teuthology.orchestra.run.smithi203.stdout: mon allow pool delete = true 2021-01-06T22:02:41.857 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.857 INFO:teuthology.orchestra.run.smithi203.stdout: mon cluster log file level = debug 2021-01-06T22:02:41.857 INFO:teuthology.orchestra.run.smithi203.stdout: debug asserts on shutdown = true 2021-01-06T22:02:41.858 INFO:teuthology.orchestra.run.smithi203.stdout: mon health detail to clog = false 2021-01-06T22:02:41.858 INFO:teuthology.orchestra.run.smithi203.stdout: mon host = "172.21.15.203,172.21.15.91,[v2:172.21.15.91:3301,v1:172.21.15.91:6790]" 2021-01-06T22:02:41.858 INFO:teuthology.orchestra.run.smithi203.stdout: mon client directed command retry = 5 2021-01-06T22:02:41.858 INFO:teuthology.orchestra.run.smithi203.stdout: ms inject delay max = 1 2021-01-06T22:02:41.859 INFO:teuthology.orchestra.run.smithi203.stdout: ms inject delay probability = 0.005 2021-01-06T22:02:41.859 INFO:teuthology.orchestra.run.smithi203.stdout: ms inject delay type = osd mds 2021-01-06T22:02:41.859 INFO:teuthology.orchestra.run.smithi203.stdout: ms inject socket failures = 2500 2021-01-06T22:02:41.859 INFO:teuthology.orchestra.run.smithi203.stdout: fsid = f96dbde2-6d40-40dc-b34a-9fb254ce805f 2021-01-06T22:02:41.859 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.860 INFO:teuthology.orchestra.run.smithi203.stdout:[osd] 2021-01-06T22:02:41.860 INFO:teuthology.orchestra.run.smithi203.stdout: osd journal size = 100 2021-01-06T22:02:41.860 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.860 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub load threshold = 5.0 2021-01-06T22:02:41.860 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub max interval = 600 2021-01-06T22:02:41.861 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.861 INFO:teuthology.orchestra.run.smithi203.stdout: osd recover clone overlap = true 2021-01-06T22:02:41.861 INFO:teuthology.orchestra.run.smithi203.stdout: osd recovery max chunk = 1048576 2021-01-06T22:02:41.861 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.861 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug shutdown = true 2021-01-06T22:02:41.862 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug op order = true 2021-01-06T22:02:41.862 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug verify stray on activate = true 2021-01-06T22:02:41.862 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.862 INFO:teuthology.orchestra.run.smithi203.stdout: osd open classes on start = true 2021-01-06T22:02:41.862 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug pg log writeout = true 2021-01-06T22:02:41.863 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.863 INFO:teuthology.orchestra.run.smithi203.stdout: osd deep scrub update digest min age = 30 2021-01-06T22:02:41.863 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.863 INFO:teuthology.orchestra.run.smithi203.stdout: osd map max advance = 10 2021-01-06T22:02:41.864 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.864 INFO:teuthology.orchestra.run.smithi203.stdout: journal zero on create = true 2021-01-06T22:02:41.864 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.864 INFO:teuthology.orchestra.run.smithi203.stdout: filestore ondisk finisher threads = 3 2021-01-06T22:02:41.864 INFO:teuthology.orchestra.run.smithi203.stdout: filestore apply finisher threads = 3 2021-01-06T22:02:41.865 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.865 INFO:teuthology.orchestra.run.smithi203.stdout: bdev debug aio = true 2021-01-06T22:02:41.865 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug misdirected ops = true 2021-01-06T22:02:41.865 INFO:teuthology.orchestra.run.smithi203.stdout: bdev async discard = True 2021-01-06T22:02:41.865 INFO:teuthology.orchestra.run.smithi203.stdout: bdev enable discard = True 2021-01-06T22:02:41.866 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore allocator = bitmap 2021-01-06T22:02:41.866 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore block size = 96636764160 2021-01-06T22:02:41.866 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore fsck on mount = True 2021-01-06T22:02:41.866 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluefs = 20 2021-01-06T22:02:41.866 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluestore = 20 2021-01-06T22:02:41.867 INFO:teuthology.orchestra.run.smithi203.stdout: debug filestore = 20 2021-01-06T22:02:41.867 INFO:teuthology.orchestra.run.smithi203.stdout: debug journal = 20 2021-01-06T22:02:41.867 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 20 2021-01-06T22:02:41.867 INFO:teuthology.orchestra.run.smithi203.stdout: debug osd = 25 2021-01-06T22:02:41.867 INFO:teuthology.orchestra.run.smithi203.stdout: debug rocksdb = 10 2021-01-06T22:02:41.868 INFO:teuthology.orchestra.run.smithi203.stdout: filestore flush min = 0 2021-01-06T22:02:41.868 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd backfillfull_ratio = 0.85 2021-01-06T22:02:41.868 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd full ratio = 0.9 2021-01-06T22:02:41.868 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd nearfull ratio = 0.8 2021-01-06T22:02:41.868 INFO:teuthology.orchestra.run.smithi203.stdout: osd failsafe full ratio = 0.95 2021-01-06T22:02:41.869 INFO:teuthology.orchestra.run.smithi203.stdout: osd max markdown count = 1000 2021-01-06T22:02:41.869 INFO:teuthology.orchestra.run.smithi203.stdout: osd objectstore = bluestore 2021-01-06T22:02:41.869 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2021-01-06T22:02:41.869 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.869 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr] 2021-01-06T22:02:41.870 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2021-01-06T22:02:41.870 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgr = 20 2021-01-06T22:02:41.870 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2021-01-06T22:02:41.870 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2021-01-06T22:02:41.870 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2021-01-06T22:02:41.871 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2021-01-06T22:02:41.871 INFO:teuthology.orchestra.run.smithi203.stdout: mgr/telemetry/nag = false 2021-01-06T22:02:41.871 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.872 INFO:teuthology.orchestra.run.smithi203.stdout:[mon] 2021-01-06T22:02:41.872 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2021-01-06T22:02:41.872 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2021-01-06T22:02:41.872 INFO:teuthology.orchestra.run.smithi203.stdout: debug paxos = 20 2021-01-06T22:02:41.872 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2021-01-06T22:02:41.873 INFO:teuthology.orchestra.run.smithi203.stdout: mon data avail warn = 5 2021-01-06T22:02:41.873 INFO:teuthology.orchestra.run.smithi203.stdout: mon mgr mkfs grace = 240 2021-01-06T22:02:41.873 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2021-01-06T22:02:41.873 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd reporter subtree level = osd 2021-01-06T22:02:41.873 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd prime pg temp = true 2021-01-06T22:02:41.874 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2021-01-06T22:02:41.874 INFO:teuthology.orchestra.run.smithi203.stdout: mon op complaint time = 120 2021-01-06T22:02:41.874 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:02:41.874 INFO:teuthology.orchestra.run.smithi203.stdout:[client] 2021-01-06T22:02:41.875 INFO:teuthology.orchestra.run.smithi203.stdout: rgw cache enabled = true 2021-01-06T22:02:41.875 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable ops log = true 2021-01-06T22:02:41.875 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable usage log = true 2021-01-06T22:02:41.875 INFO:teuthology.orchestra.run.smithi203.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-01-06T22:02:41.876 INFO:teuthology.orchestra.run.smithi203.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-01-06T22:02:41.876 INFO:teuthology.orchestra.run.smithi203.stdout: client mount timeout = 600 2021-01-06T22:02:41.876 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2021-01-06T22:02:41.876 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2021-01-06T22:02:41.877 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 15m 2021-01-06T22:02:41.877 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 15m 2021-01-06T22:02:41.877 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.a] 2021-01-06T22:02:41.877 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.b] 2021-01-06T22:02:41.877 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.c] 2021-01-06T22:02:41.877 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2021-01-06T22:02:41.878 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2021-01-06T22:02:41.878 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.e] 2021-01-06T22:02:41.878 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2021-01-06T22:02:41.878 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2021-01-06T22:02:41.878 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.f] 2021-01-06T22:02:41.879 INFO:teuthology.orchestra.run.smithi203.stdout:[mds] 2021-01-06T22:02:41.879 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds = 20 2021-01-06T22:02:41.879 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2021-01-06T22:02:41.879 INFO:teuthology.orchestra.run.smithi203.stdout: mds bal frag = True 2021-01-06T22:02:41.881 INFO:teuthology.orchestra.run.smithi203.stdout: mds bal fragment size max = 10000 2021-01-06T22:02:41.882 INFO:teuthology.orchestra.run.smithi203.stdout: mds bal merge size = 5 2021-01-06T22:02:41.882 INFO:teuthology.orchestra.run.smithi203.stdout: mds bal split bits = 3 2021-01-06T22:02:41.882 INFO:teuthology.orchestra.run.smithi203.stdout: mds bal split size = 100 2021-01-06T22:02:41.882 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug frag = True 2021-01-06T22:02:41.882 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug scatterstat = True 2021-01-06T22:02:41.883 INFO:teuthology.orchestra.run.smithi203.stdout: mds op complaint time = 180 2021-01-06T22:02:41.883 INFO:teuthology.orchestra.run.smithi203.stdout: mds verify scatter = True 2021-01-06T22:02:41.883 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2021-01-06T22:02:41.883 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 15m 2021-01-06T22:02:41.883 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 15m 2021-01-06T22:02:41.885 INFO:tasks.ceph:Creating admin key on mon.a... 2021-01-06T22:02:41.885 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2021-01-06T22:02:41.973 INFO:tasks.ceph:Copying monmap to all nodes... 2021-01-06T22:02:41.974 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:02:41.974 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2021-01-06T22:02:42.021 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:02:42.022 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2021-01-06T22:02:42.072 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi203.front.sepia.ceph.com 2021-01-06T22:02:42.072 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:02:42.073 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.keyring 2021-01-06T22:02:42.073 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-01-06T22:02:42.142 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:02:42.143 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-01-06T22:02:42.189 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi091.front.sepia.ceph.com 2021-01-06T22:02:42.189 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:02:42.190 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/etc/ceph/ceph.keyring 2021-01-06T22:02:42.190 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-01-06T22:02:42.215 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:02:42.215 DEBUG:teuthology.orchestra.run.smithi091:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-01-06T22:02:42.265 INFO:tasks.ceph:Setting up mon nodes... 2021-01-06T22:02:42.266 INFO:tasks.ceph:Setting up mgr nodes... 2021-01-06T22:02:42.266 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2021-01-06T22:02:42.309 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2021-01-06T22:02:42.311 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2021-01-06T22:02:42.352 INFO:teuthology.orchestra.run.smithi091.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2021-01-06T22:02:42.354 INFO:tasks.ceph:Setting up mds nodes... 2021-01-06T22:02:42.355 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2021-01-06T22:02:42.398 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2021-01-06T22:02:42.400 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2021-01-06T22:02:42.460 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2021-01-06T22:02:42.544 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2021-01-06T22:02:42.546 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2021-01-06T22:02:42.601 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2021-01-06T22:02:42.689 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2021-01-06T22:02:42.722 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2021-01-06T22:02:42.779 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2021-01-06T22:02:42.822 INFO:teuthology.orchestra.run.smithi091.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2021-01-06T22:02:42.824 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2021-01-06T22:02:42.881 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2021-01-06T22:02:42.966 INFO:teuthology.orchestra.run.smithi091.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2021-01-06T22:02:42.968 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2021-01-06T22:02:43.026 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2021-01-06T22:02:43.108 INFO:teuthology.orchestra.run.smithi091.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2021-01-06T22:02:43.110 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2021-01-06T22:02:43.167 INFO:tasks.ceph_client:Setting up client nodes... 2021-01-06T22:02:43.169 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2021-01-06T22:02:43.205 INFO:teuthology.orchestra.run.smithi203.stdout:creating /etc/ceph/ceph.client.0.keyring 2021-01-06T22:02:43.215 INFO:tasks.ceph:Running mkfs on osd nodes... 2021-01-06T22:02:43.216 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi203.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi091.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2021-01-06T22:02:43.217 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2021-01-06T22:02:43.277 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-01-06T22:02:43.277 INFO:tasks.ceph:role: osd.0 2021-01-06T22:02:43.277 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi203.front.sepia.ceph.com 2021-01-06T22:02:43.278 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-01-06T22:02:43.816 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2021-01-06T22:02:43.816 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2021-01-06T22:02:43.817 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-01-06T22:02:43.817 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-01-06T22:02:43.817 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2021-01-06T22:02:43.817 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-01-06T22:02:43.818 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-01-06T22:02:43.818 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-01-06T22:02:43.818 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-01-06T22:02:43.819 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2021-01-06T22:02:43.819 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2021-01-06T22:02:43.905 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2021-01-06T22:02:44.032 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: /sbin/restorecon: command not found 2021-01-06T22:02:44.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-01-06T22:02:44.034 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2021-01-06T22:02:44.091 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-01-06T22:02:44.091 INFO:tasks.ceph:role: osd.1 2021-01-06T22:02:44.092 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi203.front.sepia.ceph.com 2021-01-06T22:02:44.092 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-01-06T22:02:44.576 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2021-01-06T22:02:44.576 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2021-01-06T22:02:44.577 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-01-06T22:02:44.577 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-01-06T22:02:44.577 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2021-01-06T22:02:44.577 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-01-06T22:02:44.577 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-01-06T22:02:44.578 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-01-06T22:02:44.578 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-01-06T22:02:44.579 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2021-01-06T22:02:44.579 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2021-01-06T22:02:44.598 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2021-01-06T22:02:44.650 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: /sbin/restorecon: command not found 2021-01-06T22:02:44.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-01-06T22:02:44.652 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2021-01-06T22:02:44.710 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-01-06T22:02:44.711 INFO:tasks.ceph:role: osd.2 2021-01-06T22:02:44.712 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi203.front.sepia.ceph.com 2021-01-06T22:02:44.712 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-01-06T22:02:45.226 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2021-01-06T22:02:45.227 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2021-01-06T22:02:45.227 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-01-06T22:02:45.227 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-01-06T22:02:45.228 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2021-01-06T22:02:45.228 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-01-06T22:02:45.228 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-01-06T22:02:45.228 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-01-06T22:02:45.228 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-01-06T22:02:45.229 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2021-01-06T22:02:45.230 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2021-01-06T22:02:45.279 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2021-01-06T22:02:45.289 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: /sbin/restorecon: command not found 2021-01-06T22:02:45.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-01-06T22:02:45.289 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2021-01-06T22:02:45.351 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-01-06T22:02:45.351 INFO:tasks.ceph:role: osd.3 2021-01-06T22:02:45.351 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi203.front.sepia.ceph.com 2021-01-06T22:02:45.352 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2021-01-06T22:02:45.925 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2021-01-06T22:02:45.926 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2021-01-06T22:02:45.926 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-01-06T22:02:45.926 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-01-06T22:02:45.926 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2021-01-06T22:02:45.926 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-01-06T22:02:45.927 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-01-06T22:02:45.927 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-01-06T22:02:45.927 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-01-06T22:02:45.927 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2021-01-06T22:02:45.928 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2021-01-06T22:02:45.992 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2021-01-06T22:02:46.009 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: /sbin/restorecon: command not found 2021-01-06T22:02:46.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-01-06T22:02:46.011 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-01-06T22:02:46.122 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:46.124+0000 7f0b3bbaef00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2021-01-06T22:02:46.124 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:46.128+0000 7f0b3bbaef00 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2021-01-06T22:02:46.124 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:46.128+0000 7f0b3bbaef00 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2021-01-06T22:02:46.124 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:46.128+0000 7f0b3bbaef00 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2021-01-06T22:02:46.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:46.128+0000 7f0b3bbaef00 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2021-01-06T22:02:46.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:46.132+0000 7f0b3bbaef00 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2021-01-06T22:02:49.095 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2021-01-06T22:02:49.113 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-01-06T22:02:49.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:49.192+0000 7f39b6104f00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2021-01-06T22:02:49.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:49.192+0000 7f39b6104f00 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2021-01-06T22:02:49.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:49.192+0000 7f39b6104f00 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2021-01-06T22:02:49.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:49.192+0000 7f39b6104f00 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2021-01-06T22:02:49.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:49.192+0000 7f39b6104f00 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2021-01-06T22:02:49.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:49.192+0000 7f39b6104f00 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2021-01-06T22:02:52.151 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2021-01-06T22:02:52.211 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-01-06T22:02:52.293 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:52.292+0000 7fdb4f424f00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2021-01-06T22:02:52.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:52.292+0000 7fdb4f424f00 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2021-01-06T22:02:52.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:52.292+0000 7fdb4f424f00 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2021-01-06T22:02:52.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:52.292+0000 7fdb4f424f00 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2021-01-06T22:02:52.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:52.292+0000 7fdb4f424f00 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2021-01-06T22:02:52.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:52.292+0000 7fdb4f424f00 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2021-01-06T22:02:55.247 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2021-01-06T22:02:55.307 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-01-06T22:02:55.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:55.388+0000 7fe7763b5f00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2021-01-06T22:02:55.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:55.388+0000 7fe7763b5f00 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2021-01-06T22:02:55.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:55.388+0000 7fe7763b5f00 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2021-01-06T22:02:55.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:55.388+0000 7fe7763b5f00 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2021-01-06T22:02:55.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:55.388+0000 7fe7763b5f00 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2021-01-06T22:02:55.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:02:55.388+0000 7fe7763b5f00 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2021-01-06T22:02:58.343 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2021-01-06T22:02:58.402 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2021-01-06T22:02:58.418 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-01-06T22:02:58.419 INFO:tasks.ceph:role: osd.4 2021-01-06T22:02:58.419 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi091.front.sepia.ceph.com 2021-01-06T22:02:58.419 DEBUG:teuthology.orchestra.run.smithi091:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-01-06T22:02:58.955 INFO:teuthology.orchestra.run.smithi091.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2021-01-06T22:02:58.955 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 attr=2, projid32bit=1 2021-01-06T22:02:58.955 INFO:teuthology.orchestra.run.smithi091.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-01-06T22:02:58.956 INFO:teuthology.orchestra.run.smithi091.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-01-06T22:02:58.956 INFO:teuthology.orchestra.run.smithi091.stdout: = sunit=0 swidth=0 blks 2021-01-06T22:02:58.956 INFO:teuthology.orchestra.run.smithi091.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-01-06T22:02:58.957 INFO:teuthology.orchestra.run.smithi091.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-01-06T22:02:58.957 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-01-06T22:02:58.958 INFO:teuthology.orchestra.run.smithi091.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-01-06T22:02:58.959 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi091.front.sepia.ceph.com -o noatime 2021-01-06T22:02:58.959 DEBUG:teuthology.orchestra.run.smithi091:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2021-01-06T22:02:59.056 DEBUG:teuthology.orchestra.run.smithi091:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2021-01-06T22:02:59.070 INFO:teuthology.orchestra.run.smithi091.stderr:sudo: /sbin/restorecon: command not found 2021-01-06T22:02:59.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-01-06T22:02:59.071 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2021-01-06T22:02:59.122 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-01-06T22:02:59.123 INFO:tasks.ceph:role: osd.5 2021-01-06T22:02:59.123 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi091.front.sepia.ceph.com 2021-01-06T22:02:59.123 DEBUG:teuthology.orchestra.run.smithi091:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-01-06T22:02:59.634 INFO:teuthology.orchestra.run.smithi091.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2021-01-06T22:02:59.634 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 attr=2, projid32bit=1 2021-01-06T22:02:59.635 INFO:teuthology.orchestra.run.smithi091.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-01-06T22:02:59.635 INFO:teuthology.orchestra.run.smithi091.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-01-06T22:02:59.636 INFO:teuthology.orchestra.run.smithi091.stdout: = sunit=0 swidth=0 blks 2021-01-06T22:02:59.636 INFO:teuthology.orchestra.run.smithi091.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-01-06T22:02:59.636 INFO:teuthology.orchestra.run.smithi091.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-01-06T22:02:59.638 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-01-06T22:02:59.638 INFO:teuthology.orchestra.run.smithi091.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-01-06T22:02:59.639 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi091.front.sepia.ceph.com -o noatime 2021-01-06T22:02:59.640 DEBUG:teuthology.orchestra.run.smithi091:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2021-01-06T22:02:59.697 DEBUG:teuthology.orchestra.run.smithi091:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2021-01-06T22:02:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:sudo: /sbin/restorecon: command not found 2021-01-06T22:02:59.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-01-06T22:02:59.710 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2021-01-06T22:02:59.764 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-01-06T22:02:59.764 INFO:tasks.ceph:role: osd.6 2021-01-06T22:02:59.764 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi091.front.sepia.ceph.com 2021-01-06T22:02:59.765 DEBUG:teuthology.orchestra.run.smithi091:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-01-06T22:03:00.265 INFO:teuthology.orchestra.run.smithi091.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2021-01-06T22:03:00.266 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 attr=2, projid32bit=1 2021-01-06T22:03:00.266 INFO:teuthology.orchestra.run.smithi091.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-01-06T22:03:00.266 INFO:teuthology.orchestra.run.smithi091.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-01-06T22:03:00.267 INFO:teuthology.orchestra.run.smithi091.stdout: = sunit=0 swidth=0 blks 2021-01-06T22:03:00.267 INFO:teuthology.orchestra.run.smithi091.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-01-06T22:03:00.267 INFO:teuthology.orchestra.run.smithi091.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-01-06T22:03:00.268 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-01-06T22:03:00.268 INFO:teuthology.orchestra.run.smithi091.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-01-06T22:03:00.270 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi091.front.sepia.ceph.com -o noatime 2021-01-06T22:03:00.271 DEBUG:teuthology.orchestra.run.smithi091:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2021-01-06T22:03:00.324 DEBUG:teuthology.orchestra.run.smithi091:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2021-01-06T22:03:00.335 INFO:teuthology.orchestra.run.smithi091.stderr:sudo: /sbin/restorecon: command not found 2021-01-06T22:03:00.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-01-06T22:03:00.337 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2021-01-06T22:03:00.393 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-01-06T22:03:00.394 INFO:tasks.ceph:role: osd.7 2021-01-06T22:03:00.394 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi091.front.sepia.ceph.com 2021-01-06T22:03:00.395 DEBUG:teuthology.orchestra.run.smithi091:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2021-01-06T22:03:00.930 INFO:teuthology.orchestra.run.smithi091.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2021-01-06T22:03:00.930 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 attr=2, projid32bit=1 2021-01-06T22:03:00.931 INFO:teuthology.orchestra.run.smithi091.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-01-06T22:03:00.931 INFO:teuthology.orchestra.run.smithi091.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-01-06T22:03:00.931 INFO:teuthology.orchestra.run.smithi091.stdout: = sunit=0 swidth=0 blks 2021-01-06T22:03:00.931 INFO:teuthology.orchestra.run.smithi091.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-01-06T22:03:00.931 INFO:teuthology.orchestra.run.smithi091.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-01-06T22:03:00.932 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-01-06T22:03:00.932 INFO:teuthology.orchestra.run.smithi091.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-01-06T22:03:00.933 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi091.front.sepia.ceph.com -o noatime 2021-01-06T22:03:00.933 DEBUG:teuthology.orchestra.run.smithi091:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2021-01-06T22:03:00.976 DEBUG:teuthology.orchestra.run.smithi091:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2021-01-06T22:03:01.030 INFO:teuthology.orchestra.run.smithi091.stderr:sudo: /sbin/restorecon: command not found 2021-01-06T22:03:01.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-01-06T22:03:01.032 DEBUG:teuthology.orchestra.run.smithi091:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-01-06T22:03:01.125 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:01.126+0000 7f7f83c3df00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2021-01-06T22:03:01.126 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:01.126+0000 7f7f83c3df00 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2021-01-06T22:03:01.126 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:01.126+0000 7f7f83c3df00 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2021-01-06T22:03:01.127 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:01.126+0000 7f7f83c3df00 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2021-01-06T22:03:01.127 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:01.126+0000 7f7f83c3df00 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2021-01-06T22:03:01.136 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:01.138+0000 7f7f83c3df00 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2021-01-06T22:03:04.091 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2021-01-06T22:03:04.105 DEBUG:teuthology.orchestra.run.smithi091:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-01-06T22:03:04.185 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:04.186+0000 7fe3458c5f00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2021-01-06T22:03:04.185 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:04.186+0000 7fe3458c5f00 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2021-01-06T22:03:04.185 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:04.186+0000 7fe3458c5f00 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2021-01-06T22:03:04.186 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:04.186+0000 7fe3458c5f00 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2021-01-06T22:03:04.186 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:04.186+0000 7fe3458c5f00 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2021-01-06T22:03:04.186 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:04.186+0000 7fe3458c5f00 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2021-01-06T22:03:07.139 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2021-01-06T22:03:07.204 DEBUG:teuthology.orchestra.run.smithi091:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-01-06T22:03:07.290 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:07.290+0000 7f0c9da4df00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2021-01-06T22:03:07.291 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:07.290+0000 7f0c9da4df00 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2021-01-06T22:03:07.291 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:07.290+0000 7f0c9da4df00 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2021-01-06T22:03:07.291 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:07.290+0000 7f0c9da4df00 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2021-01-06T22:03:07.292 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:07.290+0000 7f0c9da4df00 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2021-01-06T22:03:07.293 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:07.294+0000 7f0c9da4df00 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2021-01-06T22:03:10.242 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2021-01-06T22:03:10.301 DEBUG:teuthology.orchestra.run.smithi091:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-01-06T22:03:10.380 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:10.382+0000 7f2eceef5f00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2021-01-06T22:03:10.381 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:10.382+0000 7f2eceef5f00 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2021-01-06T22:03:10.381 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:10.382+0000 7f2eceef5f00 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2021-01-06T22:03:10.381 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:10.382+0000 7f2eceef5f00 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2021-01-06T22:03:10.381 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:10.382+0000 7f2eceef5f00 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2021-01-06T22:03:10.382 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:10.382+0000 7f2eceef5f00 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2021-01-06T22:03:10.726 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:03:10.729 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:03:13.334 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2021-01-06T22:03:13.350 INFO:tasks.ceph:Reading keys from all nodes... 2021-01-06T22:03:13.351 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:03:13.351 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2021-01-06T22:03:13.366 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:03:13.367 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2021-01-06T22:03:13.424 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:03:13.425 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2021-01-06T22:03:13.480 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:03:13.481 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2021-01-06T22:03:13.537 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:03:13.537 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2021-01-06T22:03:13.594 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:03:13.594 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2021-01-06T22:03:13.650 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:03:13.650 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2021-01-06T22:03:13.706 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:03:13.706 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2021-01-06T22:03:13.763 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:03:13.763 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2021-01-06T22:03:13.781 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:03:13.782 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2021-01-06T22:03:13.837 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:03:13.837 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2021-01-06T22:03:13.893 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:03:13.894 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2021-01-06T22:03:13.949 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:03:13.949 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2021-01-06T22:03:14.003 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:03:14.003 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2021-01-06T22:03:14.060 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:03:14.060 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2021-01-06T22:03:14.116 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:03:14.116 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2021-01-06T22:03:14.173 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:03:14.173 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2021-01-06T22:03:14.181 INFO:tasks.ceph:Adding keys to all mons... 2021-01-06T22:03:14.181 DEBUG:teuthology.orchestra.run.smithi091:> sudo tee -a /etc/ceph/ceph.keyring 2021-01-06T22:03:14.216 DEBUG:teuthology.orchestra.run.smithi203:> sudo tee -a /etc/ceph/ceph.keyring 2021-01-06T22:03:14.226 INFO:teuthology.orchestra.run.smithi091.stdout:[mgr.y] 2021-01-06T22:03:14.226 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCCM/ZfTB+wEhAAho0Gw7CR8gRdSSUb01a1BQ== 2021-01-06T22:03:14.227 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.a] 2021-01-06T22:03:14.227 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCCM/Zf3z3+FxAAvFh04cDz8Qv/nLuSgUqR7g== 2021-01-06T22:03:14.227 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.c] 2021-01-06T22:03:14.227 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCCM/ZfJJ2zIBAAKjxj1hBkp7Mj/ajodSBm6g== 2021-01-06T22:03:14.228 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.e] 2021-01-06T22:03:14.228 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCCM/ZfGxdEKRAA/xubP0YR7LGSPIL600MW/g== 2021-01-06T22:03:14.228 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.0] 2021-01-06T22:03:14.229 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCGM/Zf4ceQBxAATebveWKPNgGpqWglWtW+ow== 2021-01-06T22:03:14.229 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.1] 2021-01-06T22:03:14.229 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCJM/ZfR+LJCxAAS1BAlUD7uWX+xjkTYS4FhQ== 2021-01-06T22:03:14.229 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.2] 2021-01-06T22:03:14.230 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCMM/ZfiiK9ERAAZBAZZDE5YeRthj8QlWiVVQ== 2021-01-06T22:03:14.230 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.3] 2021-01-06T22:03:14.230 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCPM/Zfzdk1FxAA8r76e3AgTVn7lzT8qsSQsQ== 2021-01-06T22:03:14.230 INFO:teuthology.orchestra.run.smithi091.stdout:[mgr.x] 2021-01-06T22:03:14.231 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCCM/Zf1qBQFRAAywuuiizxWgMSufnYfDHTvA== 2021-01-06T22:03:14.231 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.b] 2021-01-06T22:03:14.231 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCCM/ZfKKFGMRAAduuo4M9i4g31LRK/tJLPdw== 2021-01-06T22:03:14.231 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.d] 2021-01-06T22:03:14.231 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCCM/Zf8A7zORAAPkn1V6uPDgWTYJc0eyX8vw== 2021-01-06T22:03:14.232 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.f] 2021-01-06T22:03:14.232 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCDM/Zf9qPKBhAAsIGYoXQ/nHLDq33r4AlVRQ== 2021-01-06T22:03:14.232 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.4] 2021-01-06T22:03:14.232 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCVM/ZfRrvXBxAAhZSnxl2J0efRKvSMz7wvCg== 2021-01-06T22:03:14.233 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.5] 2021-01-06T22:03:14.233 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCYM/Zf/AFhCxAAnEOXHJ0jQISc7v16678x5A== 2021-01-06T22:03:14.233 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.6] 2021-01-06T22:03:14.233 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCbM/ZfN0mdERAA8EqYek6Z0vsMbAfTTJ0GGg== 2021-01-06T22:03:14.233 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.7] 2021-01-06T22:03:14.234 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCeM/Zf7TsHFxAAEo/A10YAbq61Ewca/ANlww== 2021-01-06T22:03:14.234 INFO:teuthology.orchestra.run.smithi091.stdout:[client.0] 2021-01-06T22:03:14.234 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCDM/ZftG1gDBAAleM3oU4PJ9rLdttzmhJB1Q== 2021-01-06T22:03:14.236 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.y] 2021-01-06T22:03:14.236 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCCM/ZfTB+wEhAAho0Gw7CR8gRdSSUb01a1BQ== 2021-01-06T22:03:14.236 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2021-01-06T22:03:14.236 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCCM/Zf3z3+FxAAvFh04cDz8Qv/nLuSgUqR7g== 2021-01-06T22:03:14.237 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2021-01-06T22:03:14.237 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCCM/ZfJJ2zIBAAKjxj1hBkp7Mj/ajodSBm6g== 2021-01-06T22:03:14.237 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.e] 2021-01-06T22:03:14.237 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCCM/ZfGxdEKRAA/xubP0YR7LGSPIL600MW/g== 2021-01-06T22:03:14.238 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.0] 2021-01-06T22:03:14.238 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCGM/Zf4ceQBxAATebveWKPNgGpqWglWtW+ow== 2021-01-06T22:03:14.238 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.1] 2021-01-06T22:03:14.238 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCJM/ZfR+LJCxAAS1BAlUD7uWX+xjkTYS4FhQ== 2021-01-06T22:03:14.239 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.2] 2021-01-06T22:03:14.239 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCMM/ZfiiK9ERAAZBAZZDE5YeRthj8QlWiVVQ== 2021-01-06T22:03:14.239 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.3] 2021-01-06T22:03:14.239 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCPM/Zfzdk1FxAA8r76e3AgTVn7lzT8qsSQsQ== 2021-01-06T22:03:14.239 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.x] 2021-01-06T22:03:14.240 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCCM/Zf1qBQFRAAywuuiizxWgMSufnYfDHTvA== 2021-01-06T22:03:14.240 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2021-01-06T22:03:14.240 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCCM/ZfKKFGMRAAduuo4M9i4g31LRK/tJLPdw== 2021-01-06T22:03:14.240 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2021-01-06T22:03:14.241 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCCM/Zf8A7zORAAPkn1V6uPDgWTYJc0eyX8vw== 2021-01-06T22:03:14.241 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.f] 2021-01-06T22:03:14.241 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCDM/Zf9qPKBhAAsIGYoXQ/nHLDq33r4AlVRQ== 2021-01-06T22:03:14.241 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.4] 2021-01-06T22:03:14.241 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCVM/ZfRrvXBxAAhZSnxl2J0efRKvSMz7wvCg== 2021-01-06T22:03:14.242 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.5] 2021-01-06T22:03:14.242 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCYM/Zf/AFhCxAAnEOXHJ0jQISc7v16678x5A== 2021-01-06T22:03:14.242 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.6] 2021-01-06T22:03:14.242 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCbM/ZfN0mdERAA8EqYek6Z0vsMbAfTTJ0GGg== 2021-01-06T22:03:14.243 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.7] 2021-01-06T22:03:14.243 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCeM/Zf7TsHFxAAEo/A10YAbq61Ewca/ANlww== 2021-01-06T22:03:14.243 INFO:teuthology.orchestra.run.smithi203.stdout:[client.0] 2021-01-06T22:03:14.243 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCDM/ZftG1gDBAAleM3oU4PJ9rLdttzmhJB1Q== 2021-01-06T22:03:14.244 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-01-06T22:03:14.280 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-01-06T22:03:14.329 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-01-06T22:03:14.364 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-01-06T22:03:14.415 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-01-06T22:03:14.417 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-01-06T22:03:14.502 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-01-06T22:03:14.505 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-01-06T22:03:14.591 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-06T22:03:14.594 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-06T22:03:14.677 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-06T22:03:14.680 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-06T22:03:14.761 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-06T22:03:14.763 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-06T22:03:14.844 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-06T22:03:14.848 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-06T22:03:14.928 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-01-06T22:03:14.931 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-01-06T22:03:15.016 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-01-06T22:03:15.018 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-01-06T22:03:15.104 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-01-06T22:03:15.107 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-01-06T22:03:15.153 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-01-06T22:03:15.156 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-01-06T22:03:15.240 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-06T22:03:15.242 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-06T22:03:15.283 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-06T22:03:15.328 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-06T22:03:15.371 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-06T22:03:15.412 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-06T22:03:15.456 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-06T22:03:15.496 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-06T22:03:15.540 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-01-06T22:03:15.581 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-01-06T22:03:15.627 INFO:tasks.ceph:Running mkfs on mon nodes... 2021-01-06T22:03:15.628 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2021-01-06T22:03:15.643 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-01-06T22:03:15.740 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2021-01-06T22:03:15.753 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2021-01-06T22:03:15.768 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-01-06T22:03:15.877 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2021-01-06T22:03:15.893 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2021-01-06T22:03:15.947 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-01-06T22:03:16.062 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2021-01-06T22:03:16.078 DEBUG:teuthology.orchestra.run.smithi091:> rm -- /home/ubuntu/cephtest/ceph.monmap 2021-01-06T22:03:16.124 DEBUG:teuthology.orchestra.run.smithi203:> rm -- /home/ubuntu/cephtest/ceph.monmap 2021-01-06T22:03:16.130 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2021-01-06T22:03:16.131 INFO:tasks.ceph.mon.a:Restarting daemon 2021-01-06T22:03:16.131 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2021-01-06T22:03:16.176 INFO:tasks.ceph.mon.a:Started 2021-01-06T22:03:16.177 INFO:tasks.ceph.mon.b:Restarting daemon 2021-01-06T22:03:16.177 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2021-01-06T22:03:16.179 INFO:tasks.ceph.mon.b:Started 2021-01-06T22:03:16.180 INFO:tasks.ceph.mon.c:Restarting daemon 2021-01-06T22:03:16.180 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2021-01-06T22:03:16.182 INFO:tasks.ceph.mon.c:Started 2021-01-06T22:03:16.183 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2021-01-06T22:03:16.183 INFO:tasks.ceph.mgr.y:Restarting daemon 2021-01-06T22:03:16.184 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2021-01-06T22:03:16.186 INFO:tasks.ceph.mgr.y:Started 2021-01-06T22:03:16.186 INFO:tasks.ceph.mgr.x:Restarting daemon 2021-01-06T22:03:16.186 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2021-01-06T22:03:16.189 INFO:tasks.ceph.mgr.x:Started 2021-01-06T22:03:16.189 INFO:tasks.ceph:Setting crush tunables to default 2021-01-06T22:03:16.189 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd crush tunables default 2021-01-06T22:03:16.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.436+0000 7fda5ffff700 1 Processor -- start 2021-01-06T22:03:16.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.436+0000 7fda5ffff700 1 -- start start 2021-01-06T22:03:16.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.436+0000 7fda5ffff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 0x7fda580de6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:16.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.436+0000 7fda5ffff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda5800d770 con 0x7fda5804e980 2021-01-06T22:03:16.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.436+0000 7fda5ffff700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda580deef0 con 0x7fda5804c740 2021-01-06T22:03:16.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.440+0000 7fda5ffff700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda580df190 con 0x7fda5804e5a0 2021-01-06T22:03:16.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.440+0000 7fda5f7fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fda5804c740 0x7fda580d82f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:46448/0 (socket says 172.21.15.203:46448) 2021-01-06T22:03:16.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.440+0000 7fda5f7fe700 1 -- 172.21.15.203:0/3807954111 learned_addr learned my addr 172.21.15.203:0/3807954111 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:16.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.440+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 0x7fda580de6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:16.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.440+0000 7fda5effd700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 msgr2=0x7fda580de6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2021-01-06T22:03:16.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.440+0000 7fda5effd700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 msgr2=0x7fda580de6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:03:16.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.440+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 0x7fda580de6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:03:16.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.440+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 0x7fda580de6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-01-06T22:03:16.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.644+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 0x7fda580de6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:16.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.644+0000 7fda5effd700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 msgr2=0x7fda580de6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2021-01-06T22:03:16.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.644+0000 7fda5effd700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 msgr2=0x7fda580de6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:03:16.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.644+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 0x7fda580de6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:03:16.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:16.644+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 0x7fda580de6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2021-01-06T22:03:17.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:17.048+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 0x7fda580de6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:17.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:17.048+0000 7fda5effd700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 msgr2=0x7fda580de6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2021-01-06T22:03:17.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:17.048+0000 7fda5effd700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 msgr2=0x7fda580de6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:03:17.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:17.048+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 0x7fda580de6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:03:17.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:17.048+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 0x7fda580de6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2021-01-06T22:03:17.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:17.852+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 0x7fda580de6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:17.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:17.852+0000 7fda5effd700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 msgr2=0x7fda580de6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2021-01-06T22:03:17.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:17.852+0000 7fda5effd700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 msgr2=0x7fda580de6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:03:17.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:17.852+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 0x7fda580de6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:03:17.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:17.852+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 0x7fda580de6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2021-01-06T22:03:19.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5cff9700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 msgr2=0x7fda580de6d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-01-06T22:03:19.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5cff9700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.203:3300/0 conn(0x7fda5804e5a0 0x7fda580de6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:19.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5cff9700 1 -- 172.21.15.203:0/3807954111 >> v1:172.21.15.203:6789/0 conn(0x7fda5804c740 legacy=0x7fda580d82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:19.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5cff9700 1 -- 172.21.15.203:0/3807954111 >> v1:172.21.15.91:6789/0 conn(0x7fda5804e980 legacy=0x7fda5800e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:19.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5cff9700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 0x7fda40005860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:19.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5cff9700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 0x7fda400083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:19.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5cff9700 1 -- 172.21.15.203:0/3807954111 --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda580df190 con 0x7fda40005f10 2021-01-06T22:03:19.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5cff9700 1 -- 172.21.15.203:0/3807954111 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda40008a30 con 0x7fda40003410 2021-01-06T22:03:19.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 0x7fda40005860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:19.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5e7fc700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 0x7fda400083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:19.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5effd700 1 -- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 msgr2=0x7fda40005860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2021-01-06T22:03:19.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5effd700 1 -- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 msgr2=0x7fda40005860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:03:19.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 0x7fda40005860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:03:19.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 0x7fda40005860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-01-06T22:03:19.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5e7fc700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 msgr2=0x7fda400083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2021-01-06T22:03:19.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5e7fc700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 msgr2=0x7fda400083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:03:19.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5e7fc700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 0x7fda400083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:03:19.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.440+0000 7fda5e7fc700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 0x7fda400083d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-01-06T22:03:19.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.644+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 0x7fda40005860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:19.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.644+0000 7fda5e7fc700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 0x7fda400083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:19.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.644+0000 7fda5effd700 1 -- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 msgr2=0x7fda40005860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2021-01-06T22:03:19.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.644+0000 7fda5effd700 1 -- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 msgr2=0x7fda40005860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:03:19.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.644+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 0x7fda40005860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:03:19.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.644+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 0x7fda40005860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2021-01-06T22:03:19.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.644+0000 7fda5e7fc700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 msgr2=0x7fda400083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2021-01-06T22:03:19.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.644+0000 7fda5e7fc700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 msgr2=0x7fda400083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:03:19.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.644+0000 7fda5e7fc700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 0x7fda400083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:03:19.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:19.644+0000 7fda5e7fc700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 0x7fda400083d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2021-01-06T22:03:20.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.048+0000 7fda5e7fc700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 0x7fda400083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:20.044 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.048+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 0x7fda40005860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:20.044 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.048+0000 7fda5e7fc700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 msgr2=0x7fda400083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2021-01-06T22:03:20.044 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.048+0000 7fda5e7fc700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 msgr2=0x7fda400083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:03:20.045 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.048+0000 7fda5e7fc700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 0x7fda400083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:03:20.045 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.048+0000 7fda5effd700 1 -- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 msgr2=0x7fda40005860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2021-01-06T22:03:20.045 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.048+0000 7fda5effd700 1 -- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 msgr2=0x7fda40005860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:03:20.045 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.048+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 0x7fda40005860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:03:20.045 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.048+0000 7fda5e7fc700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 0x7fda400083d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2021-01-06T22:03:20.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.048+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 0x7fda40005860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2021-01-06T22:03:20.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.851+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 0x7fda40005860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:20.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.851+0000 7fda5e7fc700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 0x7fda400083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:20.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.851+0000 7fda5effd700 1 -- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 msgr2=0x7fda40005860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2021-01-06T22:03:20.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.851+0000 7fda5effd700 1 -- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 msgr2=0x7fda40005860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:03:20.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.851+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 0x7fda40005860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:03:20.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.851+0000 7fda5e7fc700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 msgr2=0x7fda400083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2021-01-06T22:03:20.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.851+0000 7fda5e7fc700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 msgr2=0x7fda400083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:03:20.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.851+0000 7fda5e7fc700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 0x7fda400083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:03:20.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.851+0000 7fda5effd700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 0x7fda40005860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2021-01-06T22:03:20.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:20.851+0000 7fda5e7fc700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 0x7fda400083d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2021-01-06T22:03:22.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5cff9700 1 -- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 msgr2=0x7fda40005860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-01-06T22:03:22.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5cff9700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40003410 0x7fda40005860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:22.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5cff9700 1 -- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 msgr2=0x7fda400083d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-01-06T22:03:22.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5cff9700 1 --2- 172.21.15.203:0/3807954111 >> v2:172.21.15.91:3300/0 conn(0x7fda40005f10 0x7fda400083d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:22.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5f7fe700 1 -- 172.21.15.203:0/3807954111 reap_dead start 2021-01-06T22:03:22.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5cff9700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda4000beb0 0x7fda4000e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:22.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5cff9700 1 -- 172.21.15.203:0/3807954111 --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda580df430 con 0x7fda40007ae0 2021-01-06T22:03:22.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5cff9700 1 -- 172.21.15.203:0/3807954111 --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4000f7a0 con 0x7fda40003850 2021-01-06T22:03:22.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5cff9700 1 -- 172.21.15.203:0/3807954111 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda580df190 con 0x7fda4000beb0 2021-01-06T22:03:22.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5f7fe700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda4000beb0 0x7fda4000e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:22.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5dffb700 1 -- 172.21.15.203:0/3807954111 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 736469953 0 0) 0x7fda4000f7a0 con 0x7fda40003850 2021-01-06T22:03:22.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5dffb700 1 -- 172.21.15.203:0/3807954111 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda4c003230 con 0x7fda40003850 2021-01-06T22:03:22.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5dffb700 1 -- 172.21.15.203:0/3807954111 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596412083 0 0) 0x7fda580df430 con 0x7fda40007ae0 2021-01-06T22:03:22.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5dffb700 1 -- 172.21.15.203:0/3807954111 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda4000f7a0 con 0x7fda40007ae0 2021-01-06T22:03:22.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5f7fe700 1 -- 172.21.15.203:0/3807954111 >> v1:172.21.15.203:6789/0 conn(0x7fda40003850 legacy=0x7fda400063c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:22.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5dffb700 1 -- 172.21.15.203:0/3807954111 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 150537604 0 0) 0x7fda4c003230 con 0x7fda40003850 2021-01-06T22:03:22.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5f7fe700 1 -- 172.21.15.203:0/3807954111 >> v1:172.21.15.91:6789/0 conn(0x7fda40007ae0 legacy=0x7fda40007ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:22.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5f7fe700 1 -- 172.21.15.203:0/3807954111 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda40010a20 con 0x7fda4000beb0 2021-01-06T22:03:22.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5f7fe700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda4000beb0 0x7fda4000e2a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fda600270b0 tx=0x7fda6005ba60).ready entity=mon.2 client_cookie=31a55bda77ce12e2 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:22.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5dffb700 1 -- 172.21.15.203:0/3807954111 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda500155b0 con 0x7fda4000beb0 2021-01-06T22:03:22.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5dffb700 1 -- 172.21.15.203:0/3807954111 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda60060040 con 0x7fda4000beb0 2021-01-06T22:03:22.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5dffb700 1 -- 172.21.15.203:0/3807954111 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6005da60 con 0x7fda4000beb0 2021-01-06T22:03:22.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5ffff700 1 -- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda4000beb0 msgr2=0x7fda4000e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:22.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.442+0000 7fda5ffff700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda4000beb0 0x7fda4000e2a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fda600270b0 tx=0x7fda6005ba60).stop 2021-01-06T22:03:22.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.446+0000 7fda5ffff700 1 -- 172.21.15.203:0/3807954111 shutdown_connections 2021-01-06T22:03:22.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.446+0000 7fda5ffff700 1 --2- 172.21.15.203:0/3807954111 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda4000beb0 0x7fda4000e2a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:22.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.446+0000 7fda5ffff700 1 -- 172.21.15.203:0/3807954111 >> 172.21.15.203:0/3807954111 conn(0x7fda5800c830 msgr2=0x7fda580098a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:22.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.446+0000 7fda5ffff700 1 -- 172.21.15.203:0/3807954111 shutdown_connections 2021-01-06T22:03:22.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.446+0000 7fda5ffff700 1 -- 172.21.15.203:0/3807954111 wait complete. 2021-01-06T22:03:22.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.446+0000 7fda5ffff700 1 Processor -- start 2021-01-06T22:03:22.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.446+0000 7fda5ffff700 1 -- start start 2021-01-06T22:03:22.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fda40003850 0x7fda580ee6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:22.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40007ae0 0x7fda580d9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:22.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda4000beb0 0x7fda580d8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:22.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5ffff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda580eebd0 con 0x7fda40003850 2021-01-06T22:03:22.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5ffff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda580ea870 con 0x7fda4000beb0 2021-01-06T22:03:22.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5ffff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda580eab10 con 0x7fda40007ae0 2021-01-06T22:03:22.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda4000beb0 0x7fda580d8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:22.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5effd700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fda40003850 0x7fda580ee6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:22.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda4000beb0 0x7fda580d8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:47928/0 (socket says 172.21.15.203:47928) 2021-01-06T22:03:22.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5effd700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fda40003850 0x7fda580ee6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:55772/0 (socket says 172.21.15.203:55772) 2021-01-06T22:03:22.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5f7fe700 1 -- 172.21.15.203:0/105963667 learned_addr learned my addr 172.21.15.203:0/105963667 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:22.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5e7fc700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40007ae0 0x7fda580d9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:22.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5f7fe700 1 -- 172.21.15.203:0/105963667 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40007ae0 msgr2=0x7fda580d9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:22.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5f7fe700 1 --2- 172.21.15.203:0/105963667 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40007ae0 0x7fda580d9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:22.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5f7fe700 1 -- 172.21.15.203:0/105963667 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fda40003850 msgr2=0x7fda580ee6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:22.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5f7fe700 1 --2- 172.21.15.203:0/105963667 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fda40003850 0x7fda580ee6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:22.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5f7fe700 1 -- 172.21.15.203:0/105963667 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda580eadb0 con 0x7fda4000beb0 2021-01-06T22:03:22.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5f7fe700 1 --2- 172.21.15.203:0/105963667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda4000beb0 0x7fda580d8020 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fda60045540 tx=0x7fda60085000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:22.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda46ffd700 1 -- 172.21.15.203:0/105963667 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60068360 con 0x7fda4000beb0 2021-01-06T22:03:22.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda46ffd700 1 -- 172.21.15.203:0/105963667 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6008f070 con 0x7fda4000beb0 2021-01-06T22:03:22.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5ffff700 1 -- 172.21.15.203:0/105963667 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda580eb050 con 0x7fda4000beb0 2021-01-06T22:03:22.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda46ffd700 1 -- 172.21.15.203:0/105963667 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda600684c0 con 0x7fda4000beb0 2021-01-06T22:03:22.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda5ffff700 1 -- 172.21.15.203:0/105963667 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda580eb590 con 0x7fda4000beb0 2021-01-06T22:03:22.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.450+0000 7fda46ffd700 1 -- 172.21.15.203:0/105963667 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fda60094420 con 0x7fda4000beb0 2021-01-06T22:03:22.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.454+0000 7fda46ffd700 1 -- 172.21.15.203:0/105963667 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fda600688f0 con 0x7fda4000beb0 2021-01-06T22:03:22.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.454+0000 7fda5ffff700 1 -- 172.21.15.203:0/105963667 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda58004ff0 con 0x7fda4000beb0 2021-01-06T22:03:22.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.458+0000 7fda46ffd700 1 -- 172.21.15.203:0/105963667 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71894 (secure 0 0 0) 0x7fda60099060 con 0x7fda4000beb0 2021-01-06T22:03:22.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.538+0000 7fda5ffff700 1 -- 172.21.15.203:0/105963667 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fda5800ace0 con 0x7fda4000beb0 2021-01-06T22:03:22.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.550+0000 7fda46ffd700 1 -- 172.21.15.203:0/105963667 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fda600447f0 con 0x7fda4000beb0 2021-01-06T22:03:22.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.554+0000 7fda5ffff700 1 -- 172.21.15.203:0/105963667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda4000beb0 msgr2=0x7fda580d8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:22.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.554+0000 7fda5ffff700 1 --2- 172.21.15.203:0/105963667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda4000beb0 0x7fda580d8020 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fda60045540 tx=0x7fda60085000).stop 2021-01-06T22:03:22.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.554+0000 7fda5ffff700 1 -- 172.21.15.203:0/105963667 shutdown_connections 2021-01-06T22:03:22.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.554+0000 7fda5ffff700 1 --2- 172.21.15.203:0/105963667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda4000beb0 0x7fda580d8020 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:22.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.554+0000 7fda5ffff700 1 --2- 172.21.15.203:0/105963667 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fda40003850 0x7fda580ee6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:22.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.554+0000 7fda5ffff700 1 --2- 172.21.15.203:0/105963667 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda40007ae0 0x7fda580d9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:22.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.554+0000 7fda5ffff700 1 -- 172.21.15.203:0/105963667 >> 172.21.15.203:0/105963667 conn(0x7fda5800c830 msgr2=0x7fda580dd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:22.556 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.554+0000 7fda5ffff700 1 -- 172.21.15.203:0/105963667 shutdown_connections 2021-01-06T22:03:22.556 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:22.554+0000 7fda5ffff700 1 -- 172.21.15.203:0/105963667 wait complete. 2021-01-06T22:03:22.557 INFO:teuthology.orchestra.run.smithi203.stderr:adjusted tunables profile to default 2021-01-06T22:03:22.567 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2021-01-06T22:03:22.567 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:03:22.567 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2021-01-06T22:03:22.582 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:03:22.582 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2021-01-06T22:03:22.635 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:03:22.635 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2021-01-06T22:03:22.690 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:03:22.690 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2021-01-06T22:03:22.745 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:03:22.745 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2021-01-06T22:03:22.760 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:03:22.761 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2021-01-06T22:03:22.816 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:03:22.816 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2021-01-06T22:03:22.870 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2021-01-06T22:03:22.870 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2021-01-06T22:03:22.927 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd new e0b35c09-147c-4ccc-b566-c48b81e25412 0 2021-01-06T22:03:23.100 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b767c7700 1 Processor -- start 2021-01-06T22:03:23.100 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b767c7700 1 -- start start 2021-01-06T22:03:23.101 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b767c7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4b68050860 0x7f4b680453d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.102 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b767c7700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b68035c80 0x7f4b680ddd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.102 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b767c7700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b6804c6d0 con 0x7f4b68044ff0 2021-01-06T22:03:23.102 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b767c7700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b680de280 con 0x7f4b68050860 2021-01-06T22:03:23.102 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b767c7700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b68005c10 con 0x7f4b68035c80 2021-01-06T22:03:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b75fc6700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b68035c80 0x7f4b680ddd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b75fc6700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b68035c80 0x7f4b680ddd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.91:52812/0 (socket says 172.21.15.91:52812) 2021-01-06T22:03:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b75fc6700 1 -- 172.21.15.91:0/3700867260 learned_addr learned my addr 172.21.15.91:0/3700867260 (peer_addr_for_me v2:172.21.15.91:0/0) 2021-01-06T22:03:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b75fc6700 1 -- 172.21.15.91:0/3700867260 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b68035c80 msgr2=0x7f4b680ddd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send send error: (32) Broken pipe 2021-01-06T22:03:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b75fc6700 1 --2- 172.21.15.91:0/3700867260 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b68035c80 0x7f4b680ddd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).write auth request write failed r=-32 ((32) Broken pipe) 2021-01-06T22:03:23.104 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b75fc6700 1 --2- 172.21.15.91:0/3700867260 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b68035c80 0x7f4b680ddd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-01-06T22:03:23.104 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b74fc4700 1 --2- 172.21.15.91:0/3700867260 >> v2:172.21.15.203:3300/0 conn(0x7f4b68050860 0x7f4b680453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.104 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b74fc4700 1 -- 172.21.15.91:0/3700867260 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b68035c80 msgr2=0x7f4b680ddd90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-01-06T22:03:23.104 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b74fc4700 1 --2- 172.21.15.91:0/3700867260 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b68035c80 0x7f4b680ddd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.104 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b74fc4700 1 -- 172.21.15.91:0/3700867260 >> v1:172.21.15.203:6789/0 conn(0x7f4b68044ff0 legacy=0x7f4b680da530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.105 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.102+0000 7f4b74fc4700 1 -- 172.21.15.91:0/3700867260 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b68050d30 con 0x7f4b68050860 2021-01-06T22:03:23.107 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b74fc4700 1 --2- 172.21.15.91:0/3700867260 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b68050860 0x7f4b680453d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f4b6400b9b0 tx=0x7f4b64006de0).ready entity=mon.1 client_cookie=7fa38bcf15d363a7 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:23.108 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b5bfff700 1 -- 172.21.15.91:0/3700867260 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64009830 con 0x7f4b68050860 2021-01-06T22:03:23.108 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b5bfff700 1 -- 172.21.15.91:0/3700867260 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b64009990 con 0x7f4b68050860 2021-01-06T22:03:23.108 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b5bfff700 1 -- 172.21.15.91:0/3700867260 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64009d10 con 0x7f4b68050860 2021-01-06T22:03:23.108 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b767c7700 1 -- 172.21.15.91:0/3700867260 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b68050860 msgr2=0x7f4b680453d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.108 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b767c7700 1 --2- 172.21.15.91:0/3700867260 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b68050860 0x7f4b680453d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f4b6400b9b0 tx=0x7f4b64006de0).stop 2021-01-06T22:03:23.109 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b767c7700 1 -- 172.21.15.91:0/3700867260 shutdown_connections 2021-01-06T22:03:23.109 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b767c7700 1 --2- 172.21.15.91:0/3700867260 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b68035c80 0x7f4b680ddd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.109 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b767c7700 1 --2- 172.21.15.91:0/3700867260 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b68050860 0x7f4b680453d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.109 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b767c7700 1 -- 172.21.15.91:0/3700867260 >> 172.21.15.91:0/3700867260 conn(0x7f4b6800cb20 msgr2=0x7f4b68050030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:23.109 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b767c7700 1 -- 172.21.15.91:0/3700867260 shutdown_connections 2021-01-06T22:03:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b767c7700 1 -- 172.21.15.91:0/3700867260 wait complete. 2021-01-06T22:03:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b767c7700 1 Processor -- start 2021-01-06T22:03:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b767c7700 1 -- start start 2021-01-06T22:03:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b767c7700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b68035c80 0x7f4b68049be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b767c7700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b68044ff0 0x7f4b680f0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.111 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b767c7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b6804a0d0 0x7f4b680f0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.112 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b767c7700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b68049510 con 0x7f4b68035c80 2021-01-06T22:03:23.112 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b767c7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680dff90 con 0x7f4b6804a0d0 2021-01-06T22:03:23.112 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b767c7700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6802fd60 con 0x7f4b68044ff0 2021-01-06T22:03:23.112 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.106+0000 7f4b74fc4700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b68044ff0 0x7f4b680f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.112 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b757c5700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b68035c80 0x7f4b68049be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.113 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b74fc4700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b68044ff0 0x7f4b680f0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.91:52818/0 (socket says 172.21.15.91:52818) 2021-01-06T22:03:23.113 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b74fc4700 1 -- 172.21.15.91:0/2371063648 learned_addr learned my addr 172.21.15.91:0/2371063648 (peer_addr_for_me v2:172.21.15.91:0/0) 2021-01-06T22:03:23.113 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b75fc6700 1 --2- 172.21.15.91:0/2371063648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b6804a0d0 0x7f4b680f0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.113 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b74fc4700 1 -- 172.21.15.91:0/2371063648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b6804a0d0 msgr2=0x7f4b680f0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.114 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b74fc4700 1 --2- 172.21.15.91:0/2371063648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b6804a0d0 0x7f4b680f0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.114 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b74fc4700 1 -- 172.21.15.91:0/2371063648 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b68035c80 msgr2=0x7f4b68049be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.114 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b74fc4700 1 --2- 172.21.15.91:0/2371063648 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b68035c80 0x7f4b68049be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.114 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b74fc4700 1 -- 172.21.15.91:0/2371063648 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b68030000 con 0x7f4b68044ff0 2021-01-06T22:03:23.114 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b75fc6700 1 --2- 172.21.15.91:0/2371063648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b6804a0d0 0x7f4b680f0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:23.115 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b74fc4700 1 --2- 172.21.15.91:0/2371063648 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b68044ff0 0x7f4b680f0490 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f4b6804dae0 tx=0x7f4b64016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:23.115 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b61ffb700 1 -- 172.21.15.91:0/2371063648 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64012e10 con 0x7f4b68044ff0 2021-01-06T22:03:23.115 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b61ffb700 1 -- 172.21.15.91:0/2371063648 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6401c420 con 0x7f4b68044ff0 2021-01-06T22:03:23.115 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b767c7700 1 -- 172.21.15.91:0/2371063648 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b68049740 con 0x7f4b68044ff0 2021-01-06T22:03:23.116 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b61ffb700 1 -- 172.21.15.91:0/2371063648 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6401c7c0 con 0x7f4b68044ff0 2021-01-06T22:03:23.116 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b767c7700 1 -- 172.21.15.91:0/2371063648 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b680302a0 con 0x7f4b68044ff0 2021-01-06T22:03:23.116 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b61ffb700 1 -- 172.21.15.91:0/2371063648 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f4b6400f070 con 0x7f4b68044ff0 2021-01-06T22:03:23.116 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b61ffb700 1 -- 172.21.15.91:0/2371063648 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f4b64026960 con 0x7f4b68044ff0 2021-01-06T22:03:23.117 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.110+0000 7f4b767c7700 1 -- 172.21.15.91:0/2371063648 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b68049920 con 0x7f4b68044ff0 2021-01-06T22:03:23.117 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.114+0000 7f4b61ffb700 1 -- 172.21.15.91:0/2371063648 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71894 (secure 0 0 0) 0x7f4b68049920 con 0x7f4b68044ff0 2021-01-06T22:03:23.191 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.194+0000 7f4b767c7700 1 -- 172.21.15.91:0/2371063648 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e0b35c09-147c-4ccc-b566-c48b81e25412", "id": 0} v 0) v1 -- 0x7f4b68005440 con 0x7f4b68044ff0 2021-01-06T22:03:23.204 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.206+0000 7f4b61ffb700 1 -- 172.21.15.91:0/2371063648 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e0b35c09-147c-4ccc-b566-c48b81e25412", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4b64016070 con 0x7f4b68044ff0 2021-01-06T22:03:23.204 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.206+0000 7f4b767c7700 1 -- 172.21.15.91:0/2371063648 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b68044ff0 msgr2=0x7f4b680f0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.204 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.206+0000 7f4b767c7700 1 --2- 172.21.15.91:0/2371063648 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b68044ff0 0x7f4b680f0490 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f4b6804dae0 tx=0x7f4b64016000).stop 2021-01-06T22:03:23.204 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.206+0000 7f4b767c7700 1 -- 172.21.15.91:0/2371063648 shutdown_connections 2021-01-06T22:03:23.205 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.206+0000 7f4b767c7700 1 --2- 172.21.15.91:0/2371063648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b6804a0d0 0x7f4b680f0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.206 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.206+0000 7f4b767c7700 1 --2- 172.21.15.91:0/2371063648 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b68035c80 0x7f4b68049be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.206 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.206+0000 7f4b767c7700 1 --2- 172.21.15.91:0/2371063648 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b68044ff0 0x7f4b680f0490 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.206 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.206+0000 7f4b767c7700 1 -- 172.21.15.91:0/2371063648 >> 172.21.15.91:0/2371063648 conn(0x7f4b6800cb20 msgr2=0x7f4b6804eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:23.206 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.206+0000 7f4b767c7700 1 -- 172.21.15.91:0/2371063648 shutdown_connections 2021-01-06T22:03:23.207 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.206+0000 7f4b767c7700 1 -- 172.21.15.91:0/2371063648 wait complete. 2021-01-06T22:03:23.208 INFO:teuthology.orchestra.run.smithi091.stdout:0 2021-01-06T22:03:23.214 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd new 8add59a0-2d58-4c1b-a8d1-62049ee9d85f 1 2021-01-06T22:03:23.332 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.334+0000 7fc008929700 1 Processor -- start 2021-01-06T22:03:23.333 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.334+0000 7fc008929700 1 -- start start 2021-01-06T22:03:23.337 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.334+0000 7fc008929700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc004064380 0x7fc0040c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.337 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.334+0000 7fc008929700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc0041478c0 0x7fc0040bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.337 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.334+0000 7fc008929700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0040bb2d0 con 0x7fc00414e310 2021-01-06T22:03:23.338 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.334+0000 7fc008929700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00414a560 con 0x7fc0041478c0 2021-01-06T22:03:23.338 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.334+0000 7fc008929700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00414a800 con 0x7fc004064380 2021-01-06T22:03:23.338 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.334+0000 7fc00259c700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc00414e310 0x7fc0041471b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52182/0 (socket says 172.21.15.91:52182) 2021-01-06T22:03:23.338 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc00259c700 1 -- 172.21.15.91:0/3057420902 learned_addr learned my addr 172.21.15.91:0/3057420902 (peer_addr_for_me v1:172.21.15.91:0/0) 2021-01-06T22:03:23.339 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc00359e700 1 --2- 172.21.15.91:0/3057420902 >> v2:172.21.15.91:3300/0 conn(0x7fc0041478c0 0x7fc0040bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.339 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc002d9d700 1 --2- 172.21.15.91:0/3057420902 >> v2:172.21.15.203:3300/0 conn(0x7fc004064380 0x7fc0040c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.339 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc001d9b700 1 -- 172.21.15.91:0/3057420902 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2275574056 0 0) 0x7fc0040bb2d0 con 0x7fc00414e310 2021-01-06T22:03:23.339 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc001d9b700 1 -- 172.21.15.91:0/3057420902 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfe4003230 con 0x7fc00414e310 2021-01-06T22:03:23.340 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc00359e700 1 -- 172.21.15.91:0/3057420902 >> v2:172.21.15.203:3300/0 conn(0x7fc004064380 msgr2=0x7fc0040c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:23.340 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc00359e700 1 --2- 172.21.15.91:0/3057420902 >> v2:172.21.15.203:3300/0 conn(0x7fc004064380 0x7fc0040c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.340 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc00359e700 1 -- 172.21.15.91:0/3057420902 >> v1:172.21.15.91:6789/0 conn(0x7fc00414e310 legacy=0x7fc0041471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.340 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc00359e700 1 -- 172.21.15.91:0/3057420902 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00414aaa0 con 0x7fc0041478c0 2021-01-06T22:03:23.341 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc00359e700 1 --2- 172.21.15.91:0/3057420902 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0041478c0 0x7fc0040bb730 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fc0040c07d0 tx=0x7fbfec006de0).ready entity=mon.0 client_cookie=904bcc387f6e5ea server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:23.341 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc001d9b700 1 -- 172.21.15.91:0/3057420902 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0096d0 con 0x7fc0041478c0 2021-01-06T22:03:23.341 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc001d9b700 1 -- 172.21.15.91:0/3057420902 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec009830 con 0x7fc0041478c0 2021-01-06T22:03:23.341 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc001d9b700 1 -- 172.21.15.91:0/3057420902 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec009bb0 con 0x7fc0041478c0 2021-01-06T22:03:23.342 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc008929700 1 -- 172.21.15.91:0/3057420902 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0041478c0 msgr2=0x7fc0040bb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.342 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc008929700 1 --2- 172.21.15.91:0/3057420902 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0041478c0 0x7fc0040bb730 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fc0040c07d0 tx=0x7fbfec006de0).stop 2021-01-06T22:03:23.342 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc008929700 1 -- 172.21.15.91:0/3057420902 shutdown_connections 2021-01-06T22:03:23.343 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc008929700 1 --2- 172.21.15.91:0/3057420902 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0041478c0 0x7fc0040bb730 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.343 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc008929700 1 --2- 172.21.15.91:0/3057420902 >> v2:172.21.15.203:3300/0 conn(0x7fc004064380 0x7fc0040c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.343 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc008929700 1 -- 172.21.15.91:0/3057420902 >> 172.21.15.91:0/3057420902 conn(0x7fc0040667b0 msgr2=0x7fc0040c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:23.343 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc008929700 1 -- 172.21.15.91:0/3057420902 shutdown_connections 2021-01-06T22:03:23.344 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc008929700 1 -- 172.21.15.91:0/3057420902 wait complete. 2021-01-06T22:03:23.344 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc008929700 1 Processor -- start 2021-01-06T22:03:23.344 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.338+0000 7fc008929700 1 -- start start 2021-01-06T22:03:23.347 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc008929700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc004064380 0x7fc0040b92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.347 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc008929700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0041478c0 0x7fc00415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.347 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc008929700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc00414e310 0x7fc004146140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.348 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc008929700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc004146ba0 con 0x7fc0041478c0 2021-01-06T22:03:23.348 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc008929700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00415e200 con 0x7fc004064380 2021-01-06T22:03:23.348 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc008929700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040b9910 con 0x7fc00414e310 2021-01-06T22:03:23.349 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc00259c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0041478c0 0x7fc00415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.349 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc00359e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc00414e310 0x7fc004146140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.349 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc00359e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc00414e310 0x7fc004146140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.91:52832/0 (socket says 172.21.15.91:52832) 2021-01-06T22:03:23.349 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc00359e700 1 -- 172.21.15.91:0/3118794492 learned_addr learned my addr 172.21.15.91:0/3118794492 (peer_addr_for_me v2:172.21.15.91:0/0) 2021-01-06T22:03:23.350 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc00259c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0041478c0 0x7fc00415dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36332/0 (socket says 172.21.15.91:36332) 2021-01-06T22:03:23.350 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc002d9d700 1 --2- 172.21.15.91:0/3118794492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc004064380 0x7fc0040b92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.350 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc00259c700 1 -- 172.21.15.91:0/3118794492 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc00414e310 msgr2=0x7fc004146140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.350 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc00259c700 1 --2- 172.21.15.91:0/3118794492 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc00414e310 0x7fc004146140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.351 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc00259c700 1 -- 172.21.15.91:0/3118794492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc004064380 msgr2=0x7fc0040b92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.351 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc00259c700 1 --2- 172.21.15.91:0/3118794492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc004064380 0x7fc0040b92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.351 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc00259c700 1 -- 172.21.15.91:0/3118794492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0040b9bb0 con 0x7fc0041478c0 2021-01-06T22:03:23.351 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc00359e700 1 --2- 172.21.15.91:0/3118794492 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc00414e310 0x7fc004146140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:23.352 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc00259c700 1 --2- 172.21.15.91:0/3118794492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0041478c0 0x7fc00415dd10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fc00414e730 tx=0x7fbff4007a10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:23.352 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fbffaffd700 1 -- 172.21.15.91:0/3118794492 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff40072e0 con 0x7fc0041478c0 2021-01-06T22:03:23.352 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fbffaffd700 1 -- 172.21.15.91:0/3118794492 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff4007440 con 0x7fc0041478c0 2021-01-06T22:03:23.352 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc008929700 1 -- 172.21.15.91:0/3118794492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0040b9e50 con 0x7fc0041478c0 2021-01-06T22:03:23.352 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fbffaffd700 1 -- 172.21.15.91:0/3118794492 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4005810 con 0x7fc0041478c0 2021-01-06T22:03:23.353 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc008929700 1 -- 172.21.15.91:0/3118794492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00413b460 con 0x7fc0041478c0 2021-01-06T22:03:23.353 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fbffaffd700 1 -- 172.21.15.91:0/3118794492 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fbff401a060 con 0x7fc0041478c0 2021-01-06T22:03:23.353 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fbffaffd700 1 -- 172.21.15.91:0/3118794492 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7fbff4013770 con 0x7fc0041478c0 2021-01-06T22:03:23.353 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.342+0000 7fc008929700 1 -- 172.21.15.91:0/3118794492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0040450a0 con 0x7fc0041478c0 2021-01-06T22:03:23.354 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.346+0000 7fbffaffd700 1 -- 172.21.15.91:0/3118794492 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71894 (secure 0 0 0) 0x7fbff4018070 con 0x7fc0041478c0 2021-01-06T22:03:23.426 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.430+0000 7fc008929700 1 -- 172.21.15.91:0/3118794492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8add59a0-2d58-4c1b-a8d1-62049ee9d85f", "id": 1} v 0) v1 -- 0x7fc004061f70 con 0x7fc0041478c0 2021-01-06T22:03:23.436 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.438+0000 7fbffaffd700 1 -- 172.21.15.91:0/3118794492 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8add59a0-2d58-4c1b-a8d1-62049ee9d85f", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbff40139b0 con 0x7fc0041478c0 2021-01-06T22:03:23.436 INFO:teuthology.orchestra.run.smithi091.stdout:1 2021-01-06T22:03:23.437 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.438+0000 7fc008929700 1 -- 172.21.15.91:0/3118794492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0041478c0 msgr2=0x7fc00415dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.438 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.438+0000 7fc008929700 1 --2- 172.21.15.91:0/3118794492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0041478c0 0x7fc00415dd10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fc00414e730 tx=0x7fbff4007a10).stop 2021-01-06T22:03:23.438 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.438+0000 7fc008929700 1 -- 172.21.15.91:0/3118794492 shutdown_connections 2021-01-06T22:03:23.438 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.438+0000 7fc008929700 1 --2- 172.21.15.91:0/3118794492 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc00414e310 0x7fc004146140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.438 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.438+0000 7fc008929700 1 --2- 172.21.15.91:0/3118794492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc004064380 0x7fc0040b92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.438 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.438+0000 7fc008929700 1 --2- 172.21.15.91:0/3118794492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0041478c0 0x7fc00415dd10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.439 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.438+0000 7fc008929700 1 -- 172.21.15.91:0/3118794492 >> 172.21.15.91:0/3118794492 conn(0x7fc0040667b0 msgr2=0x7fc0040bfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:23.439 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.438+0000 7fc008929700 1 -- 172.21.15.91:0/3118794492 shutdown_connections 2021-01-06T22:03:23.440 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.438+0000 7fc008929700 1 -- 172.21.15.91:0/3118794492 wait complete. 2021-01-06T22:03:23.452 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd new b294a411-c7c9-477e-8eae-3b7bd0563ed3 2 2021-01-06T22:03:23.573 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.574+0000 7f7331ae3700 1 Processor -- start 2021-01-06T22:03:23.573 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.574+0000 7f7331ae3700 1 -- start start 2021-01-06T22:03:23.575 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.574+0000 7f7331ae3700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f732c0ba440 0x7f732c0b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.575 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.574+0000 7f7331ae3700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f732c0ba820 0x7f732c14c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.575 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.574+0000 7f7331ae3700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f732c061600 0x7f732c14c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.576 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.574+0000 7f7331ae3700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732c14cbb0 con 0x7f732c0ba820 2021-01-06T22:03:23.576 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.574+0000 7f7331ae3700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732c0bc690 con 0x7f732c061600 2021-01-06T22:03:23.576 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.574+0000 7f7331ae3700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c0bc8c0 con 0x7f732c0ba440 2021-01-06T22:03:23.576 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7323fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f732c0ba820 0x7f732c14c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.576 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7323fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f732c0ba820 0x7f732c14c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36338/0 (socket says 172.21.15.91:36338) 2021-01-06T22:03:23.577 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7323fff700 1 -- 172.21.15.91:0/343662881 learned_addr learned my addr 172.21.15.91:0/343662881 (peer_addr_for_me v2:172.21.15.91:0/0) 2021-01-06T22:03:23.577 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f732b7fe700 1 --2- 172.21.15.91:0/343662881 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f732c0ba440 0x7f732c0b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.577 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f732bfff700 1 --2- 172.21.15.91:0/343662881 >> v2:172.21.15.203:3300/0 conn(0x7f732c061600 0x7f732c14c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.577 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7323fff700 1 -- 172.21.15.91:0/343662881 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f732c0ba440 msgr2=0x7f732c0b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:23.578 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7323fff700 1 --2- 172.21.15.91:0/343662881 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f732c0ba440 0x7f732c0b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.578 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7323fff700 1 -- 172.21.15.91:0/343662881 >> v2:172.21.15.203:3300/0 conn(0x7f732c061600 msgr2=0x7f732c14c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:23.578 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7323fff700 1 --2- 172.21.15.91:0/343662881 >> v2:172.21.15.203:3300/0 conn(0x7f732c061600 0x7f732c14c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.578 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7323fff700 1 -- 172.21.15.91:0/343662881 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c0bcb60 con 0x7f732c0ba820 2021-01-06T22:03:23.578 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f732bfff700 1 --2- 172.21.15.91:0/343662881 >> v2:172.21.15.203:3300/0 conn(0x7f732c061600 0x7f732c14c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:23.579 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7323fff700 1 --2- 172.21.15.91:0/343662881 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f732c0ba820 0x7f732c14c1d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f732c060b20 tx=0x7f7314006ea0).ready entity=mon.0 client_cookie=ddb2b4808809628a server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:23.579 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f732affd700 1 -- 172.21.15.91:0/343662881 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314009860 con 0x7f732c0ba820 2021-01-06T22:03:23.579 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f732affd700 1 -- 172.21.15.91:0/343662881 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73140099c0 con 0x7f732c0ba820 2021-01-06T22:03:23.579 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f732affd700 1 -- 172.21.15.91:0/343662881 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314009d40 con 0x7f732c0ba820 2021-01-06T22:03:23.583 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7331ae3700 1 -- 172.21.15.91:0/343662881 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f732c0ba820 msgr2=0x7f732c14c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.583 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7331ae3700 1 --2- 172.21.15.91:0/343662881 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f732c0ba820 0x7f732c14c1d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f732c060b20 tx=0x7f7314006ea0).stop 2021-01-06T22:03:23.583 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7331ae3700 1 -- 172.21.15.91:0/343662881 shutdown_connections 2021-01-06T22:03:23.584 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7331ae3700 1 --2- 172.21.15.91:0/343662881 >> v2:172.21.15.203:3300/0 conn(0x7f732c061600 0x7f732c14c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.584 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7331ae3700 1 --2- 172.21.15.91:0/343662881 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f732c0ba440 0x7f732c0b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.584 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7331ae3700 1 --2- 172.21.15.91:0/343662881 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f732c0ba820 0x7f732c14c1d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.584 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7331ae3700 1 -- 172.21.15.91:0/343662881 >> 172.21.15.91:0/343662881 conn(0x7f732c0c0c50 msgr2=0x7f732c0c1030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:23.585 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7331ae3700 1 -- 172.21.15.91:0/343662881 shutdown_connections 2021-01-06T22:03:23.585 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7331ae3700 1 -- 172.21.15.91:0/343662881 wait complete. 2021-01-06T22:03:23.585 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7331ae3700 1 Processor -- start 2021-01-06T22:03:23.585 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.578+0000 7f7331ae3700 1 -- start start 2021-01-06T22:03:23.586 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f7331ae3700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f732c061600 0x7f732c14f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.586 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f7331ae3700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f732c0ba440 0x7f732c0bfe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.586 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f7331ae3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f732c0ba820 0x7f732c15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.586 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f7331ae3700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c14e920 con 0x7f732c061600 2021-01-06T22:03:23.587 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f7331ae3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c142ee0 con 0x7f732c0ba820 2021-01-06T22:03:23.587 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f7331ae3700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c143140 con 0x7f732c0ba440 2021-01-06T22:03:23.587 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f7323fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f732c0ba440 0x7f732c0bfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.587 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f732b7fe700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f732c061600 0x7f732c14f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.588 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f732b7fe700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f732c061600 0x7f732c14f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36346/0 (socket says 172.21.15.91:36346) 2021-01-06T22:03:23.588 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f732b7fe700 1 -- 172.21.15.91:0/2461986404 learned_addr learned my addr 172.21.15.91:0/2461986404 (peer_addr_for_me v2:172.21.15.91:0/0) 2021-01-06T22:03:23.588 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f7323fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f732c0ba440 0x7f732c0bfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.91:52840/0 (socket says 172.21.15.91:52840) 2021-01-06T22:03:23.588 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f732bfff700 1 --2- 172.21.15.91:0/2461986404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f732c0ba820 0x7f732c15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.588 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f7323fff700 1 -- 172.21.15.91:0/2461986404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f732c0ba820 msgr2=0x7f732c15f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f7323fff700 1 --2- 172.21.15.91:0/2461986404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f732c0ba820 0x7f732c15f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f7323fff700 1 -- 172.21.15.91:0/2461986404 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f732c061600 msgr2=0x7f732c14f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f7323fff700 1 --2- 172.21.15.91:0/2461986404 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f732c061600 0x7f732c14f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f7323fff700 1 -- 172.21.15.91:0/2461986404 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c14eb50 con 0x7f732c0ba440 2021-01-06T22:03:23.590 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f732b7fe700 1 --2- 172.21.15.91:0/2461986404 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f732c061600 0x7f732c14f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:23.590 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f732bfff700 1 --2- 172.21.15.91:0/2461986404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f732c0ba820 0x7f732c15f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:23.590 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f7323fff700 1 --2- 172.21.15.91:0/2461986404 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f732c0ba440 0x7f732c0bfe70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f732c0b7d60 tx=0x7f7314016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:23.590 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f73237fe700 1 -- 172.21.15.91:0/2461986404 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314020070 con 0x7f732c0ba440 2021-01-06T22:03:23.591 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f73237fe700 1 -- 172.21.15.91:0/2461986404 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7314012d90 con 0x7f732c0ba440 2021-01-06T22:03:23.591 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f7331ae3700 1 -- 172.21.15.91:0/2461986404 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732c14edf0 con 0x7f732c0ba440 2021-01-06T22:03:23.591 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f73237fe700 1 -- 172.21.15.91:0/2461986404 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731401c670 con 0x7f732c0ba440 2021-01-06T22:03:23.591 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f7331ae3700 1 -- 172.21.15.91:0/2461986404 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732c146780 con 0x7f732c0ba440 2021-01-06T22:03:23.591 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f73237fe700 1 -- 172.21.15.91:0/2461986404 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f731400f070 con 0x7f732c0ba440 2021-01-06T22:03:23.592 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f73237fe700 1 -- 172.21.15.91:0/2461986404 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7f731401cd00 con 0x7f732c0ba440 2021-01-06T22:03:23.592 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.582+0000 7f73217fa700 1 -- 172.21.15.91:0/2461986404 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72fc004f60 con 0x7f732c0ba440 2021-01-06T22:03:23.592 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.586+0000 7f73237fe700 1 -- 172.21.15.91:0/2461986404 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71894 (secure 0 0 0) 0x7f731402b060 con 0x7f732c0ba440 2021-01-06T22:03:23.668 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.670+0000 7f73217fa700 1 -- 172.21.15.91:0/2461986404 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b294a411-c7c9-477e-8eae-3b7bd0563ed3", "id": 2} v 0) v1 -- 0x7f72fc0052a0 con 0x7f732c0ba440 2021-01-06T22:03:23.678 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.678+0000 7f73237fe700 1 -- 172.21.15.91:0/2461986404 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b294a411-c7c9-477e-8eae-3b7bd0563ed3", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7314016070 con 0x7f732c0ba440 2021-01-06T22:03:23.678 INFO:teuthology.orchestra.run.smithi091.stdout:2 2021-01-06T22:03:23.680 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.682+0000 7f73217fa700 1 -- 172.21.15.91:0/2461986404 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f732c0ba440 msgr2=0x7f732c0bfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.680 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.682+0000 7f73217fa700 1 --2- 172.21.15.91:0/2461986404 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f732c0ba440 0x7f732c0bfe70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f732c0b7d60 tx=0x7f7314016000).stop 2021-01-06T22:03:23.680 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.682+0000 7f73217fa700 1 -- 172.21.15.91:0/2461986404 shutdown_connections 2021-01-06T22:03:23.681 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.682+0000 7f73217fa700 1 --2- 172.21.15.91:0/2461986404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f732c0ba820 0x7f732c15f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.681 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.682+0000 7f73217fa700 1 --2- 172.21.15.91:0/2461986404 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f732c061600 0x7f732c14f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.681 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.682+0000 7f73217fa700 1 --2- 172.21.15.91:0/2461986404 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f732c0ba440 0x7f732c0bfe70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.681 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.682+0000 7f73217fa700 1 -- 172.21.15.91:0/2461986404 >> 172.21.15.91:0/2461986404 conn(0x7f732c0c0c50 msgr2=0x7f732c0c1220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:23.682 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.682+0000 7f73217fa700 1 -- 172.21.15.91:0/2461986404 shutdown_connections 2021-01-06T22:03:23.682 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.682+0000 7f73217fa700 1 -- 172.21.15.91:0/2461986404 wait complete. 2021-01-06T22:03:23.693 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd new 5878a8f7-1a42-41c3-9810-0ac41a54dd6f 3 2021-01-06T22:03:23.815 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c95a3e700 1 Processor -- start 2021-01-06T22:03:23.815 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c95a3e700 1 -- start start 2021-01-06T22:03:23.816 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c95a3e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c900c03f0 0x7f3c900aa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.816 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c95a3e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3c900c13c0 0x7f3c90150110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.817 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c95a3e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c900a9ca0 con 0x7f3c900c0fe0 2021-01-06T22:03:23.817 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c95a3e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c90066a50 con 0x7f3c900c13c0 2021-01-06T22:03:23.817 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c95a3e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c90066cf0 con 0x7f3c900c03f0 2021-01-06T22:03:23.817 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c94a3c700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f3c900c0fe0 0x7f3c90060d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52204/0 (socket says 172.21.15.91:52204) 2021-01-06T22:03:23.817 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c94a3c700 1 -- 172.21.15.91:0/2643218945 learned_addr learned my addr 172.21.15.91:0/2643218945 (peer_addr_for_me v1:172.21.15.91:0/0) 2021-01-06T22:03:23.818 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c9523d700 1 --2- 172.21.15.91:0/2643218945 >> v2:172.21.15.91:3300/0 conn(0x7f3c900c13c0 0x7f3c90150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.818 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c8ffff700 1 --2- 172.21.15.91:0/2643218945 >> v2:172.21.15.203:3300/0 conn(0x7f3c900c03f0 0x7f3c900aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.821 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c9523d700 1 -- 172.21.15.91:0/2643218945 >> v2:172.21.15.203:3300/0 conn(0x7f3c900c03f0 msgr2=0x7f3c900aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:23.822 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c9523d700 1 --2- 172.21.15.91:0/2643218945 >> v2:172.21.15.203:3300/0 conn(0x7f3c900c03f0 0x7f3c900aa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.822 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c9523d700 1 -- 172.21.15.91:0/2643218945 >> v1:172.21.15.91:6789/0 conn(0x7f3c900c0fe0 legacy=0x7f3c90060d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.822 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c8f7fe700 1 -- 172.21.15.91:0/2643218945 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481221455 0 0) 0x7f3c900a9ca0 con 0x7f3c900c0fe0 2021-01-06T22:03:23.822 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c9523d700 1 -- 172.21.15.91:0/2643218945 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c90068460 con 0x7f3c900c13c0 2021-01-06T22:03:23.823 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c9523d700 1 --2- 172.21.15.91:0/2643218945 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c900c13c0 0x7f3c90150110 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f3c90057dc0 tx=0x7f3c78006de0).ready entity=mon.0 client_cookie=532d448db52bd6f7 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:23.823 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c8f7fe700 1 -- 172.21.15.91:0/2643218945 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c780097b0 con 0x7f3c900c13c0 2021-01-06T22:03:23.823 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c8f7fe700 1 -- 172.21.15.91:0/2643218945 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c78009910 con 0x7f3c900c13c0 2021-01-06T22:03:23.823 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c8f7fe700 1 -- 172.21.15.91:0/2643218945 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78009c90 con 0x7f3c900c13c0 2021-01-06T22:03:23.823 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c95a3e700 1 -- 172.21.15.91:0/2643218945 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c900c13c0 msgr2=0x7f3c90150110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.824 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c95a3e700 1 --2- 172.21.15.91:0/2643218945 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c900c13c0 0x7f3c90150110 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f3c90057dc0 tx=0x7f3c78006de0).stop 2021-01-06T22:03:23.824 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c95a3e700 1 -- 172.21.15.91:0/2643218945 shutdown_connections 2021-01-06T22:03:23.824 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c95a3e700 1 --2- 172.21.15.91:0/2643218945 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c900c13c0 0x7f3c90150110 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.824 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c95a3e700 1 --2- 172.21.15.91:0/2643218945 >> v2:172.21.15.203:3300/0 conn(0x7f3c900c03f0 0x7f3c900aa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.825 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.818+0000 7f3c95a3e700 1 -- 172.21.15.91:0/2643218945 >> 172.21.15.91:0/2643218945 conn(0x7f3c90061530 msgr2=0x7f3c90061910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:23.825 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c95a3e700 1 -- 172.21.15.91:0/2643218945 shutdown_connections 2021-01-06T22:03:23.825 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c95a3e700 1 -- 172.21.15.91:0/2643218945 wait complete. 2021-01-06T22:03:23.825 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c95a3e700 1 Processor -- start 2021-01-06T22:03:23.825 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c95a3e700 1 -- start start 2021-01-06T22:03:23.826 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c95a3e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c900c03f0 0x7f3c9014d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.826 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c95a3e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c900c0fe0 0x7f3c90161590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.826 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c95a3e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c900c13c0 0x7f3c900b9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:23.826 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c95a3e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900a28d0 con 0x7f3c900c03f0 2021-01-06T22:03:23.826 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c95a3e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900a2b70 con 0x7f3c900c0fe0 2021-01-06T22:03:23.827 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c95a3e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900a2e10 con 0x7f3c900c13c0 2021-01-06T22:03:23.827 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c94a3c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c900c03f0 0x7f3c9014d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.827 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c9523d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c900c13c0 0x7f3c900b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.827 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c94a3c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c900c03f0 0x7f3c9014d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36356/0 (socket says 172.21.15.91:36356) 2021-01-06T22:03:23.827 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c94a3c700 1 -- 172.21.15.91:0/648378667 learned_addr learned my addr 172.21.15.91:0/648378667 (peer_addr_for_me v2:172.21.15.91:0/0) 2021-01-06T22:03:23.828 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c8ffff700 1 --2- 172.21.15.91:0/648378667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c900c0fe0 0x7f3c90161590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:23.828 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c94a3c700 1 -- 172.21.15.91:0/648378667 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c900c13c0 msgr2=0x7f3c900b9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.828 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c94a3c700 1 --2- 172.21.15.91:0/648378667 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c900c13c0 0x7f3c900b9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.828 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c94a3c700 1 -- 172.21.15.91:0/648378667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c900c0fe0 msgr2=0x7f3c90161590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.828 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c94a3c700 1 --2- 172.21.15.91:0/648378667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c900c0fe0 0x7f3c90161590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.829 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c94a3c700 1 -- 172.21.15.91:0/648378667 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c90160bb0 con 0x7f3c900c03f0 2021-01-06T22:03:23.829 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c9523d700 1 --2- 172.21.15.91:0/648378667 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c900c13c0 0x7f3c900b9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:23.829 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c94a3c700 1 --2- 172.21.15.91:0/648378667 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c900c03f0 0x7f3c9014d530 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f3c8800cae0 tx=0x7f3c880013b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:23.829 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c8cff9700 1 -- 172.21.15.91:0/648378667 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c88005b70 con 0x7f3c900c03f0 2021-01-06T22:03:23.830 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c8cff9700 1 -- 172.21.15.91:0/648378667 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c88005cd0 con 0x7f3c900c03f0 2021-01-06T22:03:23.830 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c95a3e700 1 -- 172.21.15.91:0/648378667 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c90160e50 con 0x7f3c900c03f0 2021-01-06T22:03:23.830 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c8cff9700 1 -- 172.21.15.91:0/648378667 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c88007320 con 0x7f3c900c03f0 2021-01-06T22:03:23.830 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.822+0000 7f3c95a3e700 1 -- 172.21.15.91:0/648378667 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c900be520 con 0x7f3c900c03f0 2021-01-06T22:03:23.831 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.826+0000 7f3c8cff9700 1 -- 172.21.15.91:0/648378667 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f3c88010030 con 0x7f3c900c03f0 2021-01-06T22:03:23.831 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.826+0000 7f3c8cff9700 1 -- 172.21.15.91:0/648378667 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7f3c88005e40 con 0x7f3c900c03f0 2021-01-06T22:03:23.831 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.826+0000 7f3c95a3e700 1 -- 172.21.15.91:0/648378667 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c900450a0 con 0x7f3c900c03f0 2021-01-06T22:03:23.832 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.830+0000 7f3c8cff9700 1 -- 172.21.15.91:0/648378667 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71894 (secure 0 0 0) 0x7f3c8800a970 con 0x7f3c900c03f0 2021-01-06T22:03:23.908 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.910+0000 7f3c95a3e700 1 -- 172.21.15.91:0/648378667 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5878a8f7-1a42-41c3-9810-0ac41a54dd6f", "id": 3} v 0) v1 -- 0x7f3c900679c0 con 0x7f3c900c03f0 2021-01-06T22:03:23.912 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.914+0000 7f3c8cff9700 1 -- 172.21.15.91:0/648378667 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5878a8f7-1a42-41c3-9810-0ac41a54dd6f", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3c8800abd0 con 0x7f3c900c03f0 2021-01-06T22:03:23.912 INFO:teuthology.orchestra.run.smithi091.stdout:3 2021-01-06T22:03:23.913 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.914+0000 7f3c95a3e700 1 -- 172.21.15.91:0/648378667 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c900c03f0 msgr2=0x7f3c9014d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:23.914 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.914+0000 7f3c95a3e700 1 --2- 172.21.15.91:0/648378667 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c900c03f0 0x7f3c9014d530 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f3c8800cae0 tx=0x7f3c880013b0).stop 2021-01-06T22:03:23.914 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.914+0000 7f3c95a3e700 1 -- 172.21.15.91:0/648378667 shutdown_connections 2021-01-06T22:03:23.914 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.914+0000 7f3c95a3e700 1 --2- 172.21.15.91:0/648378667 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c900c13c0 0x7f3c900b9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.915 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.914+0000 7f3c95a3e700 1 --2- 172.21.15.91:0/648378667 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c900c03f0 0x7f3c9014d530 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.915 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.914+0000 7f3c95a3e700 1 --2- 172.21.15.91:0/648378667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c900c0fe0 0x7f3c90161590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:23.915 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.914+0000 7f3c95a3e700 1 -- 172.21.15.91:0/648378667 >> 172.21.15.91:0/648378667 conn(0x7f3c90061530 msgr2=0x7f3c901511c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:23.915 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.914+0000 7f3c95a3e700 1 -- 172.21.15.91:0/648378667 shutdown_connections 2021-01-06T22:03:23.916 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:23.914+0000 7f3c95a3e700 1 -- 172.21.15.91:0/648378667 wait complete. 2021-01-06T22:03:23.926 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd new f641a345-81e4-4400-a44c-d2f82fef7f6f 4 2021-01-06T22:03:24.040 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.042+0000 7fa2f2d13700 1 Processor -- start 2021-01-06T22:03:24.040 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.042+0000 7fa2f2d13700 1 -- start start 2021-01-06T22:03:24.041 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.042+0000 7fa2f2d13700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec0c1430 0x7fa2ec148140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:24.042 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.042+0000 7fa2f2d13700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ec0bb7e0 con 0x7fa2ec0bb230 2021-01-06T22:03:24.042 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.042+0000 7fa2f2d13700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ec0bba80 con 0x7fa2ec147d60 2021-01-06T22:03:24.042 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.042+0000 7fa2f2d13700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec0bbd20 con 0x7fa2ec0c1430 2021-01-06T22:03:24.042 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.042+0000 7fa2f2512700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fa2ec0bb230 0x7fa2ec0602b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52218/0 (socket says 172.21.15.91:52218) 2021-01-06T22:03:24.043 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.042+0000 7fa2f2512700 1 -- 172.21.15.91:0/2306557598 learned_addr learned my addr 172.21.15.91:0/2306557598 (peer_addr_for_me v1:172.21.15.91:0/0) 2021-01-06T22:03:24.043 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.042+0000 7fa2f1510700 1 --2- 172.21.15.91:0/2306557598 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec0c1430 0x7fa2ec148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:24.045 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f0d0f700 1 -- 172.21.15.91:0/2306557598 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992170164 0 0) 0x7fa2ec0bb7e0 con 0x7fa2ec0bb230 2021-01-06T22:03:24.045 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f0d0f700 1 -- 172.21.15.91:0/2306557598 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2d4003230 con 0x7fa2ec0bb230 2021-01-06T22:03:24.045 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f0d0f700 1 -- 172.21.15.91:0/2306557598 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624274080 0 0) 0x7fa2ec0bba80 con 0x7fa2ec147d60 2021-01-06T22:03:24.045 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f1510700 1 -- 172.21.15.91:0/2306557598 >> v1:172.21.15.203:6789/0 conn(0x7fa2ec147d60 legacy=0x7fa2ec0671a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:24.046 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f1510700 1 -- 172.21.15.91:0/2306557598 >> v1:172.21.15.91:6789/0 conn(0x7fa2ec0bb230 legacy=0x7fa2ec0602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:24.046 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f1510700 1 -- 172.21.15.91:0/2306557598 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec0bbfc0 con 0x7fa2ec0c1430 2021-01-06T22:03:24.046 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f1510700 1 --2- 172.21.15.91:0/2306557598 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec0c1430 0x7fa2ec148140 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fa2dc00b9b0 tx=0x7fa2dc006de0).ready entity=mon.2 client_cookie=a38ef5eeef2ed766 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:24.046 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f0d0f700 1 -- 172.21.15.91:0/2306557598 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc0097b0 con 0x7fa2ec0c1430 2021-01-06T22:03:24.046 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f0d0f700 1 -- 172.21.15.91:0/2306557598 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2dc009910 con 0x7fa2ec0c1430 2021-01-06T22:03:24.047 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f0d0f700 1 -- 172.21.15.91:0/2306557598 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc009c90 con 0x7fa2ec0c1430 2021-01-06T22:03:24.047 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f2d13700 1 -- 172.21.15.91:0/2306557598 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec0c1430 msgr2=0x7fa2ec148140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:24.047 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f2d13700 1 --2- 172.21.15.91:0/2306557598 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec0c1430 0x7fa2ec148140 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fa2dc00b9b0 tx=0x7fa2dc006de0).stop 2021-01-06T22:03:24.047 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f2d13700 1 -- 172.21.15.91:0/2306557598 shutdown_connections 2021-01-06T22:03:24.047 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f2d13700 1 --2- 172.21.15.91:0/2306557598 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec0c1430 0x7fa2ec148140 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:24.048 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f2d13700 1 -- 172.21.15.91:0/2306557598 >> 172.21.15.91:0/2306557598 conn(0x7fa2ec0667b0 msgr2=0x7fa2ec0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:24.048 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f2d13700 1 -- 172.21.15.91:0/2306557598 shutdown_connections 2021-01-06T22:03:24.048 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f2d13700 1 -- 172.21.15.91:0/2306557598 wait complete. 2021-01-06T22:03:24.048 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f2d13700 1 Processor -- start 2021-01-06T22:03:24.048 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f2d13700 1 -- start start 2021-01-06T22:03:24.050 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f2d13700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2ec0bb230 0x7fa2ec144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:24.051 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f2d13700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec0c1430 0x7fa2ec14d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:24.051 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f2d13700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2ec147d60 0x7fa2ec15cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:24.051 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f2d13700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec14a340 con 0x7fa2ec147d60 2021-01-06T22:03:24.052 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f2d13700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec14a5e0 con 0x7fa2ec0bb230 2021-01-06T22:03:24.052 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f2d13700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec14a880 con 0x7fa2ec0c1430 2021-01-06T22:03:24.052 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f1510700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec0c1430 0x7fa2ec14d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:24.053 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f1510700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec0c1430 0x7fa2ec14d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.91:52866/0 (socket says 172.21.15.91:52866) 2021-01-06T22:03:24.053 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f1510700 1 -- 172.21.15.91:0/3669952217 learned_addr learned my addr 172.21.15.91:0/3669952217 (peer_addr_for_me v2:172.21.15.91:0/0) 2021-01-06T22:03:24.053 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.046+0000 7fa2f2512700 1 --2- 172.21.15.91:0/3669952217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2ec147d60 0x7fa2ec15cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:24.053 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.050+0000 7fa2f1d11700 1 --2- 172.21.15.91:0/3669952217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2ec0bb230 0x7fa2ec144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:24.053 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.050+0000 7fa2f2512700 1 -- 172.21.15.91:0/3669952217 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec0c1430 msgr2=0x7fa2ec14d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:24.054 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.050+0000 7fa2f2512700 1 --2- 172.21.15.91:0/3669952217 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec0c1430 0x7fa2ec14d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:24.054 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.050+0000 7fa2f2512700 1 -- 172.21.15.91:0/3669952217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2ec0bb230 msgr2=0x7fa2ec144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:24.054 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.050+0000 7fa2f2512700 1 --2- 172.21.15.91:0/3669952217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2ec0bb230 0x7fa2ec144ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:24.054 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.050+0000 7fa2f2512700 1 -- 172.21.15.91:0/3669952217 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec14ab20 con 0x7fa2ec147d60 2021-01-06T22:03:24.055 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.050+0000 7fa2f1510700 1 --2- 172.21.15.91:0/3669952217 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec0c1430 0x7fa2ec14d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:24.055 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.050+0000 7fa2f2512700 1 --2- 172.21.15.91:0/3669952217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2ec147d60 0x7fa2ec15cc90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fa2e400ca60 tx=0x7fa2e4007d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:24.055 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.050+0000 7fa2d9ffb700 1 -- 172.21.15.91:0/3669952217 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e400ce40 con 0x7fa2ec147d60 2021-01-06T22:03:24.056 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.050+0000 7fa2d9ffb700 1 -- 172.21.15.91:0/3669952217 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e4005b70 con 0x7fa2ec147d60 2021-01-06T22:03:24.056 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.050+0000 7fa2d9ffb700 1 -- 172.21.15.91:0/3669952217 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4007110 con 0x7fa2ec147d60 2021-01-06T22:03:24.056 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.050+0000 7fa2f2d13700 1 -- 172.21.15.91:0/3669952217 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ec14adc0 con 0x7fa2ec147d60 2021-01-06T22:03:24.056 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.050+0000 7fa2f2d13700 1 -- 172.21.15.91:0/3669952217 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ec14be30 con 0x7fa2ec147d60 2021-01-06T22:03:24.057 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.050+0000 7fa2d9ffb700 1 -- 172.21.15.91:0/3669952217 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fa2e4010030 con 0x7fa2ec147d60 2021-01-06T22:03:24.057 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.050+0000 7fa2d9ffb700 1 -- 172.21.15.91:0/3669952217 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7fa2e40076e0 con 0x7fa2ec147d60 2021-01-06T22:03:24.057 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.050+0000 7fa2f2d13700 1 -- 172.21.15.91:0/3669952217 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ec0bf820 con 0x7fa2ec147d60 2021-01-06T22:03:24.058 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.054+0000 7fa2d9ffb700 1 -- 172.21.15.91:0/3669952217 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71894 (secure 0 0 0) 0x7fa2ec0bf820 con 0x7fa2ec147d60 2021-01-06T22:03:24.138 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.138+0000 7fa2f2d13700 1 -- 172.21.15.91:0/3669952217 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f641a345-81e4-4400-a44c-d2f82fef7f6f", "id": 4} v 0) v1 -- 0x7fa2ec0450a0 con 0x7fa2ec147d60 2021-01-06T22:03:24.900 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.142+0000 7fa2d9ffb700 1 -- 172.21.15.91:0/3669952217 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f641a345-81e4-4400-a44c-d2f82fef7f6f", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa2e400a920 con 0x7fa2ec147d60 2021-01-06T22:03:24.901 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.142+0000 7fa2f2d13700 1 -- 172.21.15.91:0/3669952217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2ec147d60 msgr2=0x7fa2ec15cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:24.901 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.142+0000 7fa2f2d13700 1 --2- 172.21.15.91:0/3669952217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2ec147d60 0x7fa2ec15cc90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fa2e400ca60 tx=0x7fa2e4007d50).stop 2021-01-06T22:03:24.901 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.142+0000 7fa2f2d13700 1 -- 172.21.15.91:0/3669952217 shutdown_connections 2021-01-06T22:03:24.902 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.142+0000 7fa2f2d13700 1 --2- 172.21.15.91:0/3669952217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2ec147d60 0x7fa2ec15cc90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:24.902 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.142+0000 7fa2f2d13700 1 --2- 172.21.15.91:0/3669952217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2ec0bb230 0x7fa2ec144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:24.902 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.142+0000 7fa2f2d13700 1 --2- 172.21.15.91:0/3669952217 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec0c1430 0x7fa2ec14d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:24.902 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.142+0000 7fa2f2d13700 1 -- 172.21.15.91:0/3669952217 >> 172.21.15.91:0/3669952217 conn(0x7fa2ec0667b0 msgr2=0x7fa2ec0bfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:24.902 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.142+0000 7fa2f2d13700 1 -- 172.21.15.91:0/3669952217 shutdown_connections 2021-01-06T22:03:24.903 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:24.142+0000 7fa2f2d13700 1 -- 172.21.15.91:0/3669952217 wait complete. 2021-01-06T22:03:24.903 INFO:teuthology.orchestra.run.smithi091.stdout:4 2021-01-06T22:03:24.903 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd new c1ff150d-68c6-4e6b-abd4-76af27bb2103 5 2021-01-06T22:03:25.032 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.034+0000 7f63fe925700 1 Processor -- start 2021-01-06T22:03:25.033 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.034+0000 7f63fe925700 1 -- start start 2021-01-06T22:03:25.035 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.034+0000 7f63fe925700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63f8064380 0x7f63f80c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:25.035 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.034+0000 7f63fe925700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f63f814e310 0x7f63f81471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:25.036 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.034+0000 7f63fe925700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f80bb2d0 con 0x7f63f8064760 2021-01-06T22:03:25.036 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.034+0000 7f63fe925700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f814a560 con 0x7f63f814e310 2021-01-06T22:03:25.036 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.034+0000 7f63fe925700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f814a800 con 0x7f63f8064380 2021-01-06T22:03:25.036 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fd923700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63f8064380 0x7f63f80c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:25.036 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fe124700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f63f8064760 0x7f63f80bb730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52232/0 (socket says 172.21.15.91:52232) 2021-01-06T22:03:25.037 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fd122700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f63f814e310 0x7f63f81471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:25.037 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fe124700 1 -- 172.21.15.91:0/1701528726 learned_addr learned my addr 172.21.15.91:0/1701528726 (peer_addr_for_me v1:172.21.15.91:0/0) 2021-01-06T22:03:25.037 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fd923700 1 -- 172.21.15.91:0/1701528726 >> v2:172.21.15.91:3300/0 conn(0x7f63f814e310 msgr2=0x7f63f81471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:25.037 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fd923700 1 --2- 172.21.15.91:0/1701528726 >> v2:172.21.15.91:3300/0 conn(0x7f63f814e310 0x7f63f81471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:25.037 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fd923700 1 -- 172.21.15.91:0/1701528726 >> v1:172.21.15.91:6789/0 conn(0x7f63f8064760 legacy=0x7f63f80bb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:25.038 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fd923700 1 -- 172.21.15.91:0/1701528726 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f814aaa0 con 0x7f63f8064380 2021-01-06T22:03:25.038 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fd923700 1 --2- 172.21.15.91:0/1701528726 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63f8064380 0x7f63f80c0fa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f63f400b980 tx=0x7f63f4006de0).ready entity=mon.2 client_cookie=9b8ed42a8ddd7ee4 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:25.040 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fc921700 1 -- 172.21.15.91:0/1701528726 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4009840 con 0x7f63f8064380 2021-01-06T22:03:25.040 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fc921700 1 -- 172.21.15.91:0/1701528726 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f40099a0 con 0x7f63f8064380 2021-01-06T22:03:25.040 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fc921700 1 -- 172.21.15.91:0/1701528726 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4009d20 con 0x7f63f8064380 2021-01-06T22:03:25.044 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fe925700 1 -- 172.21.15.91:0/1701528726 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63f8064380 msgr2=0x7f63f80c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:25.044 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fe925700 1 --2- 172.21.15.91:0/1701528726 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63f8064380 0x7f63f80c0fa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f63f400b980 tx=0x7f63f4006de0).stop 2021-01-06T22:03:25.045 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fe925700 1 -- 172.21.15.91:0/1701528726 shutdown_connections 2021-01-06T22:03:25.045 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fe925700 1 --2- 172.21.15.91:0/1701528726 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63f8064380 0x7f63f80c0fa0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:25.045 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fe925700 1 --2- 172.21.15.91:0/1701528726 >> v2:172.21.15.91:3300/0 conn(0x7f63f814e310 0x7f63f81471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:25.045 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fe925700 1 -- 172.21.15.91:0/1701528726 >> 172.21.15.91:0/1701528726 conn(0x7f63f80667b0 msgr2=0x7f63f80c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:25.045 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fe925700 1 -- 172.21.15.91:0/1701528726 shutdown_connections 2021-01-06T22:03:25.046 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fe925700 1 -- 172.21.15.91:0/1701528726 wait complete. 2021-01-06T22:03:25.046 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fe925700 1 Processor -- start 2021-01-06T22:03:25.046 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.037+0000 7f63fe925700 1 -- start start 2021-01-06T22:03:25.046 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe925700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f8064380 0x7f63f8143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:25.047 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe925700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63f8064760 0x7f63f8141210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:25.047 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe925700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63f814e310 0x7f63f80b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:25.047 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe925700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f8143ac0 con 0x7f63f8064760 2021-01-06T22:03:25.047 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe925700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f80b4820 con 0x7f63f8064380 2021-01-06T22:03:25.047 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe925700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f80b4ac0 con 0x7f63f814e310 2021-01-06T22:03:25.048 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe124700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63f814e310 0x7f63f80b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:25.048 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fd122700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63f8064760 0x7f63f8141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:25.048 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe124700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63f814e310 0x7f63f80b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.91:52880/0 (socket says 172.21.15.91:52880) 2021-01-06T22:03:25.048 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe124700 1 -- 172.21.15.91:0/2726716925 learned_addr learned my addr 172.21.15.91:0/2726716925 (peer_addr_for_me v2:172.21.15.91:0/0) 2021-01-06T22:03:25.048 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fd923700 1 --2- 172.21.15.91:0/2726716925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f8064380 0x7f63f8143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:25.049 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe124700 1 -- 172.21.15.91:0/2726716925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f8064380 msgr2=0x7f63f8143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:25.049 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe124700 1 --2- 172.21.15.91:0/2726716925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f8064380 0x7f63f8143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:25.049 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe124700 1 -- 172.21.15.91:0/2726716925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63f8064760 msgr2=0x7f63f8141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:25.049 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe124700 1 --2- 172.21.15.91:0/2726716925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63f8064760 0x7f63f8141210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:25.049 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe124700 1 -- 172.21.15.91:0/2726716925 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f80b4d60 con 0x7f63f814e310 2021-01-06T22:03:25.050 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe124700 1 --2- 172.21.15.91:0/2726716925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63f814e310 0x7f63f80b54a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f63f0007c30 tx=0x7f63f0007f00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:25.050 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63e5ffb700 1 -- 172.21.15.91:0/2726716925 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0005b70 con 0x7f63f814e310 2021-01-06T22:03:25.050 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63e5ffb700 1 -- 172.21.15.91:0/2726716925 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f0015070 con 0x7f63f814e310 2021-01-06T22:03:25.050 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe925700 1 -- 172.21.15.91:0/2726716925 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f80b5000 con 0x7f63f814e310 2021-01-06T22:03:25.050 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63e5ffb700 1 -- 172.21.15.91:0/2726716925 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0005cd0 con 0x7f63f814e310 2021-01-06T22:03:25.051 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe925700 1 -- 172.21.15.91:0/2726716925 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f8141be0 con 0x7f63f814e310 2021-01-06T22:03:25.051 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63e5ffb700 1 -- 172.21.15.91:0/2726716925 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f63f0021070 con 0x7f63f814e310 2021-01-06T22:03:25.051 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63e5ffb700 1 -- 172.21.15.91:0/2726716925 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7f63f0005e30 con 0x7f63f814e310 2021-01-06T22:03:25.051 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.041+0000 7f63fe925700 1 -- 172.21.15.91:0/2726716925 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f80450a0 con 0x7f63f814e310 2021-01-06T22:03:25.052 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.045+0000 7f63e5ffb700 1 -- 172.21.15.91:0/2726716925 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71894 (secure 0 0 0) 0x7f63f00398f0 con 0x7f63f814e310 2021-01-06T22:03:25.083 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.085+0000 7f63e5ffb700 1 -- 172.21.15.91:0/2726716925 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mgrmap(e 2) v1 ==== 33581+0+0 (secure 0 0 0) 0x7f63f0007640 con 0x7f63f814e310 2021-01-06T22:03:25.142 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.145+0000 7f63fe925700 1 -- 172.21.15.91:0/2726716925 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c1ff150d-68c6-4e6b-abd4-76af27bb2103", "id": 5} v 0) v1 -- 0x7f63f8061f70 con 0x7f63f814e310 2021-01-06T22:03:25.586 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.165+0000 7f63e5ffb700 1 -- 172.21.15.91:0/2726716925 <== mon.2 v2:172.21.15.91:3301/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c1ff150d-68c6-4e6b-abd4-76af27bb2103", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f63f0011560 con 0x7f63f814e310 2021-01-06T22:03:25.587 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.165+0000 7f63fe925700 1 -- 172.21.15.91:0/2726716925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63f814e310 msgr2=0x7f63f80b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:25.587 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.165+0000 7f63fe925700 1 --2- 172.21.15.91:0/2726716925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63f814e310 0x7f63f80b54a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f63f0007c30 tx=0x7f63f0007f00).stop 2021-01-06T22:03:25.587 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.165+0000 7f63fe925700 1 -- 172.21.15.91:0/2726716925 shutdown_connections 2021-01-06T22:03:25.587 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.165+0000 7f63fe925700 1 --2- 172.21.15.91:0/2726716925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63f814e310 0x7f63f80b54a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:25.588 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.165+0000 7f63fe925700 1 --2- 172.21.15.91:0/2726716925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f8064380 0x7f63f8143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:25.588 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.165+0000 7f63fe925700 1 --2- 172.21.15.91:0/2726716925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63f8064760 0x7f63f8141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:25.588 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.165+0000 7f63fe925700 1 -- 172.21.15.91:0/2726716925 >> 172.21.15.91:0/2726716925 conn(0x7f63f80667b0 msgr2=0x7f63f80be790 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:25.588 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.165+0000 7f63fe925700 1 -- 172.21.15.91:0/2726716925 shutdown_connections 2021-01-06T22:03:25.588 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.165+0000 7f63fe925700 1 -- 172.21.15.91:0/2726716925 wait complete. 2021-01-06T22:03:25.589 INFO:teuthology.orchestra.run.smithi091.stdout:5 2021-01-06T22:03:25.589 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd new 35a58e1a-0a0a-4f02-a477-adc029038d9b 6 2021-01-06T22:03:25.706 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.705+0000 7fd753fff700 1 Processor -- start 2021-01-06T22:03:25.706 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.705+0000 7fd753fff700 1 -- start start 2021-01-06T22:03:25.708 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd753fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd7540be730 0x7fd7540657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:25.708 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd753fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd75414ba00 0x7fd75414be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:25.708 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd753fff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd75414ee80 con 0x7fd7540be350 2021-01-06T22:03:25.708 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd753fff700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd75414f120 con 0x7fd7540be730 2021-01-06T22:03:25.709 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd753fff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75414f3c0 con 0x7fd75414ba00 2021-01-06T22:03:25.709 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd7537fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd75414ba00 0x7fd75414be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:25.709 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd752ffd700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fd7540be350 0x7fd7540ba350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52240/0 (socket says 172.21.15.91:52240) 2021-01-06T22:03:25.709 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd752ffd700 1 -- 172.21.15.91:0/3754107269 learned_addr learned my addr 172.21.15.91:0/3754107269 (peer_addr_for_me v1:172.21.15.91:0/0) 2021-01-06T22:03:25.709 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd7527fc700 1 --2- 172.21.15.91:0/3754107269 >> v2:172.21.15.203:3300/0 conn(0x7fd7540be730 0x7fd7540657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:25.710 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd751ffb700 1 -- 172.21.15.91:0/3754107269 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 703787448 0 0) 0x7fd75414ee80 con 0x7fd7540be350 2021-01-06T22:03:25.710 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd751ffb700 1 -- 172.21.15.91:0/3754107269 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd734003230 con 0x7fd7540be350 2021-01-06T22:03:25.710 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd7537fe700 1 -- 172.21.15.91:0/3754107269 >> v2:172.21.15.203:3300/0 conn(0x7fd7540be730 msgr2=0x7fd7540657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:25.710 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd7537fe700 1 --2- 172.21.15.91:0/3754107269 >> v2:172.21.15.203:3300/0 conn(0x7fd7540be730 0x7fd7540657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:25.710 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd7537fe700 1 -- 172.21.15.91:0/3754107269 >> v1:172.21.15.91:6789/0 conn(0x7fd7540be350 legacy=0x7fd7540ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:25.711 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd7537fe700 1 -- 172.21.15.91:0/3754107269 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75414f660 con 0x7fd75414ba00 2021-01-06T22:03:25.711 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd7537fe700 1 --2- 172.21.15.91:0/3754107269 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd75414ba00 0x7fd75414be60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fd74c009c90 tx=0x7fd74c0091d0).ready entity=mon.2 client_cookie=718d0765fd8f1234 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:25.711 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd751ffb700 1 -- 172.21.15.91:0/3754107269 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74c00ba70 con 0x7fd75414ba00 2021-01-06T22:03:25.711 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd751ffb700 1 -- 172.21.15.91:0/3754107269 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74c00bbd0 con 0x7fd75414ba00 2021-01-06T22:03:25.711 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd751ffb700 1 -- 172.21.15.91:0/3754107269 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74c011400 con 0x7fd75414ba00 2021-01-06T22:03:25.714 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd753fff700 1 -- 172.21.15.91:0/3754107269 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd75414ba00 msgr2=0x7fd75414be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:25.714 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd753fff700 1 --2- 172.21.15.91:0/3754107269 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd75414ba00 0x7fd75414be60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fd74c009c90 tx=0x7fd74c0091d0).stop 2021-01-06T22:03:25.715 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd753fff700 1 -- 172.21.15.91:0/3754107269 shutdown_connections 2021-01-06T22:03:25.715 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd753fff700 1 --2- 172.21.15.91:0/3754107269 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd75414ba00 0x7fd75414be60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:25.715 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd753fff700 1 --2- 172.21.15.91:0/3754107269 >> v2:172.21.15.203:3300/0 conn(0x7fd7540be730 0x7fd7540657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:25.715 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd753fff700 1 -- 172.21.15.91:0/3754107269 >> 172.21.15.91:0/3754107269 conn(0x7fd754067e20 msgr2=0x7fd754068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:25.716 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd753fff700 1 -- 172.21.15.91:0/3754107269 shutdown_connections 2021-01-06T22:03:25.716 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd753fff700 1 -- 172.21.15.91:0/3754107269 wait complete. 2021-01-06T22:03:25.716 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd753fff700 1 Processor -- start 2021-01-06T22:03:25.716 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.709+0000 7fd753fff700 1 -- start start 2021-01-06T22:03:25.716 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd753fff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7540be350 0x7fd75415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:25.717 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd753fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7540be730 0x7fd75415b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:25.717 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd753fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd75414ba00 0x7fd754065310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:25.717 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd753fff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7541462f0 con 0x7fd7540be350 2021-01-06T22:03:25.717 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd753fff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd754146590 con 0x7fd7540be730 2021-01-06T22:03:25.717 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd753fff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd754146830 con 0x7fd75414ba00 2021-01-06T22:03:25.718 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd752ffd700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7540be350 0x7fd75415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:25.718 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd752ffd700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7540be350 0x7fd75415ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36392/0 (socket says 172.21.15.91:36392) 2021-01-06T22:03:25.718 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd752ffd700 1 -- 172.21.15.91:0/458233156 learned_addr learned my addr 172.21.15.91:0/458233156 (peer_addr_for_me v2:172.21.15.91:0/0) 2021-01-06T22:03:25.718 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd7537fe700 1 --2- 172.21.15.91:0/458233156 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd75414ba00 0x7fd754065310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:25.718 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd7527fc700 1 --2- 172.21.15.91:0/458233156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7540be730 0x7fd75415b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:25.719 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd752ffd700 1 -- 172.21.15.91:0/458233156 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd75414ba00 msgr2=0x7fd754065310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:25.719 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd752ffd700 1 --2- 172.21.15.91:0/458233156 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd75414ba00 0x7fd754065310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:25.719 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd752ffd700 1 -- 172.21.15.91:0/458233156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7540be730 msgr2=0x7fd75415b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:25.719 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd752ffd700 1 --2- 172.21.15.91:0/458233156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7540be730 0x7fd75415b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:25.719 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd752ffd700 1 -- 172.21.15.91:0/458233156 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75415af20 con 0x7fd7540be350 2021-01-06T22:03:25.720 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd752ffd700 1 --2- 172.21.15.91:0/458233156 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7540be350 0x7fd75415ed80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fd74400c6d0 tx=0x7fd744007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:25.720 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd74affd700 1 -- 172.21.15.91:0/458233156 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440072e0 con 0x7fd7540be350 2021-01-06T22:03:25.720 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd74affd700 1 -- 172.21.15.91:0/458233156 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd744007440 con 0x7fd7540be350 2021-01-06T22:03:25.720 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd753fff700 1 -- 172.21.15.91:0/458233156 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75415b1c0 con 0x7fd7540be350 2021-01-06T22:03:25.720 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd74affd700 1 -- 172.21.15.91:0/458233156 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440057f0 con 0x7fd7540be350 2021-01-06T22:03:25.721 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd753fff700 1 -- 172.21.15.91:0/458233156 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75413dbb0 con 0x7fd7540be350 2021-01-06T22:03:25.721 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd74affd700 1 -- 172.21.15.91:0/458233156 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 2) v1 ==== 33581+0+0 (secure 0 0 0) 0x7fd744010070 con 0x7fd7540be350 2021-01-06T22:03:25.721 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd753fff700 1 -- 172.21.15.91:0/458233156 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7540450a0 con 0x7fd7540be350 2021-01-06T22:03:25.721 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.713+0000 7fd74affd700 1 -- 172.21.15.91:0/458233156 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1820+0+0 (secure 0 0 0) 0x7fd74403bcc0 con 0x7fd7540be350 2021-01-06T22:03:25.722 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.717+0000 7fd74affd700 1 -- 172.21.15.91:0/458233156 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71894 (secure 0 0 0) 0x7fd744010d60 con 0x7fd7540be350 2021-01-06T22:03:25.833 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.833+0000 7fd753fff700 1 -- 172.21.15.91:0/458233156 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd new", "uuid": "35a58e1a-0a0a-4f02-a477-adc029038d9b", "id": 6} v 0) v1 -- 0x7fd75415b350 con 0x7fd7540be350 2021-01-06T22:03:25.843 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.845+0000 7fd74affd700 1 -- 172.21.15.91:0/458233156 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "35a58e1a-0a0a-4f02-a477-adc029038d9b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd7440360b0 con 0x7fd7540be350 2021-01-06T22:03:25.844 INFO:teuthology.orchestra.run.smithi091.stdout:6 2021-01-06T22:03:25.845 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.845+0000 7fd753fff700 1 -- 172.21.15.91:0/458233156 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7540be350 msgr2=0x7fd75415ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:25.845 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.845+0000 7fd753fff700 1 --2- 172.21.15.91:0/458233156 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7540be350 0x7fd75415ed80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fd74400c6d0 tx=0x7fd744007ac0).stop 2021-01-06T22:03:25.845 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.845+0000 7fd753fff700 1 -- 172.21.15.91:0/458233156 shutdown_connections 2021-01-06T22:03:25.846 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.845+0000 7fd753fff700 1 --2- 172.21.15.91:0/458233156 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd75414ba00 0x7fd754065310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:25.846 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.845+0000 7fd753fff700 1 --2- 172.21.15.91:0/458233156 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7540be350 0x7fd75415ed80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:25.846 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.845+0000 7fd753fff700 1 --2- 172.21.15.91:0/458233156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7540be730 0x7fd75415b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:25.846 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.845+0000 7fd753fff700 1 -- 172.21.15.91:0/458233156 >> 172.21.15.91:0/458233156 conn(0x7fd754067e20 msgr2=0x7fd7540ba580 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:25.847 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.845+0000 7fd753fff700 1 -- 172.21.15.91:0/458233156 shutdown_connections 2021-01-06T22:03:25.847 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.845+0000 7fd753fff700 1 -- 172.21.15.91:0/458233156 wait complete. 2021-01-06T22:03:25.859 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd new 809a2d89-4303-482a-8558-aef907814234 7 2021-01-06T22:03:25.978 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.977+0000 7f1091615700 1 Processor -- start 2021-01-06T22:03:26.069 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.977+0000 7f1091615700 1 -- start start 2021-01-06T22:03:26.074 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1091615700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f108c063010 0x7f108c151160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:26.074 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1091615700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f108c0633f0 0x7f108c14e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:26.075 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1091615700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108c0602b0 con 0x7f108c14e8b0 2021-01-06T22:03:26.075 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1091615700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108c14b760 con 0x7f108c063010 2021-01-06T22:03:26.075 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1091615700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0bdb20 con 0x7f108c0633f0 2021-01-06T22:03:26.075 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1083fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f108c0633f0 0x7f108c14e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:26.076 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1083fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f108c0633f0 0x7f108c14e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.91:52896/0 (socket says 172.21.15.91:52896) 2021-01-06T22:03:26.076 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1083fff700 1 -- 172.21.15.91:0/3430987161 learned_addr learned my addr 172.21.15.91:0/3430987161 (peer_addr_for_me v2:172.21.15.91:0/0) 2021-01-06T22:03:26.076 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f108bfff700 1 --2- 172.21.15.91:0/3430987161 >> v2:172.21.15.203:3300/0 conn(0x7f108c063010 0x7f108c151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:26.076 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1083fff700 1 -- 172.21.15.91:0/3430987161 >> v2:172.21.15.203:3300/0 conn(0x7f108c063010 msgr2=0x7f108c151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:26.077 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1083fff700 1 --2- 172.21.15.91:0/3430987161 >> v2:172.21.15.203:3300/0 conn(0x7f108c063010 0x7f108c151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:26.077 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1083fff700 1 -- 172.21.15.91:0/3430987161 >> v1:172.21.15.203:6789/0 conn(0x7f108c14e8b0 legacy=0x7f108c0be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:26.077 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1083fff700 1 -- 172.21.15.91:0/3430987161 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c151710 con 0x7f108c0633f0 2021-01-06T22:03:26.077 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1083fff700 1 --2- 172.21.15.91:0/3430987161 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f108c0633f0 0x7f108c14e3c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f107400b9e0 tx=0x7f1074006ea0).ready entity=mon.2 client_cookie=c126446b6a15de8a server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:26.078 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f108b7fe700 1 -- 172.21.15.91:0/3430987161 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074009920 con 0x7f108c0633f0 2021-01-06T22:03:26.078 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f108b7fe700 1 -- 172.21.15.91:0/3430987161 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1074009a80 con 0x7f108c0633f0 2021-01-06T22:03:26.078 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f108b7fe700 1 -- 172.21.15.91:0/3430987161 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074009e00 con 0x7f108c0633f0 2021-01-06T22:03:26.078 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1091615700 1 -- 172.21.15.91:0/3430987161 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f108c0633f0 msgr2=0x7f108c14e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:26.079 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1091615700 1 --2- 172.21.15.91:0/3430987161 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f108c0633f0 0x7f108c14e3c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f107400b9e0 tx=0x7f1074006ea0).stop 2021-01-06T22:03:26.079 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1091615700 1 -- 172.21.15.91:0/3430987161 shutdown_connections 2021-01-06T22:03:26.079 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1091615700 1 --2- 172.21.15.91:0/3430987161 >> v2:172.21.15.203:3300/0 conn(0x7f108c063010 0x7f108c151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:26.079 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1091615700 1 --2- 172.21.15.91:0/3430987161 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f108c0633f0 0x7f108c14e3c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:26.080 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1091615700 1 -- 172.21.15.91:0/3430987161 >> 172.21.15.91:0/3430987161 conn(0x7f108c067e20 msgr2=0x7f108c068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:26.080 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1091615700 1 -- 172.21.15.91:0/3430987161 shutdown_connections 2021-01-06T22:03:26.080 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1091615700 1 -- 172.21.15.91:0/3430987161 wait complete. 2021-01-06T22:03:26.080 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1091615700 1 Processor -- start 2021-01-06T22:03:26.080 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.981+0000 7f1091615700 1 -- start start 2021-01-06T22:03:26.081 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f1091615700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f108c063010 0x7f108c161120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:26.081 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f1091615700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108c0633f0 0x7f108c161610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:26.081 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f1091615700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f108c14e8b0 0x7f108c145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:26.081 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f1091615700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c161b00 con 0x7f108c063010 2021-01-06T22:03:26.082 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f1091615700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c15fed0 con 0x7f108c0633f0 2021-01-06T22:03:26.082 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f1091615700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c160170 con 0x7f108c14e8b0 2021-01-06T22:03:26.082 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f108bfff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f108c063010 0x7f108c161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:26.082 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f108bfff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f108c063010 0x7f108c161120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36404/0 (socket says 172.21.15.91:36404) 2021-01-06T22:03:26.083 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f1090e14700 1 --2- 172.21.15.91:0/3604280988 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f108c14e8b0 0x7f108c145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:26.083 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f108bfff700 1 -- 172.21.15.91:0/3604280988 learned_addr learned my addr 172.21.15.91:0/3604280988 (peer_addr_for_me v2:172.21.15.91:0/0) 2021-01-06T22:03:26.083 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f1083fff700 1 --2- 172.21.15.91:0/3604280988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108c0633f0 0x7f108c161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:26.084 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f108bfff700 1 -- 172.21.15.91:0/3604280988 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f108c14e8b0 msgr2=0x7f108c145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:26.084 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f108bfff700 1 --2- 172.21.15.91:0/3604280988 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f108c14e8b0 0x7f108c145770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:26.084 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f108bfff700 1 -- 172.21.15.91:0/3604280988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108c0633f0 msgr2=0x7f108c161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:26.084 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f108bfff700 1 --2- 172.21.15.91:0/3604280988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108c0633f0 0x7f108c161610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:26.084 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f108bfff700 1 -- 172.21.15.91:0/3604280988 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c160410 con 0x7f108c063010 2021-01-06T22:03:26.085 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f108bfff700 1 --2- 172.21.15.91:0/3604280988 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f108c063010 0x7f108c161120 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f107c00d8e0 tx=0x7f107c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:26.085 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f1088ff9700 1 -- 172.21.15.91:0/3604280988 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c0125f0 con 0x7f108c063010 2021-01-06T22:03:26.085 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f1088ff9700 1 -- 172.21.15.91:0/3604280988 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c012750 con 0x7f108c063010 2021-01-06T22:03:26.085 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f1091615700 1 -- 172.21.15.91:0/3604280988 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f108c1606b0 con 0x7f108c063010 2021-01-06T22:03:26.086 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f1088ff9700 1 -- 172.21.15.91:0/3604280988 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c012af0 con 0x7f108c063010 2021-01-06T22:03:26.086 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f1091615700 1 -- 172.21.15.91:0/3604280988 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108c160c80 con 0x7f108c063010 2021-01-06T22:03:26.086 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f1088ff9700 1 -- 172.21.15.91:0/3604280988 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 2) v1 ==== 33581+0+0 (secure 0 0 0) 0x7f107c013480 con 0x7f108c063010 2021-01-06T22:03:26.087 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f1091615700 1 -- 172.21.15.91:0/3604280988 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108c0450a0 con 0x7f108c063010 2021-01-06T22:03:26.087 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.985+0000 7f1088ff9700 1 -- 172.21.15.91:0/3604280988 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1959+0+0 (secure 0 0 0) 0x7f107c03c990 con 0x7f108c063010 2021-01-06T22:03:26.087 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:25.989+0000 7f1088ff9700 1 -- 172.21.15.91:0/3604280988 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71894 (secure 0 0 0) 0x7f107c012e60 con 0x7f108c063010 2021-01-06T22:03:26.087 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:26.073+0000 7f1091615700 1 -- 172.21.15.91:0/3604280988 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd new", "uuid": "809a2d89-4303-482a-8558-aef907814234", "id": 7} v 0) v1 -- 0x7f108c0008d0 con 0x7f108c063010 2021-01-06T22:03:26.089 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:26.081+0000 7f1088ff9700 1 -- 172.21.15.91:0/3604280988 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "809a2d89-4303-482a-8558-aef907814234", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f107c01a310 con 0x7f108c063010 2021-01-06T22:03:26.089 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:26.085+0000 7f1091615700 1 -- 172.21.15.91:0/3604280988 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f108c063010 msgr2=0x7f108c161120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:26.089 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:26.085+0000 7f1091615700 1 --2- 172.21.15.91:0/3604280988 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f108c063010 0x7f108c161120 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f107c00d8e0 tx=0x7f107c008040).stop 2021-01-06T22:03:26.090 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:26.085+0000 7f1091615700 1 -- 172.21.15.91:0/3604280988 shutdown_connections 2021-01-06T22:03:26.090 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:26.085+0000 7f1091615700 1 --2- 172.21.15.91:0/3604280988 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f108c14e8b0 0x7f108c145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:26.090 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:26.085+0000 7f1091615700 1 --2- 172.21.15.91:0/3604280988 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f108c063010 0x7f108c161120 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:26.090 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:26.085+0000 7f1091615700 1 --2- 172.21.15.91:0/3604280988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108c0633f0 0x7f108c161610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:26.090 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:26.085+0000 7f1091615700 1 -- 172.21.15.91:0/3604280988 >> 172.21.15.91:0/3604280988 conn(0x7f108c067e20 msgr2=0x7f108c0bb680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:26.091 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:26.085+0000 7f1091615700 1 -- 172.21.15.91:0/3604280988 shutdown_connections 2021-01-06T22:03:26.091 INFO:teuthology.orchestra.run.smithi091.stderr:2021-01-06T22:03:26.085+0000 7f1091615700 1 -- 172.21.15.91:0/3604280988 wait complete. 2021-01-06T22:03:26.092 INFO:teuthology.orchestra.run.smithi091.stdout:7 2021-01-06T22:03:26.099 INFO:tasks.ceph.osd.0:Restarting daemon 2021-01-06T22:03:26.099 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2021-01-06T22:03:26.101 INFO:tasks.ceph.osd.0:Started 2021-01-06T22:03:26.102 INFO:tasks.ceph.osd.1:Restarting daemon 2021-01-06T22:03:26.102 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2021-01-06T22:03:26.104 INFO:tasks.ceph.osd.1:Started 2021-01-06T22:03:26.104 INFO:tasks.ceph.osd.2:Restarting daemon 2021-01-06T22:03:26.105 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2021-01-06T22:03:26.107 INFO:tasks.ceph.osd.2:Started 2021-01-06T22:03:26.107 INFO:tasks.ceph.osd.3:Restarting daemon 2021-01-06T22:03:26.108 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2021-01-06T22:03:26.109 INFO:tasks.ceph.osd.3:Started 2021-01-06T22:03:26.110 INFO:tasks.ceph.osd.4:Restarting daemon 2021-01-06T22:03:26.110 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2021-01-06T22:03:26.112 INFO:tasks.ceph.osd.4:Started 2021-01-06T22:03:26.113 INFO:tasks.ceph.osd.5:Restarting daemon 2021-01-06T22:03:26.113 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2021-01-06T22:03:26.115 INFO:tasks.ceph.osd.5:Started 2021-01-06T22:03:26.115 INFO:tasks.ceph.osd.6:Restarting daemon 2021-01-06T22:03:26.116 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2021-01-06T22:03:26.117 INFO:tasks.ceph.osd.6:Started 2021-01-06T22:03:26.118 INFO:tasks.ceph.osd.7:Restarting daemon 2021-01-06T22:03:26.118 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2021-01-06T22:03:26.120 INFO:tasks.ceph.osd.7:Started 2021-01-06T22:03:26.121 INFO:tasks.ceph:Waiting for OSDs to come up 2021-01-06T22:03:26.492 INFO:tasks.ceph.osd.5.smithi091.stderr:2021-01-06T22:03:26.493+0000 7fd325432f00 -1 Falling back to public interface 2021-01-06T22:03:26.712 INFO:tasks.ceph.osd.1.smithi203.stderr:2021-01-06T22:03:26.708+0000 7f907e860f00 -1 Falling back to public interface 2021-01-06T22:03:27.075 INFO:tasks.ceph.osd.4.smithi091.stderr:2021-01-06T22:03:26.737+0000 7fcf1ad7af00 -1 Falling back to public interface 2021-01-06T22:03:27.076 INFO:tasks.ceph.osd.6.smithi091.stderr:2021-01-06T22:03:26.741+0000 7f5623dd3f00 -1 Falling back to public interface 2021-01-06T22:03:27.077 INFO:tasks.ceph.osd.7.smithi091.stderr:2021-01-06T22:03:26.753+0000 7f7641962f00 -1 Falling back to public interface 2021-01-06T22:03:27.077 INFO:tasks.ceph.osd.2.smithi203.stderr:2021-01-06T22:03:26.748+0000 7fa525bfbf00 -1 Falling back to public interface 2021-01-06T22:03:27.078 INFO:tasks.ceph.osd.2.smithi203.stderr:2021-01-06T22:03:26.752+0000 7fa5208cc700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6822/0: (98) Address already in use 2021-01-06T22:03:27.079 INFO:tasks.ceph.osd.0.smithi203.stderr:2021-01-06T22:03:26.748+0000 7fd305a42f00 -1 Falling back to public interface 2021-01-06T22:03:27.087 INFO:tasks.ceph.osd.3.smithi203.stderr:2021-01-06T22:03:26.748+0000 7fd02fe1df00 -1 Falling back to public interface 2021-01-06T22:03:27.282 DEBUG:teuthology.orchestra.run.smithi203:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2021-01-06T22:03:27.403 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b2a6a4700 1 Processor -- start 2021-01-06T22:03:27.403 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b2a6a4700 1 -- start start 2021-01-06T22:03:27.406 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b2a6a4700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b241513f0 0x7f3b240c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:27.407 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b2a6a4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3b240bccb0 0x7f3b241517d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:27.407 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b2a6a4700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b240bacb0 con 0x7f3b2414ad90 2021-01-06T22:03:27.407 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b2a6a4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b240bd130 con 0x7f3b240bccb0 2021-01-06T22:03:27.408 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b2a6a4700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b24152000 con 0x7f3b241513f0 2021-01-06T22:03:27.408 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b28ea1700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3b240bccb0 0x7f3b241517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:27.408 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b28ea1700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3b240bccb0 0x7f3b241517d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48028/0 (socket says 172.21.15.203:48028) 2021-01-06T22:03:27.408 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b296a2700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b241513f0 0x7f3b240c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:27.408 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b29ea3700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f3b2414ad90 0x7f3b240641e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:41274/0 (socket says 172.21.15.203:41274) 2021-01-06T22:03:27.409 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b28ea1700 1 -- 172.21.15.203:0/3152892074 learned_addr learned my addr 172.21.15.203:0/3152892074 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:27.412 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b28ea1700 1 -- 172.21.15.203:0/3152892074 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b241513f0 msgr2=0x7f3b240c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:27.412 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b28ea1700 1 --2- 172.21.15.203:0/3152892074 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b241513f0 0x7f3b240c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:27.412 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b28ea1700 1 -- 172.21.15.203:0/3152892074 >> v1:172.21.15.91:6789/0 conn(0x7f3b2414ad90 legacy=0x7f3b240641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:27.412 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b13fff700 1 -- 172.21.15.203:0/3152892074 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1556711229 0 0) 0x7f3b1c001450 con 0x7f3b2414ad90 2021-01-06T22:03:27.413 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b28ea1700 1 -- 172.21.15.203:0/3152892074 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b24067e50 con 0x7f3b240bccb0 2021-01-06T22:03:27.413 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b28ea1700 1 --2- 172.21.15.203:0/3152892074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b240bccb0 0x7f3b241517d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3b1400b9e0 tx=0x7f3b14006ea0).ready entity=mon.1 client_cookie=cca70d3fa414b544 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:27.413 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b13fff700 1 -- 172.21.15.203:0/3152892074 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14009810 con 0x7f3b240bccb0 2021-01-06T22:03:27.413 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b13fff700 1 -- 172.21.15.203:0/3152892074 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b14009970 con 0x7f3b240bccb0 2021-01-06T22:03:27.413 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b13fff700 1 -- 172.21.15.203:0/3152892074 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14009cf0 con 0x7f3b240bccb0 2021-01-06T22:03:27.414 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b2a6a4700 1 -- 172.21.15.203:0/3152892074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b240bccb0 msgr2=0x7f3b241517d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:27.414 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b2a6a4700 1 --2- 172.21.15.203:0/3152892074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b240bccb0 0x7f3b241517d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3b1400b9e0 tx=0x7f3b14006ea0).stop 2021-01-06T22:03:27.414 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b2a6a4700 1 -- 172.21.15.203:0/3152892074 shutdown_connections 2021-01-06T22:03:27.414 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b2a6a4700 1 --2- 172.21.15.203:0/3152892074 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b241513f0 0x7f3b240c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:27.415 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b2a6a4700 1 --2- 172.21.15.203:0/3152892074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b240bccb0 0x7f3b241517d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:27.415 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b2a6a4700 1 -- 172.21.15.203:0/3152892074 >> 172.21.15.203:0/3152892074 conn(0x7f3b240667b0 msgr2=0x7f3b240c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:27.415 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b2a6a4700 1 -- 172.21.15.203:0/3152892074 shutdown_connections 2021-01-06T22:03:27.415 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.400+0000 7f3b2a6a4700 1 -- 172.21.15.203:0/3152892074 wait complete. 2021-01-06T22:03:27.415 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b2a6a4700 1 Processor -- start 2021-01-06T22:03:27.416 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b2a6a4700 1 -- start start 2021-01-06T22:03:27.416 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b2a6a4700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b240bccb0 0x7f3b240beac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:27.416 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b2a6a4700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b2414ad90 0x7f3b2414f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:27.416 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b2a6a4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b241513f0 0x7f3b2414f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:27.416 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b2a6a4700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b240c0630 con 0x7f3b2414ad90 2021-01-06T22:03:27.417 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b2a6a4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2414fb40 con 0x7f3b241513f0 2021-01-06T22:03:27.417 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b2a6a4700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b24152950 con 0x7f3b240bccb0 2021-01-06T22:03:27.417 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b29ea3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b241513f0 0x7f3b2414f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:27.417 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b28ea1700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b2414ad90 0x7f3b2414f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:27.417 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b29ea3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b241513f0 0x7f3b2414f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48032/0 (socket says 172.21.15.203:48032) 2021-01-06T22:03:27.418 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b29ea3700 1 -- 172.21.15.203:0/759506298 learned_addr learned my addr 172.21.15.203:0/759506298 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:27.418 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b296a2700 1 --2- 172.21.15.203:0/759506298 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b240bccb0 0x7f3b240beac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:27.418 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b29ea3700 1 -- 172.21.15.203:0/759506298 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b240bccb0 msgr2=0x7f3b240beac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:27.418 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b29ea3700 1 --2- 172.21.15.203:0/759506298 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b240bccb0 0x7f3b240beac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:27.418 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b29ea3700 1 -- 172.21.15.203:0/759506298 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b2414ad90 msgr2=0x7f3b2414f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:27.419 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b29ea3700 1 --2- 172.21.15.203:0/759506298 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b2414ad90 0x7f3b2414f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:27.419 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b29ea3700 1 -- 172.21.15.203:0/759506298 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b24152bf0 con 0x7f3b241513f0 2021-01-06T22:03:27.419 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.404+0000 7f3b29ea3700 1 --2- 172.21.15.203:0/759506298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b241513f0 0x7f3b2414f650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f3b1c00ca30 tx=0x7f3b1c007bd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:27.420 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.412+0000 7f3b117fa700 1 -- 172.21.15.203:0/759506298 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c005ab0 con 0x7f3b241513f0 2021-01-06T22:03:27.421 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.412+0000 7f3b117fa700 1 -- 172.21.15.203:0/759506298 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c005c10 con 0x7f3b241513f0 2021-01-06T22:03:27.421 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.412+0000 7f3b2a6a4700 1 -- 172.21.15.203:0/759506298 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b240b94b0 con 0x7f3b241513f0 2021-01-06T22:03:27.421 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.412+0000 7f3b117fa700 1 -- 172.21.15.203:0/759506298 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c007110 con 0x7f3b241513f0 2021-01-06T22:03:27.421 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.412+0000 7f3b2a6a4700 1 -- 172.21.15.203:0/759506298 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b240b99f0 con 0x7f3b241513f0 2021-01-06T22:03:27.421 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.412+0000 7f3b117fa700 1 -- 172.21.15.203:0/759506298 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 4) v1 ==== 67153+0+0 (secure 0 0 0) 0x7f3b1c010030 con 0x7f3b241513f0 2021-01-06T22:03:27.422 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.412+0000 7f3b2a6a4700 1 -- 172.21.15.203:0/759506298 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b240b8080 con 0x7f3b241513f0 2021-01-06T22:03:27.422 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.412+0000 7f3b117fa700 1 --2- 172.21.15.203:0/759506298 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3b04056340 0x7f3b04058800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:27.422 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.412+0000 7f3b117fa700 1 -- 172.21.15.203:0/759506298 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2074+0+0 (secure 0 0 0) 0x7f3b1c077c60 con 0x7f3b241513f0 2021-01-06T22:03:27.422 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.416+0000 7f3b296a2700 1 --2- 172.21.15.203:0/759506298 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3b04056340 0x7f3b04058800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:27.423 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.416+0000 7f3b296a2700 1 --2- 172.21.15.203:0/759506298 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3b04056340 0x7f3b04058800 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f3b2000cac0 tx=0x7f3b20006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:27.423 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.420+0000 7f3b117fa700 1 -- 172.21.15.203:0/759506298 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f3b1c044100 con 0x7f3b241513f0 2021-01-06T22:03:27.576 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.572+0000 7f3b2a6a4700 1 -- 172.21.15.203:0/759506298 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b240bacb0 con 0x7f3b241513f0 2021-01-06T22:03:27.577 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.572+0000 7f3b117fa700 1 -- 172.21.15.203:0/759506298 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+8574 (secure 0 0 0) 0x7f3b240bacb0 con 0x7f3b241513f0 2021-01-06T22:03:27.578 INFO:teuthology.misc.health.smithi203.stdout: 2021-01-06T22:03:27.578 INFO:teuthology.misc.health.smithi203.stdout:{"epoch":10,"fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","created":"2021-01-06T22:03:21.322671+0000","modified":"2021-01-06T22:03:26.079948+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-06T22:03:25.161465+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"e0b35c09-147c-4ccc-b566-c48b81e25412","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8add59a0-2d58-4c1b-a8d1-62049ee9d85f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b294a411-c7c9-477e-8eae-3b7bd0563ed3","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5878a8f7-1a42-41c3-9810-0ac41a54dd6f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f641a345-81e4-4400-a44c-d2f82fef7f6f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c1ff150d-68c6-4e6b-abd4-76af27bb2103","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"35a58e1a-0a0a-4f02-a477-adc029038d9b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"809a2d89-4303-482a-8558-aef907814234","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-06T22:03:27.579 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.576+0000 7f3b2a6a4700 1 -- 172.21.15.203:0/759506298 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3b04056340 msgr2=0x7f3b04058800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:27.580 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.576+0000 7f3b2a6a4700 1 --2- 172.21.15.203:0/759506298 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3b04056340 0x7f3b04058800 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f3b2000cac0 tx=0x7f3b20006de0).stop 2021-01-06T22:03:27.580 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.576+0000 7f3b2a6a4700 1 -- 172.21.15.203:0/759506298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b241513f0 msgr2=0x7f3b2414f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:27.581 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.576+0000 7f3b2a6a4700 1 --2- 172.21.15.203:0/759506298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b241513f0 0x7f3b2414f650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f3b1c00ca30 tx=0x7f3b1c007bd0).stop 2021-01-06T22:03:27.581 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.576+0000 7f3b2a6a4700 1 -- 172.21.15.203:0/759506298 shutdown_connections 2021-01-06T22:03:27.581 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.576+0000 7f3b2a6a4700 1 --2- 172.21.15.203:0/759506298 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3b04056340 0x7f3b04058800 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:27.581 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.576+0000 7f3b2a6a4700 1 --2- 172.21.15.203:0/759506298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b241513f0 0x7f3b2414f650 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:27.581 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.576+0000 7f3b2a6a4700 1 --2- 172.21.15.203:0/759506298 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b240bccb0 0x7f3b240beac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:27.582 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.576+0000 7f3b2a6a4700 1 --2- 172.21.15.203:0/759506298 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b2414ad90 0x7f3b2414f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:27.582 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.576+0000 7f3b2a6a4700 1 -- 172.21.15.203:0/759506298 >> 172.21.15.203:0/759506298 conn(0x7f3b240667b0 msgr2=0x7f3b241424e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:27.582 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.576+0000 7f3b2a6a4700 1 -- 172.21.15.203:0/759506298 shutdown_connections 2021-01-06T22:03:27.582 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:27.576+0000 7f3b2a6a4700 1 -- 172.21.15.203:0/759506298 wait complete. 2021-01-06T22:03:27.596 DEBUG:teuthology.misc:0 of 8 OSDs are up 2021-01-06T22:03:27.912 INFO:tasks.ceph.osd.5.smithi091.stderr:2021-01-06T22:03:27.916+0000 7fd325432f00 -1 osd.5 0 log_to_monitors {default=true} 2021-01-06T22:03:27.913 INFO:tasks.ceph.osd.0.smithi203.stderr:2021-01-06T22:03:27.908+0000 7fd305a42f00 -1 osd.0 0 log_to_monitors {default=true} 2021-01-06T22:03:28.150 INFO:tasks.ceph.osd.1.smithi203.stderr:2021-01-06T22:03:28.135+0000 7f907e860f00 -1 osd.1 0 log_to_monitors {default=true} 2021-01-06T22:03:28.154 INFO:tasks.ceph.osd.4.smithi091.stderr:2021-01-06T22:03:28.156+0000 7fcf1ad7af00 -1 osd.4 0 log_to_monitors {default=true} 2021-01-06T22:03:28.197 INFO:tasks.ceph.osd.3.smithi203.stderr:2021-01-06T22:03:28.195+0000 7fd02fe1df00 -1 osd.3 0 log_to_monitors {default=true} 2021-01-06T22:03:28.204 INFO:tasks.ceph.osd.7.smithi091.stderr:2021-01-06T22:03:28.204+0000 7f7641962f00 -1 osd.7 0 log_to_monitors {default=true} 2021-01-06T22:03:28.218 INFO:tasks.ceph.osd.6.smithi091.stderr:2021-01-06T22:03:28.220+0000 7f5623dd3f00 -1 osd.6 0 log_to_monitors {default=true} 2021-01-06T22:03:28.224 INFO:tasks.ceph.osd.2.smithi203.stderr:2021-01-06T22:03:28.223+0000 7fa525bfbf00 -1 osd.2 0 log_to_monitors {default=true} 2021-01-06T22:03:29.110 INFO:tasks.ceph.osd.5.smithi091.stderr:2021-01-06T22:03:29.111+0000 7fd31c8fc700 -1 osd.5 0 waiting for initial osdmap 2021-01-06T22:03:29.113 INFO:tasks.ceph.osd.0.smithi203.stderr:2021-01-06T22:03:29.111+0000 7fd2fcf0c700 -1 osd.0 0 waiting for initial osdmap 2021-01-06T22:03:30.113 INFO:tasks.ceph.osd.4.smithi091.stderr:2021-01-06T22:03:30.115+0000 7fcf12244700 -1 osd.4 0 waiting for initial osdmap 2021-01-06T22:03:30.115 INFO:tasks.ceph.osd.1.smithi203.stderr:2021-01-06T22:03:30.115+0000 7f9075d2a700 -1 osd.1 0 waiting for initial osdmap 2021-01-06T22:03:30.116 INFO:tasks.ceph.osd.6.smithi091.stderr:2021-01-06T22:03:30.115+0000 7f561b29d700 -1 osd.6 0 waiting for initial osdmap 2021-01-06T22:03:30.117 INFO:tasks.ceph.osd.3.smithi203.stderr:2021-01-06T22:03:30.115+0000 7fd0272e7700 -1 osd.3 0 waiting for initial osdmap 2021-01-06T22:03:30.118 INFO:tasks.ceph.osd.7.smithi091.stderr:2021-01-06T22:03:30.119+0000 7f7638e2c700 -1 osd.7 0 waiting for initial osdmap 2021-01-06T22:03:30.122 INFO:tasks.ceph.osd.2.smithi203.stderr:2021-01-06T22:03:30.119+0000 7fa51d0c5700 -1 osd.2 0 waiting for initial osdmap 2021-01-06T22:03:34.406 DEBUG:teuthology.orchestra.run.smithi203:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2021-01-06T22:03:34.525 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.523+0000 7f18a9d7e700 1 Processor -- start 2021-01-06T22:03:34.526 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.523+0000 7f18a9d7e700 1 -- start start 2021-01-06T22:03:34.527 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.523+0000 7f18a9d7e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f18a40c44f0 0x7f18a40c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:34.527 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.523+0000 7f18a9d7e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f18a40c48d0 0x7f18a40c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:34.527 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.523+0000 7f18a9d7e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18a4060fc0 0x7f18a4150730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:34.528 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.523+0000 7f18a9d7e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a40c1d60 con 0x7f18a40c48d0 2021-01-06T22:03:34.528 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.523+0000 7f18a9d7e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a40c0f50 con 0x7f18a40c44f0 2021-01-06T22:03:34.528 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.523+0000 7f18a9d7e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a4066cf0 con 0x7f18a4060fc0 2021-01-06T22:03:34.529 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a3fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f18a40c48d0 0x7f18a40c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:34.530 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a3fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f18a40c48d0 0x7f18a40c1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:56122/0 (socket says 172.21.15.203:56122) 2021-01-06T22:03:34.530 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a3fff700 1 -- 172.21.15.203:0/71070760 learned_addr learned my addr 172.21.15.203:0/71070760 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:34.530 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a8d7c700 1 --2- 172.21.15.203:0/71070760 >> v2:172.21.15.203:3300/0 conn(0x7f18a40c44f0 0x7f18a40c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:34.530 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a957d700 1 --2- 172.21.15.203:0/71070760 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18a4060fc0 0x7f18a4150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:34.531 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a8d7c700 1 -- 172.21.15.203:0/71070760 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18a4060fc0 msgr2=0x7f18a4150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:34.531 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a8d7c700 1 --2- 172.21.15.203:0/71070760 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18a4060fc0 0x7f18a4150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:34.531 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a8d7c700 1 -- 172.21.15.203:0/71070760 >> v2:172.21.15.91:3300/0 conn(0x7f18a40c48d0 msgr2=0x7f18a40c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:34.531 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a8d7c700 1 --2- 172.21.15.203:0/71070760 >> v2:172.21.15.91:3300/0 conn(0x7f18a40c48d0 0x7f18a40c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:34.532 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a8d7c700 1 -- 172.21.15.203:0/71070760 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a4060750 con 0x7f18a40c44f0 2021-01-06T22:03:34.532 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a3fff700 1 --2- 172.21.15.203:0/71070760 >> v2:172.21.15.91:3300/0 conn(0x7f18a40c48d0 0x7f18a40c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:34.532 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a8d7c700 1 --2- 172.21.15.203:0/71070760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18a40c44f0 0x7f18a40c1380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f18a405c9f0 tx=0x7f189c0091d0).ready entity=mon.1 client_cookie=4943e6eeecce5fd6 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:34.532 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a37fe700 1 -- 172.21.15.203:0/71070760 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c00b9c0 con 0x7f18a40c44f0 2021-01-06T22:03:34.532 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a37fe700 1 -- 172.21.15.203:0/71070760 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189c00bb20 con 0x7f18a40c44f0 2021-01-06T22:03:34.533 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a37fe700 1 -- 172.21.15.203:0/71070760 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c00bea0 con 0x7f18a40c44f0 2021-01-06T22:03:34.535 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a9d7e700 1 -- 172.21.15.203:0/71070760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18a40c44f0 msgr2=0x7f18a40c1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:34.536 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.527+0000 7f18a9d7e700 1 --2- 172.21.15.203:0/71070760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18a40c44f0 0x7f18a40c1380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f18a405c9f0 tx=0x7f189c0091d0).stop 2021-01-06T22:03:34.536 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a9d7e700 1 -- 172.21.15.203:0/71070760 shutdown_connections 2021-01-06T22:03:34.536 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a9d7e700 1 --2- 172.21.15.203:0/71070760 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18a4060fc0 0x7f18a4150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:34.537 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a9d7e700 1 --2- 172.21.15.203:0/71070760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18a40c44f0 0x7f18a40c1380 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:34.537 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a9d7e700 1 --2- 172.21.15.203:0/71070760 >> v2:172.21.15.91:3300/0 conn(0x7f18a40c48d0 0x7f18a40c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:34.537 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a9d7e700 1 -- 172.21.15.203:0/71070760 >> 172.21.15.203:0/71070760 conn(0x7f18a4056cd0 msgr2=0x7f18a4061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:34.537 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a9d7e700 1 -- 172.21.15.203:0/71070760 shutdown_connections 2021-01-06T22:03:34.538 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a9d7e700 1 -- 172.21.15.203:0/71070760 wait complete. 2021-01-06T22:03:34.538 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a9d7e700 1 Processor -- start 2021-01-06T22:03:34.538 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a9d7e700 1 -- start start 2021-01-06T22:03:34.538 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a9d7e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18a4060fc0 0x7f18a40ba250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:34.538 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a9d7e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18a40c44f0 0x7f18a414e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:34.539 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a9d7e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18a40c48d0 0x7f18a4150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:34.539 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a9d7e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a40bac80 con 0x7f18a40c44f0 2021-01-06T22:03:34.539 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a9d7e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a40bced0 con 0x7f18a4060fc0 2021-01-06T22:03:34.539 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a9d7e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a4068530 con 0x7f18a40c48d0 2021-01-06T22:03:34.540 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a8d7c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18a4060fc0 0x7f18a40ba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:34.540 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18a40c44f0 0x7f18a414e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:34.540 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a8d7c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18a4060fc0 0x7f18a40ba250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48284/0 (socket says 172.21.15.203:48284) 2021-01-06T22:03:34.540 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a8d7c700 1 -- 172.21.15.203:0/2141546375 learned_addr learned my addr 172.21.15.203:0/2141546375 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:34.540 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a957d700 1 --2- 172.21.15.203:0/2141546375 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18a40c48d0 0x7f18a4150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:34.541 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a3fff700 1 -- 172.21.15.203:0/2141546375 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18a40c48d0 msgr2=0x7f18a4150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:34.541 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a3fff700 1 --2- 172.21.15.203:0/2141546375 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18a40c48d0 0x7f18a4150500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:34.541 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a3fff700 1 -- 172.21.15.203:0/2141546375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18a4060fc0 msgr2=0x7f18a40ba250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:34.541 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a3fff700 1 --2- 172.21.15.203:0/2141546375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18a4060fc0 0x7f18a40ba250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:34.542 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a3fff700 1 -- 172.21.15.203:0/2141546375 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a40baee0 con 0x7f18a40c44f0 2021-01-06T22:03:34.542 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.531+0000 7f18a3fff700 1 --2- 172.21.15.203:0/2141546375 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18a40c44f0 0x7f18a414e0c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f189400ec00 tx=0x7f1894006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:34.542 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.535+0000 7f18a0ff9700 1 -- 172.21.15.203:0/2141546375 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189400ca40 con 0x7f18a40c44f0 2021-01-06T22:03:34.542 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.535+0000 7f18a0ff9700 1 -- 172.21.15.203:0/2141546375 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189400cba0 con 0x7f18a40c44f0 2021-01-06T22:03:34.542 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.535+0000 7f18a9d7e700 1 -- 172.21.15.203:0/2141546375 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a40bb180 con 0x7f18a40c44f0 2021-01-06T22:03:34.543 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.535+0000 7f18a0ff9700 1 -- 172.21.15.203:0/2141546375 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18940123f0 con 0x7f18a40c44f0 2021-01-06T22:03:34.543 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.535+0000 7f18a9d7e700 1 -- 172.21.15.203:0/2141546375 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a40687d0 con 0x7f18a40c44f0 2021-01-06T22:03:34.544 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.535+0000 7f18a9d7e700 1 -- 172.21.15.203:0/2141546375 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a40450a0 con 0x7f18a40c44f0 2021-01-06T22:03:34.544 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.535+0000 7f18a0ff9700 1 -- 172.21.15.203:0/2141546375 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f1894022070 con 0x7f18a40c44f0 2021-01-06T22:03:34.544 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.535+0000 7f18a0ff9700 1 --2- 172.21.15.203:0/2141546375 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18840563d0 0x7f1884058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:34.545 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.535+0000 7f18a0ff9700 1 -- 172.21.15.203:0/2141546375 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f189406f780 con 0x7f18a40c44f0 2021-01-06T22:03:34.545 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.535+0000 7f18a8d7c700 1 --2- 172.21.15.203:0/2141546375 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18840563d0 0x7f1884058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:34.545 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.535+0000 7f18a8d7c700 1 --2- 172.21.15.203:0/2141546375 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18840563d0 0x7f1884058890 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f189c009fd0 tx=0x7f189c017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:34.545 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.543+0000 7f18a0ff9700 1 -- 172.21.15.203:0/2141546375 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f1894045380 con 0x7f18a40c44f0 2021-01-06T22:03:34.696 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.695+0000 7f18a9d7e700 1 -- 172.21.15.203:0/2141546375 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18a40c0ba0 con 0x7f18a40c44f0 2021-01-06T22:03:34.698 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.695+0000 7f18a0ff9700 1 -- 172.21.15.203:0/2141546375 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12153 (secure 0 0 0) 0x7f18940489a0 con 0x7f18a40c44f0 2021-01-06T22:03:34.699 INFO:teuthology.misc.health.smithi203.stdout: 2021-01-06T22:03:34.699 INFO:teuthology.misc.health.smithi203.stdout:{"epoch":16,"fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","created":"2021-01-06T22:03:21.322671+0000","modified":"2021-01-06T22:03:33.162012+0000","last_up_change":"2021-01-06T22:03:31.115611+0000","last_in_change":"2021-01-06T22:03:31.115611+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-06T22:03:25.161465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"e0b35c09-147c-4ccc-b566-c48b81e25412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":18347},{"type":"v1","addr":"172.21.15.203:6815","nonce":18347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":18347},{"type":"v1","addr":"172.21.15.203:6818","nonce":18347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":18347},{"type":"v1","addr":"172.21.15.203:6831","nonce":18347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6821","nonce":18347},{"type":"v1","addr":"172.21.15.203:6827","nonce":18347}]},"public_addr":"172.21.15.203:6815/18347","cluster_addr":"172.21.15.203:6818/18347","heartbeat_back_addr":"172.21.15.203:6831/18347","heartbeat_front_addr":"172.21.15.203:6827/18347","state":["exists","up"]},{"osd":1,"uuid":"8add59a0-2d58-4c1b-a8d1-62049ee9d85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":18337},{"type":"v1","addr":"172.21.15.203:6803","nonce":18337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":18337},{"type":"v1","addr":"172.21.15.203:6805","nonce":18337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":18337},{"type":"v1","addr":"172.21.15.203:6809","nonce":18337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":18337},{"type":"v1","addr":"172.21.15.203:6807","nonce":18337}]},"public_addr":"172.21.15.203:6803/18337","cluster_addr":"172.21.15.203:6805/18337","heartbeat_back_addr":"172.21.15.203:6809/18337","heartbeat_front_addr":"172.21.15.203:6807/18337","state":["exists","up"]},{"osd":2,"uuid":"b294a411-c7c9-477e-8eae-3b7bd0563ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":18354},{"type":"v1","addr":"172.21.15.203:6811","nonce":18354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":18354},{"type":"v1","addr":"172.21.15.203:6819","nonce":18354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6825","nonce":18354},{"type":"v1","addr":"172.21.15.203:6828","nonce":18354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6823","nonce":18354},{"type":"v1","addr":"172.21.15.203:6824","nonce":18354}]},"public_addr":"172.21.15.203:6811/18354","cluster_addr":"172.21.15.203:6819/18354","heartbeat_back_addr":"172.21.15.203:6828/18354","heartbeat_front_addr":"172.21.15.203:6824/18354","state":["exists","up"]},{"osd":3,"uuid":"5878a8f7-1a42-41c3-9810-0ac41a54dd6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":18346},{"type":"v1","addr":"172.21.15.203:6817","nonce":18346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":18346},{"type":"v1","addr":"172.21.15.203:6822","nonce":18346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":18346},{"type":"v1","addr":"172.21.15.203:6833","nonce":18346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":18346},{"type":"v1","addr":"172.21.15.203:6829","nonce":18346}]},"public_addr":"172.21.15.203:6817/18346","cluster_addr":"172.21.15.203:6822/18346","heartbeat_back_addr":"172.21.15.203:6833/18346","heartbeat_front_addr":"172.21.15.203:6829/18346","state":["exists","up"]},{"osd":4,"uuid":"f641a345-81e4-4400-a44c-d2f82fef7f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":18370},{"type":"v1","addr":"172.21.15.91:6809","nonce":18370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":18370},{"type":"v1","addr":"172.21.15.91:6811","nonce":18370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":18370},{"type":"v1","addr":"172.21.15.91:6816","nonce":18370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":18370},{"type":"v1","addr":"172.21.15.91:6813","nonce":18370}]},"public_addr":"172.21.15.91:6809/18370","cluster_addr":"172.21.15.91:6811/18370","heartbeat_back_addr":"172.21.15.91:6816/18370","heartbeat_front_addr":"172.21.15.91:6813/18370","state":["exists","up"]},{"osd":5,"uuid":"c1ff150d-68c6-4e6b-abd4-76af27bb2103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":18372},{"type":"v1","addr":"172.21.15.91:6801","nonce":18372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":18372},{"type":"v1","addr":"172.21.15.91:6803","nonce":18372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":18372},{"type":"v1","addr":"172.21.15.91:6807","nonce":18372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":18372},{"type":"v1","addr":"172.21.15.91:6805","nonce":18372}]},"public_addr":"172.21.15.91:6801/18372","cluster_addr":"172.21.15.91:6803/18372","heartbeat_back_addr":"172.21.15.91:6807/18372","heartbeat_front_addr":"172.21.15.91:6805/18372","state":["exists","up"]},{"osd":6,"uuid":"35a58e1a-0a0a-4f02-a477-adc029038d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6815","nonce":18371},{"type":"v1","addr":"172.21.15.91:6817","nonce":18371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":18371},{"type":"v1","addr":"172.21.15.91:6819","nonce":18371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":18371},{"type":"v1","addr":"172.21.15.91:6823","nonce":18371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":18371},{"type":"v1","addr":"172.21.15.91:6821","nonce":18371}]},"public_addr":"172.21.15.91:6817/18371","cluster_addr":"172.21.15.91:6819/18371","heartbeat_back_addr":"172.21.15.91:6823/18371","heartbeat_front_addr":"172.21.15.91:6821/18371","state":["exists","up"]},{"osd":7,"uuid":"809a2d89-4303-482a-8558-aef907814234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":18373},{"type":"v1","addr":"172.21.15.91:6825","nonce":18373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":18373},{"type":"v1","addr":"172.21.15.91:6827","nonce":18373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":18373},{"type":"v1","addr":"172.21.15.91:6831","nonce":18373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":18373},{"type":"v1","addr":"172.21.15.91:6829","nonce":18373}]},"public_addr":"172.21.15.91:6825/18373","cluster_addr":"172.21.15.91:6827/18373","heartbeat_back_addr":"172.21.15.91:6831/18373","heartbeat_front_addr":"172.21.15.91:6829/18373","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.875052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.103604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.179225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.158512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.206073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.894840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.229708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.248192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-06T22:03:34.700 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.699+0000 7f18a9d7e700 1 -- 172.21.15.203:0/2141546375 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18840563d0 msgr2=0x7f1884058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:34.701 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.699+0000 7f18a9d7e700 1 --2- 172.21.15.203:0/2141546375 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18840563d0 0x7f1884058890 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f189c009fd0 tx=0x7f189c017000).stop 2021-01-06T22:03:34.701 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.699+0000 7f18a9d7e700 1 -- 172.21.15.203:0/2141546375 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18a40c44f0 msgr2=0x7f18a414e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:34.701 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.699+0000 7f18a9d7e700 1 --2- 172.21.15.203:0/2141546375 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18a40c44f0 0x7f18a414e0c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f189400ec00 tx=0x7f1894006ea0).stop 2021-01-06T22:03:34.702 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.699+0000 7f18a9d7e700 1 -- 172.21.15.203:0/2141546375 shutdown_connections 2021-01-06T22:03:34.702 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.699+0000 7f18a9d7e700 1 --2- 172.21.15.203:0/2141546375 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18840563d0 0x7f1884058890 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:34.702 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.699+0000 7f18a9d7e700 1 --2- 172.21.15.203:0/2141546375 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18a40c48d0 0x7f18a4150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:34.702 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.699+0000 7f18a9d7e700 1 --2- 172.21.15.203:0/2141546375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18a4060fc0 0x7f18a40ba250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:34.703 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.699+0000 7f18a9d7e700 1 --2- 172.21.15.203:0/2141546375 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18a40c44f0 0x7f18a414e0c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:34.703 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.699+0000 7f18a9d7e700 1 -- 172.21.15.203:0/2141546375 >> 172.21.15.203:0/2141546375 conn(0x7f18a4056cd0 msgr2=0x7f18a40c2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:34.703 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.699+0000 7f18a9d7e700 1 -- 172.21.15.203:0/2141546375 shutdown_connections 2021-01-06T22:03:34.703 INFO:teuthology.misc.health.smithi203.stderr:2021-01-06T22:03:34.699+0000 7f18a9d7e700 1 -- 172.21.15.203:0/2141546375 wait complete. 2021-01-06T22:03:34.712 DEBUG:teuthology.misc:8 of 8 OSDs are up 2021-01-06T22:03:34.713 INFO:tasks.ceph:Creating RBD pool 2021-01-06T22:03:34.713 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd pool create rbd 8 2021-01-06T22:03:34.837 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.835+0000 7fb3d1dff700 1 Processor -- start 2021-01-06T22:03:34.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.835+0000 7fb3d1dff700 1 -- start start 2021-01-06T22:03:34.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.835+0000 7fb3d1dff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3cc065ed0 0x7fb3cc0b9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:34.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.835+0000 7fb3d1dff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb3cc0602b0 0x7fb3cc155790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:34.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.835+0000 7fb3d1dff700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3cc0be330 con 0x7fb3cc0662b0 2021-01-06T22:03:34.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.835+0000 7fb3d1dff700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3cc0baa00 con 0x7fb3cc0602b0 2021-01-06T22:03:34.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.835+0000 7fb3d1dff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc0bddf0 con 0x7fb3cc065ed0 2021-01-06T22:03:34.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.835+0000 7fb3cbfff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fb3cc0662b0 0x7fb3cc0bcbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:46888/0 (socket says 172.21.15.203:46888) 2021-01-06T22:03:34.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.835+0000 7fb3cbfff700 1 -- 172.21.15.203:0/3737088779 learned_addr learned my addr 172.21.15.203:0/3737088779 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:34.842 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.835+0000 7fb3d15fe700 1 --2- 172.21.15.203:0/3737088779 >> v2:172.21.15.91:3300/0 conn(0x7fb3cc0602b0 0x7fb3cc155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:34.842 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.835+0000 7fb3cb7fe700 1 -- 172.21.15.203:0/3737088779 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395617912 0 0) 0x7fb3cc0be330 con 0x7fb3cc0662b0 2021-01-06T22:03:34.842 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.835+0000 7fb3d0dfd700 1 --2- 172.21.15.203:0/3737088779 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3cc065ed0 0x7fb3cc0b9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:34.842 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.835+0000 7fb3cb7fe700 1 -- 172.21.15.203:0/3737088779 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3ac003230 con 0x7fb3cc0662b0 2021-01-06T22:03:34.843 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 -- 172.21.15.203:0/3737088779 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4141808072 0 0) 0x7fb3ac003230 con 0x7fb3cc0662b0 2021-01-06T22:03:34.843 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 -- 172.21.15.203:0/3737088779 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3cc065ed0 msgr2=0x7fb3cc0b9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:34.843 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 --2- 172.21.15.203:0/3737088779 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3cc065ed0 0x7fb3cc0b9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:34.843 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 -- 172.21.15.203:0/3737088779 >> v2:172.21.15.91:3300/0 conn(0x7fb3cc0602b0 msgr2=0x7fb3cc155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:34.843 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 --2- 172.21.15.203:0/3737088779 >> v2:172.21.15.91:3300/0 conn(0x7fb3cc0602b0 0x7fb3cc155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:34.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 -- 172.21.15.203:0/3737088779 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc0be090 con 0x7fb3cc0662b0 2021-01-06T22:03:34.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 -- 172.21.15.203:0/3737088779 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7fb3b40014e0 con 0x7fb3cc0662b0 2021-01-06T22:03:34.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 -- 172.21.15.203:0/3737088779 >> v1:172.21.15.203:6789/0 conn(0x7fb3cc0662b0 legacy=0x7fb3cc0bcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:34.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 --2- 172.21.15.203:0/3737088779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3ac003720 0x7fb3ac003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:34.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 --2- 172.21.15.203:0/3737088779 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3cc065ed0 0x7fb3ac0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:34.845 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 --2- 172.21.15.203:0/3737088779 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3ac0049c0 0x7fb3ac004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:34.845 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 -- 172.21.15.203:0/3737088779 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc0baa00 con 0x7fb3ac0049c0 2021-01-06T22:03:34.845 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 -- 172.21.15.203:0/3737088779 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc0bddf0 con 0x7fb3ac003720 2021-01-06T22:03:34.845 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 -- 172.21.15.203:0/3737088779 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0041c0 con 0x7fb3cc065ed0 2021-01-06T22:03:34.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3d0dfd700 1 --2- 172.21.15.203:0/3737088779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3ac003720 0x7fb3ac003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:34.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cbfff700 1 --2- 172.21.15.203:0/3737088779 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3cc065ed0 0x7fb3ac0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:34.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3d0dfd700 1 -- 172.21.15.203:0/3737088779 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3cc065ed0 msgr2=0x7fb3ac0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:34.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3d0dfd700 1 --2- 172.21.15.203:0/3737088779 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3cc065ed0 0x7fb3ac0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:34.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3d0dfd700 1 -- 172.21.15.203:0/3737088779 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3ac0049c0 msgr2=0x7fb3ac004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:34.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3d0dfd700 1 --2- 172.21.15.203:0/3737088779 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3ac0049c0 0x7fb3ac004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:03:34.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3d0dfd700 1 -- 172.21.15.203:0/3737088779 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac003230 con 0x7fb3ac003720 2021-01-06T22:03:34.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3d0dfd700 1 --2- 172.21.15.203:0/3737088779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3ac003720 0x7fb3ac003b40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fb3bc00bb90 tx=0x7fb3bc008040).ready entity=mon.1 client_cookie=ef0d36eb5cf029eb server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:34.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 -- 172.21.15.203:0/3737088779 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc00b2b0 con 0x7fb3ac003720 2021-01-06T22:03:34.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 -- 172.21.15.203:0/3737088779 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3bc00ea90 con 0x7fb3ac003720 2021-01-06T22:03:34.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3cb7fe700 1 -- 172.21.15.203:0/3737088779 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc00ee30 con 0x7fb3ac003720 2021-01-06T22:03:34.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3d1dff700 1 -- 172.21.15.203:0/3737088779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3ac003720 msgr2=0x7fb3ac003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:34.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3d1dff700 1 --2- 172.21.15.203:0/3737088779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3ac003720 0x7fb3ac003b40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fb3bc00bb90 tx=0x7fb3bc008040).stop 2021-01-06T22:03:34.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3d15fe700 1 -- 172.21.15.203:0/3737088779 reap_dead start 2021-01-06T22:03:34.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3d1dff700 1 -- 172.21.15.203:0/3737088779 shutdown_connections 2021-01-06T22:03:34.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3d1dff700 1 -- 172.21.15.203:0/3737088779 >> 172.21.15.203:0/3737088779 conn(0x7fb3cc0c2240 msgr2=0x7fb3cc0c2620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:34.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3d1dff700 1 -- 172.21.15.203:0/3737088779 shutdown_connections 2021-01-06T22:03:34.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.839+0000 7fb3d1dff700 1 -- 172.21.15.203:0/3737088779 wait complete. 2021-01-06T22:03:34.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3d1dff700 1 Processor -- start 2021-01-06T22:03:34.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3d1dff700 1 -- start start 2021-01-06T22:03:34.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3d1dff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3cc061bf0 0x7fb3cc15b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:34.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3d1dff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3cc14bf60 0x7fb3cc149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:34.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3d1dff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3cc061fd0 0x7fb3cc147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:34.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3d1dff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc149680 con 0x7fb3cc061fd0 2021-01-06T22:03:34.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3d1dff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc15bb20 con 0x7fb3cc14bf60 2021-01-06T22:03:34.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3d1dff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc15bdc0 con 0x7fb3cc061bf0 2021-01-06T22:03:34.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3cbfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3cc14bf60 0x7fb3cc149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:34.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3cbfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3cc14bf60 0x7fb3cc149190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48306/0 (socket says 172.21.15.203:48306) 2021-01-06T22:03:34.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3cbfff700 1 -- 172.21.15.203:0/2917656643 learned_addr learned my addr 172.21.15.203:0/2917656643 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:34.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3d0dfd700 1 --2- 172.21.15.203:0/2917656643 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3cc061bf0 0x7fb3cc15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:34.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3d15fe700 1 --2- 172.21.15.203:0/2917656643 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3cc061fd0 0x7fb3cc147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:34.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3cbfff700 1 -- 172.21.15.203:0/2917656643 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3cc061bf0 msgr2=0x7fb3cc15b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:34.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3cbfff700 1 --2- 172.21.15.203:0/2917656643 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3cc061bf0 0x7fb3cc15b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:34.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3cbfff700 1 -- 172.21.15.203:0/2917656643 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3cc061fd0 msgr2=0x7fb3cc147890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:34.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3cbfff700 1 --2- 172.21.15.203:0/2917656643 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3cc061fd0 0x7fb3cc147890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:34.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3cbfff700 1 -- 172.21.15.203:0/2917656643 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc15c060 con 0x7fb3cc14bf60 2021-01-06T22:03:34.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3d0dfd700 1 --2- 172.21.15.203:0/2917656643 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3cc061bf0 0x7fb3cc15b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:34.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3cbfff700 1 --2- 172.21.15.203:0/2917656643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3cc14bf60 0x7fb3cc149190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fb3b400eb40 tx=0x7fb3cc0bae90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:34.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3c8ff9700 1 -- 172.21.15.203:0/2917656643 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b400c9a0 con 0x7fb3cc14bf60 2021-01-06T22:03:34.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3c8ff9700 1 -- 172.21.15.203:0/2917656643 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b400cb00 con 0x7fb3cc14bf60 2021-01-06T22:03:34.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3d1dff700 1 -- 172.21.15.203:0/2917656643 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc15c300 con 0x7fb3cc14bf60 2021-01-06T22:03:34.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3c8ff9700 1 -- 172.21.15.203:0/2917656643 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b400ce80 con 0x7fb3cc14bf60 2021-01-06T22:03:34.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3d1dff700 1 -- 172.21.15.203:0/2917656643 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3cc1473f0 con 0x7fb3cc14bf60 2021-01-06T22:03:34.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.843+0000 7fb3d1dff700 1 -- 172.21.15.203:0/2917656643 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3cc1475b0 con 0x7fb3cc14bf60 2021-01-06T22:03:34.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.847+0000 7fb3c8ff9700 1 -- 172.21.15.203:0/2917656643 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fb3b4014640 con 0x7fb3cc14bf60 2021-01-06T22:03:34.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.847+0000 7fb3c8ff9700 1 --2- 172.21.15.203:0/2917656643 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb3b00563d0 0x7fb3b0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:34.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.847+0000 7fb3c8ff9700 1 -- 172.21.15.203:0/2917656643 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7fb3b406f760 con 0x7fb3cc14bf60 2021-01-06T22:03:34.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.847+0000 7fb3d0dfd700 1 --2- 172.21.15.203:0/2917656643 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb3b00563d0 0x7fb3b0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:34.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.847+0000 7fb3d0dfd700 1 --2- 172.21.15.203:0/2917656643 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb3b00563d0 0x7fb3b0058890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fb3bc00b430 tx=0x7fb3bc01a000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:34.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:34.855+0000 7fb3c8ff9700 1 -- 172.21.15.203:0/2917656643 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fb3b4045380 con 0x7fb3cc14bf60 2021-01-06T22:03:35.008 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.007+0000 7fb3d1dff700 1 -- 172.21.15.203:0/2917656643 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fb3cc05ca20 con 0x7fb3cc14bf60 2021-01-06T22:03:35.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.191+0000 7fb3c8ff9700 1 -- 172.21.15.203:0/2917656643 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7fb3b40489a0 con 0x7fb3cc14bf60 2021-01-06T22:03:35.199 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.195+0000 7fb3d1dff700 1 -- 172.21.15.203:0/2917656643 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb3b00563d0 msgr2=0x7fb3b0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:35.199 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.195+0000 7fb3d1dff700 1 --2- 172.21.15.203:0/2917656643 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb3b00563d0 0x7fb3b0058890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fb3bc00b430 tx=0x7fb3bc01a000).stop 2021-01-06T22:03:35.200 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.195+0000 7fb3d1dff700 1 -- 172.21.15.203:0/2917656643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3cc14bf60 msgr2=0x7fb3cc149190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:35.200 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.195+0000 7fb3d1dff700 1 --2- 172.21.15.203:0/2917656643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3cc14bf60 0x7fb3cc149190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fb3b400eb40 tx=0x7fb3cc0bae90).stop 2021-01-06T22:03:35.201 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.195+0000 7fb3d1dff700 1 -- 172.21.15.203:0/2917656643 shutdown_connections 2021-01-06T22:03:35.201 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.195+0000 7fb3d1dff700 1 --2- 172.21.15.203:0/2917656643 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb3b00563d0 0x7fb3b0058890 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:35.201 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.195+0000 7fb3d1dff700 1 --2- 172.21.15.203:0/2917656643 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3cc061fd0 0x7fb3cc147890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:35.202 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.195+0000 7fb3d1dff700 1 --2- 172.21.15.203:0/2917656643 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3cc061bf0 0x7fb3cc15b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:35.202 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.195+0000 7fb3d1dff700 1 --2- 172.21.15.203:0/2917656643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3cc14bf60 0x7fb3cc149190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:35.202 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.195+0000 7fb3d1dff700 1 -- 172.21.15.203:0/2917656643 >> 172.21.15.203:0/2917656643 conn(0x7fb3cc0c2240 msgr2=0x7fb3cc0bcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:35.203 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.195+0000 7fb3d1dff700 1 -- 172.21.15.203:0/2917656643 shutdown_connections 2021-01-06T22:03:35.203 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.195+0000 7fb3d1dff700 1 -- 172.21.15.203:0/2917656643 wait complete. 2021-01-06T22:03:35.204 INFO:teuthology.orchestra.run.smithi203.stderr:pool 'rbd' created 2021-01-06T22:03:35.212 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2021-01-06T22:03:35.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.335+0000 7fad93152700 1 Processor -- start 2021-01-06T22:03:35.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.335+0000 7fad93152700 1 -- start start 2021-01-06T22:03:35.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad93152700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad8c0be650 0x7fad8c066120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:35.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad93152700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fad8c0bcde0 0x7fad8c0a7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:35.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad93152700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8c063130 con 0x7fad8c065d40 2021-01-06T22:03:35.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad93152700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8c062e90 con 0x7fad8c0bcde0 2021-01-06T22:03:35.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad93152700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c1516b0 con 0x7fad8c0be650 2021-01-06T22:03:35.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad92951700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fad8c0bcde0 0x7fad8c0a7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:35.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad92150700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fad8c065d40 0x7fad8c0626e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:46906/0 (socket says 172.21.15.203:46906) 2021-01-06T22:03:35.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad92951700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fad8c0bcde0 0x7fad8c0a7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48312/0 (socket says 172.21.15.203:48312) 2021-01-06T22:03:35.342 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad92150700 1 -- 172.21.15.203:0/1627593767 learned_addr learned my addr 172.21.15.203:0/1627593767 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:35.342 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad9194f700 1 --2- 172.21.15.203:0/1627593767 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad8c0be650 0x7fad8c066120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:35.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad92951700 1 -- 172.21.15.203:0/1627593767 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad8c0be650 msgr2=0x7fad8c066120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:35.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad92951700 1 --2- 172.21.15.203:0/1627593767 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad8c0be650 0x7fad8c066120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:35.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad92951700 1 -- 172.21.15.203:0/1627593767 >> v1:172.21.15.203:6789/0 conn(0x7fad8c065d40 legacy=0x7fad8c0626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:35.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad9114e700 1 -- 172.21.15.203:0/1627593767 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2026339241 0 0) 0x7fad8c063130 con 0x7fad8c065d40 2021-01-06T22:03:35.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad92951700 1 -- 172.21.15.203:0/1627593767 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c151950 con 0x7fad8c0bcde0 2021-01-06T22:03:35.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad9194f700 1 --2- 172.21.15.203:0/1627593767 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad8c0be650 0x7fad8c066120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:35.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad92951700 1 --2- 172.21.15.203:0/1627593767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad8c0bcde0 0x7fad8c0a7b50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fad84009c90 tx=0x7fad840091d0).ready entity=mon.1 client_cookie=f4f2d78e92840815 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:35.345 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad9114e700 1 -- 172.21.15.203:0/1627593767 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8400bc50 con 0x7fad8c0bcde0 2021-01-06T22:03:35.345 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad9114e700 1 -- 172.21.15.203:0/1627593767 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8400bdb0 con 0x7fad8c0bcde0 2021-01-06T22:03:35.345 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad9114e700 1 -- 172.21.15.203:0/1627593767 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84011610 con 0x7fad8c0bcde0 2021-01-06T22:03:35.345 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad93152700 1 -- 172.21.15.203:0/1627593767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad8c0bcde0 msgr2=0x7fad8c0a7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:35.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad93152700 1 --2- 172.21.15.203:0/1627593767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad8c0bcde0 0x7fad8c0a7b50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fad84009c90 tx=0x7fad840091d0).stop 2021-01-06T22:03:35.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad93152700 1 -- 172.21.15.203:0/1627593767 shutdown_connections 2021-01-06T22:03:35.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad93152700 1 --2- 172.21.15.203:0/1627593767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad8c0bcde0 0x7fad8c0a7b50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:35.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad93152700 1 --2- 172.21.15.203:0/1627593767 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad8c0be650 0x7fad8c066120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:35.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad93152700 1 -- 172.21.15.203:0/1627593767 >> 172.21.15.203:0/1627593767 conn(0x7fad8c064ae0 msgr2=0x7fad8c064ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:35.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad93152700 1 -- 172.21.15.203:0/1627593767 shutdown_connections 2021-01-06T22:03:35.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.339+0000 7fad93152700 1 -- 172.21.15.203:0/1627593767 wait complete. 2021-01-06T22:03:35.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad93152700 1 Processor -- start 2021-01-06T22:03:35.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad93152700 1 -- start start 2021-01-06T22:03:35.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad93152700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad8c065d40 0x7fad8c14f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:35.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad93152700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad8c0bcde0 0x7fad8c161870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:35.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad93152700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad8c0be650 0x7fad8c0bb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:35.350 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad93152700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c161d60 con 0x7fad8c065d40 2021-01-06T22:03:35.350 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad93152700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0bc310 con 0x7fad8c0bcde0 2021-01-06T22:03:35.350 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad93152700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0a28d0 con 0x7fad8c0be650 2021-01-06T22:03:35.350 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad9194f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad8c0bcde0 0x7fad8c161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:35.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad9194f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad8c0bcde0 0x7fad8c161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48318/0 (socket says 172.21.15.203:48318) 2021-01-06T22:03:35.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad9194f700 1 -- 172.21.15.203:0/393055879 learned_addr learned my addr 172.21.15.203:0/393055879 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:35.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad92951700 1 --2- 172.21.15.203:0/393055879 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad8c0be650 0x7fad8c0bb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:35.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad92150700 1 --2- 172.21.15.203:0/393055879 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad8c065d40 0x7fad8c14f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:35.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad9194f700 1 -- 172.21.15.203:0/393055879 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad8c0be650 msgr2=0x7fad8c0bb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:35.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad9194f700 1 --2- 172.21.15.203:0/393055879 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad8c0be650 0x7fad8c0bb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:35.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad9194f700 1 -- 172.21.15.203:0/393055879 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad8c065d40 msgr2=0x7fad8c14f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:35.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad9194f700 1 --2- 172.21.15.203:0/393055879 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad8c065d40 0x7fad8c14f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:35.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad9194f700 1 -- 172.21.15.203:0/393055879 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c0a2b70 con 0x7fad8c0bcde0 2021-01-06T22:03:35.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad9194f700 1 --2- 172.21.15.203:0/393055879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad8c0bcde0 0x7fad8c161870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fad8800db60 tx=0x7fad88008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:35.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad7e7fc700 1 -- 172.21.15.203:0/393055879 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad88009820 con 0x7fad8c0bcde0 2021-01-06T22:03:35.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad93152700 1 -- 172.21.15.203:0/393055879 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8c0a6ca0 con 0x7fad8c0bcde0 2021-01-06T22:03:35.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad7e7fc700 1 -- 172.21.15.203:0/393055879 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad88009980 con 0x7fad8c0bcde0 2021-01-06T22:03:35.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad7e7fc700 1 -- 172.21.15.203:0/393055879 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad88009d40 con 0x7fad8c0bcde0 2021-01-06T22:03:35.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.343+0000 7fad93152700 1 -- 172.21.15.203:0/393055879 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c0a71e0 con 0x7fad8c0bcde0 2021-01-06T22:03:35.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.347+0000 7fad93152700 1 -- 172.21.15.203:0/393055879 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8c0a73a0 con 0x7fad8c0bcde0 2021-01-06T22:03:35.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.347+0000 7fad7e7fc700 1 -- 172.21.15.203:0/393055879 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fad88013690 con 0x7fad8c0bcde0 2021-01-06T22:03:35.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.347+0000 7fad7e7fc700 1 --2- 172.21.15.203:0/393055879 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fad6c0563d0 0x7fad6c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:35.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.347+0000 7fad7e7fc700 1 -- 172.21.15.203:0/393055879 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5242+0+0 (secure 0 0 0) 0x7fad8806f590 con 0x7fad8c0bcde0 2021-01-06T22:03:35.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.347+0000 7fad92150700 1 --2- 172.21.15.203:0/393055879 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fad6c0563d0 0x7fad6c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:35.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.347+0000 7fad92150700 1 --2- 172.21.15.203:0/393055879 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fad6c0563d0 0x7fad6c058890 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fad800011a0 tx=0x7fad800079f0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:35.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.355+0000 7fad7e7fc700 1 -- 172.21.15.203:0/393055879 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fad880450f0 con 0x7fad8c0bcde0 2021-01-06T22:03:35.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:35.511+0000 7fad93152700 1 -- 172.21.15.203:0/393055879 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fad8c0a8350 con 0x7fad8c0bcde0 2021-01-06T22:03:36.200 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.199+0000 7fad7e7fc700 1 -- 172.21.15.203:0/393055879 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7fad88048710 con 0x7fad8c0bcde0 2021-01-06T22:03:36.203 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.199+0000 7fad93152700 1 -- 172.21.15.203:0/393055879 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fad6c0563d0 msgr2=0x7fad6c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:36.203 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.199+0000 7fad93152700 1 --2- 172.21.15.203:0/393055879 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fad6c0563d0 0x7fad6c058890 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fad800011a0 tx=0x7fad800079f0).stop 2021-01-06T22:03:36.203 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.199+0000 7fad93152700 1 -- 172.21.15.203:0/393055879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad8c0bcde0 msgr2=0x7fad8c161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:36.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.199+0000 7fad93152700 1 --2- 172.21.15.203:0/393055879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad8c0bcde0 0x7fad8c161870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fad8800db60 tx=0x7fad88008040).stop 2021-01-06T22:03:36.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.199+0000 7fad93152700 1 -- 172.21.15.203:0/393055879 shutdown_connections 2021-01-06T22:03:36.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.199+0000 7fad93152700 1 --2- 172.21.15.203:0/393055879 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fad6c0563d0 0x7fad6c058890 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.199+0000 7fad93152700 1 --2- 172.21.15.203:0/393055879 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad8c0be650 0x7fad8c0bb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.199+0000 7fad93152700 1 --2- 172.21.15.203:0/393055879 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad8c065d40 0x7fad8c14f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.199+0000 7fad93152700 1 --2- 172.21.15.203:0/393055879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad8c0bcde0 0x7fad8c161870 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.199+0000 7fad93152700 1 -- 172.21.15.203:0/393055879 >> 172.21.15.203:0/393055879 conn(0x7fad8c064ae0 msgr2=0x7fad8c0c0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:36.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.199+0000 7fad93152700 1 -- 172.21.15.203:0/393055879 shutdown_connections 2021-01-06T22:03:36.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.199+0000 7fad93152700 1 -- 172.21.15.203:0/393055879 wait complete. 2021-01-06T22:03:36.206 INFO:teuthology.orchestra.run.smithi203.stderr:enabled application 'rbd' on pool 'rbd' 2021-01-06T22:03:36.218 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2021-01-06T22:03:36.219 INFO:tasks.ceph.mds.a:Restarting daemon 2021-01-06T22:03:36.219 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-01-06T22:03:36.222 INFO:tasks.ceph.mds.a:Started 2021-01-06T22:03:36.222 INFO:tasks.ceph.mds.c:Restarting daemon 2021-01-06T22:03:36.223 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-01-06T22:03:36.225 INFO:tasks.ceph.mds.c:Started 2021-01-06T22:03:36.225 INFO:tasks.ceph.mds.e:Restarting daemon 2021-01-06T22:03:36.225 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2021-01-06T22:03:36.227 INFO:tasks.ceph.mds.e:Started 2021-01-06T22:03:36.227 INFO:tasks.ceph.mds.b:Restarting daemon 2021-01-06T22:03:36.228 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-01-06T22:03:36.230 INFO:tasks.ceph.mds.b:Started 2021-01-06T22:03:36.230 INFO:tasks.ceph.mds.d:Restarting daemon 2021-01-06T22:03:36.231 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2021-01-06T22:03:36.233 INFO:tasks.ceph.mds.d:Started 2021-01-06T22:03:36.233 INFO:tasks.ceph.mds.f:Restarting daemon 2021-01-06T22:03:36.234 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2021-01-06T22:03:36.236 INFO:tasks.ceph.mds.f:Started 2021-01-06T22:03:36.236 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2021-01-06T22:03:36.237 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-06T22:03:36.287 INFO:tasks.ceph.mds.c.smithi203.stdout:starting mds.c at 2021-01-06T22:03:36.289 INFO:tasks.ceph.mds.a.smithi203.stdout:starting mds.a at 2021-01-06T22:03:36.307 INFO:tasks.ceph.mds.d.smithi091.stdout:starting mds.d at 2021-01-06T22:03:36.308 INFO:tasks.ceph.mds.b.smithi091.stdout:starting mds.b at 2021-01-06T22:03:36.311 INFO:tasks.ceph.mds.f.smithi091.stdout:starting mds.f at 2021-01-06T22:03:36.311 INFO:tasks.ceph.mds.e.smithi203.stdout:starting mds.e at 2021-01-06T22:03:36.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5a73d700 1 Processor -- start 2021-01-06T22:03:36.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5a73d700 1 -- start start 2021-01-06T22:03:36.373 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5a73d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3f541513f0 0x7f3f540c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:36.373 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5a73d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f5414ad90 0x7f3f5414b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:36.374 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5a73d700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f540bacb0 con 0x7f3f540bccb0 2021-01-06T22:03:36.374 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5a73d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f540bd160 con 0x7f3f541513f0 2021-01-06T22:03:36.374 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5a73d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5414b6d0 con 0x7f3f5414ad90 2021-01-06T22:03:36.374 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5973b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3f541513f0 0x7f3f540c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:36.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5973b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3f541513f0 0x7f3f540c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48366/0 (socket says 172.21.15.203:48366) 2021-01-06T22:03:36.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5973b700 1 -- 172.21.15.203:0/2243580458 learned_addr learned my addr 172.21.15.203:0/2243580458 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:36.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f59f3c700 1 --2- 172.21.15.203:0/2243580458 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f5414ad90 0x7f3f5414b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:36.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5973b700 1 -- 172.21.15.203:0/2243580458 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f5414ad90 msgr2=0x7f3f5414b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:36.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5973b700 1 --2- 172.21.15.203:0/2243580458 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f5414ad90 0x7f3f5414b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5973b700 1 -- 172.21.15.203:0/2243580458 >> v1:172.21.15.91:6789/0 conn(0x7f3f540bccb0 legacy=0x7f3f541517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:36.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5973b700 1 -- 172.21.15.203:0/2243580458 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f54152000 con 0x7f3f541513f0 2021-01-06T22:03:36.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5973b700 1 --2- 172.21.15.203:0/2243580458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f541513f0 0x7f3f540c0fa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f3f4800ba10 tx=0x7f3f48006ea0).ready entity=mon.1 client_cookie=439bb275b7550269 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:36.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f3ffff700 1 -- 172.21.15.203:0/2243580458 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48009890 con 0x7f3f541513f0 2021-01-06T22:03:36.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f3ffff700 1 -- 172.21.15.203:0/2243580458 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f480099f0 con 0x7f3f541513f0 2021-01-06T22:03:36.379 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f3ffff700 1 -- 172.21.15.203:0/2243580458 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48009d70 con 0x7f3f541513f0 2021-01-06T22:03:36.379 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5a73d700 1 -- 172.21.15.203:0/2243580458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f541513f0 msgr2=0x7f3f540c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:36.379 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.371+0000 7f3f5a73d700 1 --2- 172.21.15.203:0/2243580458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f541513f0 0x7f3f540c0fa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f3f4800ba10 tx=0x7f3f48006ea0).stop 2021-01-06T22:03:36.379 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5a73d700 1 -- 172.21.15.203:0/2243580458 shutdown_connections 2021-01-06T22:03:36.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5a73d700 1 --2- 172.21.15.203:0/2243580458 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f5414ad90 0x7f3f5414b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5a73d700 1 --2- 172.21.15.203:0/2243580458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f541513f0 0x7f3f540c0fa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5a73d700 1 -- 172.21.15.203:0/2243580458 >> 172.21.15.203:0/2243580458 conn(0x7f3f540667b0 msgr2=0x7f3f540c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:36.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5a73d700 1 -- 172.21.15.203:0/2243580458 shutdown_connections 2021-01-06T22:03:36.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5a73d700 1 -- 172.21.15.203:0/2243580458 wait complete. 2021-01-06T22:03:36.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5a73d700 1 Processor -- start 2021-01-06T22:03:36.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5a73d700 1 -- start start 2021-01-06T22:03:36.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5a73d700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f540bccb0 0x7f3f541437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:36.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5a73d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f5414ad90 0x7f3f54143ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:36.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5a73d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f541513f0 0x7f3f54160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:36.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5a73d700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f541441d0 con 0x7f3f540bccb0 2021-01-06T22:03:36.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5a73d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f541422c0 con 0x7f3f541513f0 2021-01-06T22:03:36.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5a73d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f54141db0 con 0x7f3f5414ad90 2021-01-06T22:03:36.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f59f3c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f541513f0 0x7f3f54160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:36.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f59f3c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f541513f0 0x7f3f54160ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48372/0 (socket says 172.21.15.203:48372) 2021-01-06T22:03:36.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f59f3c700 1 -- 172.21.15.203:0/4128081289 learned_addr learned my addr 172.21.15.203:0/4128081289 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:36.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f58f3a700 1 --2- 172.21.15.203:0/4128081289 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f5414ad90 0x7f3f54143ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:36.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5973b700 1 --2- 172.21.15.203:0/4128081289 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f540bccb0 0x7f3f541437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:36.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f59f3c700 1 -- 172.21.15.203:0/4128081289 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f5414ad90 msgr2=0x7f3f54143ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:36.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f59f3c700 1 --2- 172.21.15.203:0/4128081289 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f5414ad90 0x7f3f54143ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f59f3c700 1 -- 172.21.15.203:0/4128081289 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f540bccb0 msgr2=0x7f3f541437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:36.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f59f3c700 1 --2- 172.21.15.203:0/4128081289 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f540bccb0 0x7f3f541437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f59f3c700 1 -- 172.21.15.203:0/4128081289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f54142020 con 0x7f3f541513f0 2021-01-06T22:03:36.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f59f3c700 1 --2- 172.21.15.203:0/4128081289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f541513f0 0x7f3f54160ef0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f3f4c00ec20 tx=0x7f3f4c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:36.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f45ffb700 1 -- 172.21.15.203:0/4128081289 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c00c880 con 0x7f3f541513f0 2021-01-06T22:03:36.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f45ffb700 1 -- 172.21.15.203:0/4128081289 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c00c9e0 con 0x7f3f541513f0 2021-01-06T22:03:36.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5a73d700 1 -- 172.21.15.203:0/4128081289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5414f500 con 0x7f3f541513f0 2021-01-06T22:03:36.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f45ffb700 1 -- 172.21.15.203:0/4128081289 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c00cd80 con 0x7f3f541513f0 2021-01-06T22:03:36.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.375+0000 7f3f5a73d700 1 -- 172.21.15.203:0/4128081289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5414fa40 con 0x7f3f541513f0 2021-01-06T22:03:36.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.379+0000 7f3f45ffb700 1 -- 172.21.15.203:0/4128081289 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f3f4c013640 con 0x7f3f541513f0 2021-01-06T22:03:36.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.379+0000 7f3f5a73d700 1 -- 172.21.15.203:0/4128081289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5414fc00 con 0x7f3f541513f0 2021-01-06T22:03:36.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.379+0000 7f3f45ffb700 1 --2- 172.21.15.203:0/4128081289 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3f300563d0 0x7f3f30058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:36.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.379+0000 7f3f5973b700 1 --2- 172.21.15.203:0/4128081289 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3f300563d0 0x7f3f30058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:36.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.379+0000 7f3f45ffb700 1 -- 172.21.15.203:0/4128081289 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f3f4c06f460 con 0x7f3f541513f0 2021-01-06T22:03:36.389 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.379+0000 7f3f5973b700 1 --2- 172.21.15.203:0/4128081289 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3f300563d0 0x7f3f30058890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f3f4800b190 tx=0x7f3f48016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:36.389 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.387+0000 7f3f45ffb700 1 -- 172.21.15.203:0/4128081289 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f3f4c044f00 con 0x7f3f541513f0 2021-01-06T22:03:36.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.539+0000 7f3f5a73d700 1 -- 172.21.15.203:0/4128081289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f540652d0 con 0x7f3f541513f0 2021-01-06T22:03:36.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.543+0000 7f3f45ffb700 1 -- 172.21.15.203:0/4128081289 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13685 (secure 0 0 0) 0x7f3f4c048520 con 0x7f3f541513f0 2021-01-06T22:03:36.544 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:36.545 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":18,"fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","created":"2021-01-06T22:03:21.322671+0000","modified":"2021-01-06T22:03:36.187352+0000","last_up_change":"2021-01-06T22:03:31.115611+0000","last_in_change":"2021-01-06T22:03:31.115611+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-06T22:03:25.161465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-06T22:03:35.070364+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e0b35c09-147c-4ccc-b566-c48b81e25412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":18347},{"type":"v1","addr":"172.21.15.203:6815","nonce":18347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":18347},{"type":"v1","addr":"172.21.15.203:6818","nonce":18347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":18347},{"type":"v1","addr":"172.21.15.203:6831","nonce":18347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6821","nonce":18347},{"type":"v1","addr":"172.21.15.203:6827","nonce":18347}]},"public_addr":"172.21.15.203:6815/18347","cluster_addr":"172.21.15.203:6818/18347","heartbeat_back_addr":"172.21.15.203:6831/18347","heartbeat_front_addr":"172.21.15.203:6827/18347","state":["exists","up"]},{"osd":1,"uuid":"8add59a0-2d58-4c1b-a8d1-62049ee9d85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":18337},{"type":"v1","addr":"172.21.15.203:6803","nonce":18337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":18337},{"type":"v1","addr":"172.21.15.203:6805","nonce":18337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":18337},{"type":"v1","addr":"172.21.15.203:6809","nonce":18337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":18337},{"type":"v1","addr":"172.21.15.203:6807","nonce":18337}]},"public_addr":"172.21.15.203:6803/18337","cluster_addr":"172.21.15.203:6805/18337","heartbeat_back_addr":"172.21.15.203:6809/18337","heartbeat_front_addr":"172.21.15.203:6807/18337","state":["exists","up"]},{"osd":2,"uuid":"b294a411-c7c9-477e-8eae-3b7bd0563ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":18354},{"type":"v1","addr":"172.21.15.203:6811","nonce":18354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":18354},{"type":"v1","addr":"172.21.15.203:6819","nonce":18354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6825","nonce":18354},{"type":"v1","addr":"172.21.15.203:6828","nonce":18354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6823","nonce":18354},{"type":"v1","addr":"172.21.15.203:6824","nonce":18354}]},"public_addr":"172.21.15.203:6811/18354","cluster_addr":"172.21.15.203:6819/18354","heartbeat_back_addr":"172.21.15.203:6828/18354","heartbeat_front_addr":"172.21.15.203:6824/18354","state":["exists","up"]},{"osd":3,"uuid":"5878a8f7-1a42-41c3-9810-0ac41a54dd6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":18346},{"type":"v1","addr":"172.21.15.203:6817","nonce":18346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":18346},{"type":"v1","addr":"172.21.15.203:6822","nonce":18346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":18346},{"type":"v1","addr":"172.21.15.203:6833","nonce":18346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":18346},{"type":"v1","addr":"172.21.15.203:6829","nonce":18346}]},"public_addr":"172.21.15.203:6817/18346","cluster_addr":"172.21.15.203:6822/18346","heartbeat_back_addr":"172.21.15.203:6833/18346","heartbeat_front_addr":"172.21.15.203:6829/18346","state":["exists","up"]},{"osd":4,"uuid":"f641a345-81e4-4400-a44c-d2f82fef7f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":18370},{"type":"v1","addr":"172.21.15.91:6809","nonce":18370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":18370},{"type":"v1","addr":"172.21.15.91:6811","nonce":18370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":18370},{"type":"v1","addr":"172.21.15.91:6816","nonce":18370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":18370},{"type":"v1","addr":"172.21.15.91:6813","nonce":18370}]},"public_addr":"172.21.15.91:6809/18370","cluster_addr":"172.21.15.91:6811/18370","heartbeat_back_addr":"172.21.15.91:6816/18370","heartbeat_front_addr":"172.21.15.91:6813/18370","state":["exists","up"]},{"osd":5,"uuid":"c1ff150d-68c6-4e6b-abd4-76af27bb2103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":18372},{"type":"v1","addr":"172.21.15.91:6801","nonce":18372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":18372},{"type":"v1","addr":"172.21.15.91:6803","nonce":18372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":18372},{"type":"v1","addr":"172.21.15.91:6807","nonce":18372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":18372},{"type":"v1","addr":"172.21.15.91:6805","nonce":18372}]},"public_addr":"172.21.15.91:6801/18372","cluster_addr":"172.21.15.91:6803/18372","heartbeat_back_addr":"172.21.15.91:6807/18372","heartbeat_front_addr":"172.21.15.91:6805/18372","state":["exists","up"]},{"osd":6,"uuid":"35a58e1a-0a0a-4f02-a477-adc029038d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6815","nonce":18371},{"type":"v1","addr":"172.21.15.91:6817","nonce":18371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":18371},{"type":"v1","addr":"172.21.15.91:6819","nonce":18371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":18371},{"type":"v1","addr":"172.21.15.91:6823","nonce":18371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":18371},{"type":"v1","addr":"172.21.15.91:6821","nonce":18371}]},"public_addr":"172.21.15.91:6817/18371","cluster_addr":"172.21.15.91:6819/18371","heartbeat_back_addr":"172.21.15.91:6823/18371","heartbeat_front_addr":"172.21.15.91:6821/18371","state":["exists","up"]},{"osd":7,"uuid":"809a2d89-4303-482a-8558-aef907814234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":18373},{"type":"v1","addr":"172.21.15.91:6825","nonce":18373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":18373},{"type":"v1","addr":"172.21.15.91:6827","nonce":18373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":18373},{"type":"v1","addr":"172.21.15.91:6831","nonce":18373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":18373},{"type":"v1","addr":"172.21.15.91:6829","nonce":18373}]},"public_addr":"172.21.15.91:6825/18373","cluster_addr":"172.21.15.91:6827/18373","heartbeat_back_addr":"172.21.15.91:6831/18373","heartbeat_front_addr":"172.21.15.91:6829/18373","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.875052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.103604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.179225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.158512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.206073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.894840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.229708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.248192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-06T22:03:36.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.543+0000 7f3f5a73d700 1 -- 172.21.15.203:0/4128081289 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3f300563d0 msgr2=0x7f3f30058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:36.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.543+0000 7f3f5a73d700 1 --2- 172.21.15.203:0/4128081289 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3f300563d0 0x7f3f30058890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f3f4800b190 tx=0x7f3f48016000).stop 2021-01-06T22:03:36.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.543+0000 7f3f5a73d700 1 -- 172.21.15.203:0/4128081289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f541513f0 msgr2=0x7f3f54160ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:36.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.543+0000 7f3f5a73d700 1 --2- 172.21.15.203:0/4128081289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f541513f0 0x7f3f54160ef0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f3f4c00ec20 tx=0x7f3f4c008040).stop 2021-01-06T22:03:36.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.543+0000 7f3f5a73d700 1 -- 172.21.15.203:0/4128081289 shutdown_connections 2021-01-06T22:03:36.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.543+0000 7f3f5a73d700 1 --2- 172.21.15.203:0/4128081289 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3f300563d0 0x7f3f30058890 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.543+0000 7f3f5a73d700 1 --2- 172.21.15.203:0/4128081289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f541513f0 0x7f3f54160ef0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.543+0000 7f3f5a73d700 1 --2- 172.21.15.203:0/4128081289 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f540bccb0 0x7f3f541437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.543+0000 7f3f5a73d700 1 --2- 172.21.15.203:0/4128081289 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f5414ad90 0x7f3f54143ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.543+0000 7f3f5a73d700 1 -- 172.21.15.203:0/4128081289 >> 172.21.15.203:0/4128081289 conn(0x7f3f540667b0 msgr2=0x7f3f54148e10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:36.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.543+0000 7f3f5a73d700 1 -- 172.21.15.203:0/4128081289 shutdown_connections 2021-01-06T22:03:36.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.543+0000 7f3f5a73d700 1 -- 172.21.15.203:0/4128081289 wait complete. 2021-01-06T22:03:36.559 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-01-06T22:03:25.161465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-01-06T22:03:35.070364+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-01-06T22:03:36.560 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-01-06T22:03:36.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdeeacee700 1 Processor -- start 2021-01-06T22:03:36.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdeeacee700 1 -- start start 2021-01-06T22:03:36.690 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdeeacee700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdee40c45d0 0x7fdee40c1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:36.690 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdeeacee700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee4151ac0 con 0x7fdee4066940 2021-01-06T22:03:36.691 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdeeacee700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee4151d60 con 0x7fdee40c49b0 2021-01-06T22:03:36.691 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdeeacee700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee41518d0 con 0x7fdee40c45d0 2021-01-06T22:03:36.691 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdee94eb700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fdee40c49b0 0x7fdee41511c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:46976/0 (socket says 172.21.15.203:46976) 2021-01-06T22:03:36.691 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdee94eb700 1 -- 172.21.15.203:0/4120689364 learned_addr learned my addr 172.21.15.203:0/4120689364 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:36.691 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdee9cec700 1 --2- 172.21.15.203:0/4120689364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdee40c45d0 0x7fdee40c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:36.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdee8cea700 1 -- 172.21.15.203:0/4120689364 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658828626 0 0) 0x7fdee4151d60 con 0x7fdee40c49b0 2021-01-06T22:03:36.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdee8cea700 1 -- 172.21.15.203:0/4120689364 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fded0003230 con 0x7fdee40c49b0 2021-01-06T22:03:36.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdee8cea700 1 -- 172.21.15.203:0/4120689364 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732764174 0 0) 0x7fdee4151ac0 con 0x7fdee4066940 2021-01-06T22:03:36.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdee8cea700 1 -- 172.21.15.203:0/4120689364 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdee4151d60 con 0x7fdee4066940 2021-01-06T22:03:36.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdee8cea700 1 -- 172.21.15.203:0/4120689364 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1178717411 0 0) 0x7fded0003230 con 0x7fdee40c49b0 2021-01-06T22:03:36.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdee8cea700 1 -- 172.21.15.203:0/4120689364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdee40c45d0 msgr2=0x7fdee40c1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:36.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdee8cea700 1 --2- 172.21.15.203:0/4120689364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdee40c45d0 0x7fdee40c1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.687+0000 7fdee8cea700 1 -- 172.21.15.203:0/4120689364 >> v1:172.21.15.91:6789/0 conn(0x7fdee4066940 legacy=0x7fdee40bdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:36.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdee8cea700 1 -- 172.21.15.203:0/4120689364 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee4152000 con 0x7fdee40c49b0 2021-01-06T22:03:36.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdee8cea700 1 -- 172.21.15.203:0/4120689364 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7fdee0001860 con 0x7fdee40c49b0 2021-01-06T22:03:36.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdee9cec700 1 --2- 172.21.15.203:0/4120689364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdee40c45d0 0x7fdee40c1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:36.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdee8cea700 1 -- 172.21.15.203:0/4120689364 >> v1:172.21.15.203:6789/0 conn(0x7fdee40c49b0 legacy=0x7fdee41511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:36.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdee8cea700 1 --2- 172.21.15.203:0/4120689364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fded00038b0 0x7fded0003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:36.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdee8cea700 1 --2- 172.21.15.203:0/4120689364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdee40c45d0 0x7fded00044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:36.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdee8cea700 1 --2- 172.21.15.203:0/4120689364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fded00049c0 0x7fded0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:36.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdee8cea700 1 -- 172.21.15.203:0/4120689364 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded0005330 con 0x7fded00038b0 2021-01-06T22:03:36.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdee8cea700 1 -- 172.21.15.203:0/4120689364 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded000b720 con 0x7fded00049c0 2021-01-06T22:03:36.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdee8cea700 1 -- 172.21.15.203:0/4120689364 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded000b860 con 0x7fdee40c45d0 2021-01-06T22:03:36.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeea4ed700 1 --2- 172.21.15.203:0/4120689364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fded00049c0 0x7fded0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:36.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdee94eb700 1 --2- 172.21.15.203:0/4120689364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdee40c45d0 0x7fded00044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:36.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdee9cec700 1 --2- 172.21.15.203:0/4120689364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fded00038b0 0x7fded0003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:36.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeea4ed700 1 -- 172.21.15.203:0/4120689364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdee40c45d0 msgr2=0x7fded00044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:36.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeea4ed700 1 --2- 172.21.15.203:0/4120689364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdee40c45d0 0x7fded00044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeea4ed700 1 -- 172.21.15.203:0/4120689364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fded00038b0 msgr2=0x7fded0003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:36.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeea4ed700 1 --2- 172.21.15.203:0/4120689364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fded00038b0 0x7fded0003cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeea4ed700 1 -- 172.21.15.203:0/4120689364 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded0003230 con 0x7fded00049c0 2021-01-06T22:03:36.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeea4ed700 1 --2- 172.21.15.203:0/4120689364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fded00049c0 0x7fded0004e40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fdedc001ab0 tx=0x7fdedc007b80).ready entity=mon.1 client_cookie=344455f593be3260 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:36.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdee8cea700 1 -- 172.21.15.203:0/4120689364 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc007020 con 0x7fded00049c0 2021-01-06T22:03:36.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdee8cea700 1 -- 172.21.15.203:0/4120689364 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdedc007180 con 0x7fded00049c0 2021-01-06T22:03:36.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdee8cea700 1 -- 172.21.15.203:0/4120689364 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc00c8b0 con 0x7fded00049c0 2021-01-06T22:03:36.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeeacee700 1 -- 172.21.15.203:0/4120689364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fded00049c0 msgr2=0x7fded0004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:36.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeeacee700 1 --2- 172.21.15.203:0/4120689364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fded00049c0 0x7fded0004e40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fdedc001ab0 tx=0x7fdedc007b80).stop 2021-01-06T22:03:36.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeea4ed700 1 -- 172.21.15.203:0/4120689364 reap_dead start 2021-01-06T22:03:36.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeeacee700 1 -- 172.21.15.203:0/4120689364 shutdown_connections 2021-01-06T22:03:36.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeeacee700 1 -- 172.21.15.203:0/4120689364 >> 172.21.15.203:0/4120689364 conn(0x7fdee40626e0 msgr2=0x7fdee4062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:36.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeeacee700 1 -- 172.21.15.203:0/4120689364 shutdown_connections 2021-01-06T22:03:36.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeeacee700 1 -- 172.21.15.203:0/4120689364 wait complete. 2021-01-06T22:03:36.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeeacee700 1 Processor -- start 2021-01-06T22:03:36.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeeacee700 1 -- start start 2021-01-06T22:03:36.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeeacee700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee414e540 0x7fdee40bd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:36.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeeacee700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdee414e920 0x7fdee40bd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:36.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeeacee700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdee414d620 0x7fdee40bb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:36.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeeacee700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee40bcdc0 con 0x7fdee414d620 2021-01-06T22:03:36.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeeacee700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee414daa0 con 0x7fdee414e540 2021-01-06T22:03:36.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.691+0000 7fdeeacee700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee40bddd0 con 0x7fdee414e920 2021-01-06T22:03:36.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fdee9cec700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee414e540 0x7fdee40bd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:36.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fdee9cec700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee414e540 0x7fdee40bd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48390/0 (socket says 172.21.15.203:48390) 2021-01-06T22:03:36.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fdee9cec700 1 -- 172.21.15.203:0/627361576 learned_addr learned my addr 172.21.15.203:0/627361576 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:36.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fdeea4ed700 1 --2- 172.21.15.203:0/627361576 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdee414d620 0x7fdee40bb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:36.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fdee94eb700 1 --2- 172.21.15.203:0/627361576 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdee414e920 0x7fdee40bd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:36.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fdee9cec700 1 -- 172.21.15.203:0/627361576 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdee414e920 msgr2=0x7fdee40bd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:36.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fdee9cec700 1 --2- 172.21.15.203:0/627361576 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdee414e920 0x7fdee40bd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fdee9cec700 1 -- 172.21.15.203:0/627361576 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdee414d620 msgr2=0x7fdee40bb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:36.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fdee9cec700 1 --2- 172.21.15.203:0/627361576 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdee414d620 0x7fdee40bb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fdee9cec700 1 -- 172.21.15.203:0/627361576 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee414bef0 con 0x7fdee414e540 2021-01-06T22:03:36.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fdee9cec700 1 --2- 172.21.15.203:0/627361576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee414e540 0x7fdee40bd1f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fded4010c20 tx=0x7fded4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:36.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fded9ffb700 1 -- 172.21.15.203:0/627361576 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded400e930 con 0x7fdee414e540 2021-01-06T22:03:36.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fded9ffb700 1 -- 172.21.15.203:0/627361576 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded400ea90 con 0x7fdee414e540 2021-01-06T22:03:36.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fdeeacee700 1 -- 172.21.15.203:0/627361576 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee414c190 con 0x7fdee414e540 2021-01-06T22:03:36.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fded9ffb700 1 -- 172.21.15.203:0/627361576 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded400ee10 con 0x7fdee414e540 2021-01-06T22:03:36.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fdeeacee700 1 -- 172.21.15.203:0/627361576 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee40bb8a0 con 0x7fdee414e540 2021-01-06T22:03:36.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fdeeacee700 1 -- 172.21.15.203:0/627361576 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee40bcf30 con 0x7fdee414e540 2021-01-06T22:03:36.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fded9ffb700 1 -- 172.21.15.203:0/627361576 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fded4024070 con 0x7fdee414e540 2021-01-06T22:03:36.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fded9ffb700 1 --2- 172.21.15.203:0/627361576 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdec40563d0 0x7fdec4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:36.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fded9ffb700 1 -- 172.21.15.203:0/627361576 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fded406fa40 con 0x7fdee414e540 2021-01-06T22:03:36.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.695+0000 7fdee94eb700 1 --2- 172.21.15.203:0/627361576 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdec40563d0 0x7fdec4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:36.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.699+0000 7fdee94eb700 1 --2- 172.21.15.203:0/627361576 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdec40563d0 0x7fdec4058890 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fdee40c1ee0 tx=0x7fdee40c5350).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:36.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.703+0000 7fded9ffb700 1 -- 172.21.15.203:0/627361576 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fdee40bcf30 con 0x7fdee414e540 2021-01-06T22:03:36.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.855+0000 7fdeeacee700 1 -- 172.21.15.203:0/627361576 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdee40bcf30 con 0x7fdee414e540 2021-01-06T22:03:36.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.855+0000 7fded9ffb700 1 -- 172.21.15.203:0/627361576 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdee40bcf30 con 0x7fdee414e540 2021-01-06T22:03:36.858 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 1 2021-01-06T22:03:36.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.859+0000 7fdeeacee700 1 -- 172.21.15.203:0/627361576 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdec40563d0 msgr2=0x7fdec4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:36.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.859+0000 7fdeeacee700 1 --2- 172.21.15.203:0/627361576 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdec40563d0 0x7fdec4058890 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fdee40c1ee0 tx=0x7fdee40c5350).stop 2021-01-06T22:03:36.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.859+0000 7fdeeacee700 1 -- 172.21.15.203:0/627361576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee414e540 msgr2=0x7fdee40bd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:36.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.859+0000 7fdeeacee700 1 --2- 172.21.15.203:0/627361576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee414e540 0x7fdee40bd1f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fded4010c20 tx=0x7fded4008040).stop 2021-01-06T22:03:36.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.859+0000 7fdeeacee700 1 -- 172.21.15.203:0/627361576 shutdown_connections 2021-01-06T22:03:36.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.859+0000 7fdeeacee700 1 --2- 172.21.15.203:0/627361576 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdec40563d0 0x7fdec4058890 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.859+0000 7fdeeacee700 1 --2- 172.21.15.203:0/627361576 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdee414d620 0x7fdee40bb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.859+0000 7fdeeacee700 1 --2- 172.21.15.203:0/627361576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee414e540 0x7fdee40bd1f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.859+0000 7fdeeacee700 1 --2- 172.21.15.203:0/627361576 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdee414e920 0x7fdee40bd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:36.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.859+0000 7fdeeacee700 1 -- 172.21.15.203:0/627361576 >> 172.21.15.203:0/627361576 conn(0x7fdee40626e0 msgr2=0x7fdee40c3630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:36.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.859+0000 7fdeeacee700 1 -- 172.21.15.203:0/627361576 shutdown_connections 2021-01-06T22:03:36.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:36.859+0000 7fdeeacee700 1 -- 172.21.15.203:0/627361576 wait complete. 2021-01-06T22:03:36.874 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-01-06T22:03:37.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc319d3700 1 Processor -- start 2021-01-06T22:03:37.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc319d3700 1 -- start start 2021-01-06T22:03:37.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc319d3700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdc2c0bf4e0 0x7fdc2c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.007 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc319d3700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2c0676a0 con 0x7fdc2c0bf8c0 2021-01-06T22:03:37.007 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc319d3700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2c0bbef0 con 0x7fdc2c0609c0 2021-01-06T22:03:37.007 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc319d3700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2c057510 con 0x7fdc2c0bf4e0 2021-01-06T22:03:37.007 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc23fff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fdc2c0609c0 0x7fdc2c14b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:46994/0 (socket says 172.21.15.203:46994) 2021-01-06T22:03:37.008 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc23fff700 1 -- 172.21.15.203:0/1929403353 learned_addr learned my addr 172.21.15.203:0/1929403353 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:37.008 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc2affd700 1 --2- 172.21.15.203:0/1929403353 >> v2:172.21.15.203:3300/0 conn(0x7fdc2c0bf4e0 0x7fdc2c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.008 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc2a7fc700 1 -- 172.21.15.203:0/1929403353 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703742759 0 0) 0x7fdc2c0bbef0 con 0x7fdc2c0609c0 2021-01-06T22:03:37.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc2a7fc700 1 -- 172.21.15.203:0/1929403353 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc0c003230 con 0x7fdc2c0609c0 2021-01-06T22:03:37.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc2affd700 1 -- 172.21.15.203:0/1929403353 >> v1:172.21.15.203:6789/0 conn(0x7fdc2c0609c0 legacy=0x7fdc2c14b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc2affd700 1 -- 172.21.15.203:0/1929403353 >> v1:172.21.15.91:6789/0 conn(0x7fdc2c0bf8c0 legacy=0x7fdc2c158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc2affd700 1 -- 172.21.15.203:0/1929403353 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c0bc150 con 0x7fdc2c0bf4e0 2021-01-06T22:03:37.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc2affd700 1 --2- 172.21.15.203:0/1929403353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc2c0bf4e0 0x7fdc2c067d70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fdc2c0bc080 tx=0x7fdc1c006de0).ready entity=mon.1 client_cookie=5b8955123c3c7357 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:37.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc2a7fc700 1 -- 172.21.15.203:0/1929403353 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c009750 con 0x7fdc2c0bf4e0 2021-01-06T22:03:37.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc2a7fc700 1 -- 172.21.15.203:0/1929403353 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c0098b0 con 0x7fdc2c0bf4e0 2021-01-06T22:03:37.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.003+0000 7fdc2a7fc700 1 -- 172.21.15.203:0/1929403353 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c009c30 con 0x7fdc2c0bf4e0 2021-01-06T22:03:37.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.007+0000 7fdc319d3700 1 -- 172.21.15.203:0/1929403353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc2c0bf4e0 msgr2=0x7fdc2c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.007+0000 7fdc319d3700 1 --2- 172.21.15.203:0/1929403353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc2c0bf4e0 0x7fdc2c067d70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fdc2c0bc080 tx=0x7fdc1c006de0).stop 2021-01-06T22:03:37.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.007+0000 7fdc319d3700 1 -- 172.21.15.203:0/1929403353 shutdown_connections 2021-01-06T22:03:37.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.007+0000 7fdc319d3700 1 --2- 172.21.15.203:0/1929403353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc2c0bf4e0 0x7fdc2c067d70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.007+0000 7fdc319d3700 1 -- 172.21.15.203:0/1929403353 >> 172.21.15.203:0/1929403353 conn(0x7fdc2c056cd0 msgr2=0x7fdc2c0602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:37.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.007+0000 7fdc319d3700 1 -- 172.21.15.203:0/1929403353 shutdown_connections 2021-01-06T22:03:37.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.007+0000 7fdc319d3700 1 -- 172.21.15.203:0/1929403353 wait complete. 2021-01-06T22:03:37.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.007+0000 7fdc319d3700 1 Processor -- start 2021-01-06T22:03:37.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.007+0000 7fdc319d3700 1 -- start start 2021-01-06T22:03:37.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc319d3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc2c0609c0 0x7fdc2c154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc319d3700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc2c0bf4e0 0x7fdc2c150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc319d3700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc2c0bf8c0 0x7fdc2c150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc319d3700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c154710 con 0x7fdc2c0bf8c0 2021-01-06T22:03:37.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc319d3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c1512f0 con 0x7fdc2c0609c0 2021-01-06T22:03:37.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc319d3700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c14cff0 con 0x7fdc2c0bf4e0 2021-01-06T22:03:37.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc2affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc2c0609c0 0x7fdc2c154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc2affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc2c0609c0 0x7fdc2c154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48404/0 (socket says 172.21.15.203:48404) 2021-01-06T22:03:37.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc2affd700 1 -- 172.21.15.203:0/1833226471 learned_addr learned my addr 172.21.15.203:0/1833226471 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:37.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc23fff700 1 --2- 172.21.15.203:0/1833226471 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc2c0bf4e0 0x7fdc2c150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc2b7fe700 1 --2- 172.21.15.203:0/1833226471 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc2c0bf8c0 0x7fdc2c150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc2affd700 1 -- 172.21.15.203:0/1833226471 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc2c0bf4e0 msgr2=0x7fdc2c150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc2affd700 1 --2- 172.21.15.203:0/1833226471 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc2c0bf4e0 0x7fdc2c150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc2affd700 1 -- 172.21.15.203:0/1833226471 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc2c0bf8c0 msgr2=0x7fdc2c150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc2affd700 1 --2- 172.21.15.203:0/1833226471 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc2c0bf8c0 0x7fdc2c150e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc2affd700 1 -- 172.21.15.203:0/1833226471 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c14d290 con 0x7fdc2c0609c0 2021-01-06T22:03:37.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc2b7fe700 1 --2- 172.21.15.203:0/1833226471 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc2c0bf8c0 0x7fdc2c150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:37.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc2affd700 1 --2- 172.21.15.203:0/1833226471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc2c0609c0 0x7fdc2c154220 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fdc1c00b130 tx=0x7fdc1c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:37.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc22ffd700 1 -- 172.21.15.203:0/1833226471 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c01b590 con 0x7fdc2c0609c0 2021-01-06T22:03:37.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc22ffd700 1 -- 172.21.15.203:0/1833226471 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c01b6f0 con 0x7fdc2c0609c0 2021-01-06T22:03:37.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc319d3700 1 -- 172.21.15.203:0/1833226471 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2c14d530 con 0x7fdc2c0609c0 2021-01-06T22:03:37.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc22ffd700 1 -- 172.21.15.203:0/1833226471 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c01ba90 con 0x7fdc2c0609c0 2021-01-06T22:03:37.021 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.011+0000 7fdc319d3700 1 -- 172.21.15.203:0/1833226471 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c150470 con 0x7fdc2c0609c0 2021-01-06T22:03:37.021 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.015+0000 7fdc319d3700 1 -- 172.21.15.203:0/1833226471 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2c0604d0 con 0x7fdc2c0609c0 2021-01-06T22:03:37.022 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.015+0000 7fdc22ffd700 1 -- 172.21.15.203:0/1833226471 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fdc1c00f070 con 0x7fdc2c0609c0 2021-01-06T22:03:37.022 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.015+0000 7fdc22ffd700 1 --2- 172.21.15.203:0/1833226471 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdc100563d0 0x7fdc10058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.022 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.015+0000 7fdc22ffd700 1 -- 172.21.15.203:0/1833226471 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fdc1c070d10 con 0x7fdc2c0609c0 2021-01-06T22:03:37.022 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.015+0000 7fdc23fff700 1 --2- 172.21.15.203:0/1833226471 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdc100563d0 0x7fdc10058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.015+0000 7fdc23fff700 1 --2- 172.21.15.203:0/1833226471 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdc100563d0 0x7fdc10058890 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fdc2c0c1f60 tx=0x7fdc14009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:37.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.023+0000 7fdc22ffd700 1 -- 172.21.15.203:0/1833226471 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fdc2c0604d0 con 0x7fdc2c0609c0 2021-01-06T22:03:37.179 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.175+0000 7fdc319d3700 1 -- 172.21.15.203:0/1833226471 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdc2c0604d0 con 0x7fdc2c0609c0 2021-01-06T22:03:37.179 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.179+0000 7fdc22ffd700 1 -- 172.21.15.203:0/1833226471 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdc2c0604d0 con 0x7fdc2c0609c0 2021-01-06T22:03:37.180 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:03:37.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.179+0000 7fdc319d3700 1 -- 172.21.15.203:0/1833226471 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdc100563d0 msgr2=0x7fdc10058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.179+0000 7fdc319d3700 1 --2- 172.21.15.203:0/1833226471 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdc100563d0 0x7fdc10058890 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fdc2c0c1f60 tx=0x7fdc14009040).stop 2021-01-06T22:03:37.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.179+0000 7fdc319d3700 1 -- 172.21.15.203:0/1833226471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc2c0609c0 msgr2=0x7fdc2c154220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.179+0000 7fdc319d3700 1 --2- 172.21.15.203:0/1833226471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc2c0609c0 0x7fdc2c154220 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fdc1c00b130 tx=0x7fdc1c016000).stop 2021-01-06T22:03:37.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.179+0000 7fdc319d3700 1 -- 172.21.15.203:0/1833226471 shutdown_connections 2021-01-06T22:03:37.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.179+0000 7fdc319d3700 1 --2- 172.21.15.203:0/1833226471 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdc100563d0 0x7fdc10058890 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.179+0000 7fdc319d3700 1 --2- 172.21.15.203:0/1833226471 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc2c0bf8c0 0x7fdc2c150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.179+0000 7fdc319d3700 1 --2- 172.21.15.203:0/1833226471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc2c0609c0 0x7fdc2c154220 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.179+0000 7fdc319d3700 1 --2- 172.21.15.203:0/1833226471 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc2c0bf4e0 0x7fdc2c150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.179+0000 7fdc319d3700 1 -- 172.21.15.203:0/1833226471 >> 172.21.15.203:0/1833226471 conn(0x7fdc2c056cd0 msgr2=0x7fdc2c0bdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:37.185 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.179+0000 7fdc319d3700 1 -- 172.21.15.203:0/1833226471 shutdown_connections 2021-01-06T22:03:37.185 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.179+0000 7fdc319d3700 1 -- 172.21.15.203:0/1833226471 wait complete. 2021-01-06T22:03:37.194 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-01-06T22:03:37.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f543a385700 1 Processor -- start 2021-01-06T22:03:37.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f543a385700 1 -- start start 2021-01-06T22:03:37.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f543a385700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5434064380 0x7f54340c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f543a385700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f54341478c0 0x7f54340bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f543a385700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54340bb2d0 con 0x7f543414e310 2021-01-06T22:03:37.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f543a385700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543414a560 con 0x7f54341478c0 2021-01-06T22:03:37.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f543a385700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543414a800 con 0x7f5434064380 2021-01-06T22:03:37.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f5439383700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5434064380 0x7f54340c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f5439383700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5434064380 0x7f54340c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48412/0 (socket says 172.21.15.203:48412) 2021-01-06T22:03:37.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f5439383700 1 -- 172.21.15.203:0/635652960 learned_addr learned my addr 172.21.15.203:0/635652960 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:37.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f5439b84700 1 --2- 172.21.15.203:0/635652960 >> v2:172.21.15.91:3300/0 conn(0x7f54341478c0 0x7f54340bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f5439383700 1 -- 172.21.15.203:0/635652960 >> v2:172.21.15.91:3300/0 conn(0x7f54341478c0 msgr2=0x7f54340bb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:37.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f5439383700 1 --2- 172.21.15.203:0/635652960 >> v2:172.21.15.91:3300/0 conn(0x7f54341478c0 0x7f54340bb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f5439383700 1 -- 172.21.15.203:0/635652960 >> v1:172.21.15.91:6789/0 conn(0x7f543414e310 legacy=0x7f54341471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f5439383700 1 -- 172.21.15.203:0/635652960 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543414aaa0 con 0x7f5434064380 2021-01-06T22:03:37.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f5439383700 1 --2- 172.21.15.203:0/635652960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5434064380 0x7f54340c0fa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f54340c07d0 tx=0x7f5430006ea0).ready entity=mon.1 client_cookie=972cc3a31b0f5cba server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:37.317 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f542bfff700 1 -- 172.21.15.203:0/635652960 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430009960 con 0x7f5434064380 2021-01-06T22:03:37.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f542bfff700 1 -- 172.21.15.203:0/635652960 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5430009ac0 con 0x7f5434064380 2021-01-06T22:03:37.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f542bfff700 1 -- 172.21.15.203:0/635652960 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430009e40 con 0x7f5434064380 2021-01-06T22:03:37.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f543a385700 1 -- 172.21.15.203:0/635652960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5434064380 msgr2=0x7f54340c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f543a385700 1 --2- 172.21.15.203:0/635652960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5434064380 0x7f54340c0fa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f54340c07d0 tx=0x7f5430006ea0).stop 2021-01-06T22:03:37.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f543a385700 1 -- 172.21.15.203:0/635652960 shutdown_connections 2021-01-06T22:03:37.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f543a385700 1 --2- 172.21.15.203:0/635652960 >> v2:172.21.15.91:3300/0 conn(0x7f54341478c0 0x7f54340bb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f543a385700 1 --2- 172.21.15.203:0/635652960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5434064380 0x7f54340c0fa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f543a385700 1 -- 172.21.15.203:0/635652960 >> 172.21.15.203:0/635652960 conn(0x7f54340667b0 msgr2=0x7f54340c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:37.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f543a385700 1 -- 172.21.15.203:0/635652960 shutdown_connections 2021-01-06T22:03:37.320 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.311+0000 7f543a385700 1 -- 172.21.15.203:0/635652960 wait complete. 2021-01-06T22:03:37.320 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.315+0000 7f543a385700 1 Processor -- start 2021-01-06T22:03:37.320 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.315+0000 7f543a385700 1 -- start start 2021-01-06T22:03:37.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.315+0000 7f543a385700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5434064380 0x7f54340b9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.315+0000 7f543a385700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54341478c0 0x7f54340bf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.315+0000 7f543a385700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f543414e310 0x7f54340b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.315+0000 7f543a385700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54340b5eb0 con 0x7f54341478c0 2021-01-06T22:03:37.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.315+0000 7f543a385700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5434142fb0 con 0x7f543414e310 2021-01-06T22:03:37.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.315+0000 7f543a385700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5434143250 con 0x7f5434064380 2021-01-06T22:03:37.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.315+0000 7f5439b84700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f543414e310 0x7f54340b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.315+0000 7f5439b84700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f543414e310 0x7f54340b6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48420/0 (socket says 172.21.15.203:48420) 2021-01-06T22:03:37.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.315+0000 7f5439b84700 1 -- 172.21.15.203:0/2290403110 learned_addr learned my addr 172.21.15.203:0/2290403110 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:37.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.315+0000 7f5438b82700 1 --2- 172.21.15.203:0/2290403110 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54341478c0 0x7f54340bf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.315+0000 7f5439383700 1 --2- 172.21.15.203:0/2290403110 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5434064380 0x7f54340b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.319+0000 7f5439b84700 1 -- 172.21.15.203:0/2290403110 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5434064380 msgr2=0x7f54340b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.319+0000 7f5439b84700 1 --2- 172.21.15.203:0/2290403110 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5434064380 0x7f54340b9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.319+0000 7f5439b84700 1 -- 172.21.15.203:0/2290403110 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54341478c0 msgr2=0x7f54340bf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.319+0000 7f5439b84700 1 --2- 172.21.15.203:0/2290403110 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54341478c0 0x7f54340bf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.319+0000 7f5439b84700 1 -- 172.21.15.203:0/2290403110 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54341434f0 con 0x7f543414e310 2021-01-06T22:03:37.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.319+0000 7f5439b84700 1 --2- 172.21.15.203:0/2290403110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f543414e310 0x7f54340b6850 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f542c00ebc0 tx=0x7f542c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:37.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.319+0000 7f54297fa700 1 -- 172.21.15.203:0/2290403110 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c00c610 con 0x7f543414e310 2021-01-06T22:03:37.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.319+0000 7f543a385700 1 -- 172.21.15.203:0/2290403110 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54340b6110 con 0x7f543414e310 2021-01-06T22:03:37.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.319+0000 7f54297fa700 1 -- 172.21.15.203:0/2290403110 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542c00c770 con 0x7f543414e310 2021-01-06T22:03:37.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.319+0000 7f54297fa700 1 -- 172.21.15.203:0/2290403110 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c00cb10 con 0x7f543414e310 2021-01-06T22:03:37.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.319+0000 7f543a385700 1 -- 172.21.15.203:0/2290403110 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54340b9c50 con 0x7f543414e310 2021-01-06T22:03:37.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.323+0000 7f54297fa700 1 -- 172.21.15.203:0/2290403110 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f542c01d460 con 0x7f543414e310 2021-01-06T22:03:37.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.323+0000 7f54297fa700 1 --2- 172.21.15.203:0/2290403110 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f54140563d0 0x7f5414058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.323+0000 7f54297fa700 1 -- 172.21.15.203:0/2290403110 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f542c06f430 con 0x7f543414e310 2021-01-06T22:03:37.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.323+0000 7f5439383700 1 --2- 172.21.15.203:0/2290403110 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f54140563d0 0x7f5414058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.323+0000 7f543a385700 1 -- 172.21.15.203:0/2290403110 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54340450a0 con 0x7f543414e310 2021-01-06T22:03:37.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.323+0000 7f5439383700 1 --2- 172.21.15.203:0/2290403110 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f54140563d0 0x7f5414058890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f5430006ee0 tx=0x7f5430016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:37.330 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.327+0000 7f54297fa700 1 -- 172.21.15.203:0/2290403110 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f542c044f80 con 0x7f543414e310 2021-01-06T22:03:37.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.483+0000 7f543a385700 1 -- 172.21.15.203:0/2290403110 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f543405ca60 con 0x7f543414e310 2021-01-06T22:03:37.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.483+0000 7f54297fa700 1 -- 172.21.15.203:0/2290403110 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+143 (secure 0 0 0) 0x7f542c0485a0 con 0x7f543414e310 2021-01-06T22:03:37.485 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:37.486 INFO:teuthology.orchestra.run.smithi203.stdout:[ 2021-01-06T22:03:37.486 INFO:teuthology.orchestra.run.smithi203.stdout: { 2021-01-06T22:03:37.486 INFO:teuthology.orchestra.run.smithi203.stdout: "poolnum": 1, 2021-01-06T22:03:37.486 INFO:teuthology.orchestra.run.smithi203.stdout: "poolname": "device_health_metrics" 2021-01-06T22:03:37.487 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2021-01-06T22:03:37.487 INFO:teuthology.orchestra.run.smithi203.stdout: { 2021-01-06T22:03:37.487 INFO:teuthology.orchestra.run.smithi203.stdout: "poolnum": 2, 2021-01-06T22:03:37.487 INFO:teuthology.orchestra.run.smithi203.stdout: "poolname": "rbd" 2021-01-06T22:03:37.487 INFO:teuthology.orchestra.run.smithi203.stdout: } 2021-01-06T22:03:37.488 INFO:teuthology.orchestra.run.smithi203.stdout:] 2021-01-06T22:03:37.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.483+0000 7f543a385700 1 -- 172.21.15.203:0/2290403110 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f54140563d0 msgr2=0x7f5414058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.483+0000 7f543a385700 1 --2- 172.21.15.203:0/2290403110 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f54140563d0 0x7f5414058890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f5430006ee0 tx=0x7f5430016000).stop 2021-01-06T22:03:37.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.483+0000 7f543a385700 1 -- 172.21.15.203:0/2290403110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f543414e310 msgr2=0x7f54340b6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.483+0000 7f543a385700 1 --2- 172.21.15.203:0/2290403110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f543414e310 0x7f54340b6850 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f542c00ebc0 tx=0x7f542c008040).stop 2021-01-06T22:03:37.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.483+0000 7f543a385700 1 -- 172.21.15.203:0/2290403110 shutdown_connections 2021-01-06T22:03:37.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.483+0000 7f543a385700 1 --2- 172.21.15.203:0/2290403110 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f54140563d0 0x7f5414058890 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.483+0000 7f543a385700 1 --2- 172.21.15.203:0/2290403110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f543414e310 0x7f54340b6850 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.483+0000 7f543a385700 1 --2- 172.21.15.203:0/2290403110 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5434064380 0x7f54340b9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.483+0000 7f543a385700 1 --2- 172.21.15.203:0/2290403110 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54341478c0 0x7f54340bf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.483+0000 7f543a385700 1 -- 172.21.15.203:0/2290403110 >> 172.21.15.203:0/2290403110 conn(0x7f54340667b0 msgr2=0x7f54340be760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:37.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.487+0000 7f543a385700 1 -- 172.21.15.203:0/2290403110 shutdown_connections 2021-01-06T22:03:37.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.487+0000 7f543a385700 1 -- 172.21.15.203:0/2290403110 wait complete. 2021-01-06T22:03:37.501 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-01-06T22:03:37.501 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-01-06T22:03:37.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.627+0000 7f1a48b1c700 1 Processor -- start 2021-01-06T22:03:37.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.627+0000 7f1a48b1c700 1 -- start start 2021-01-06T22:03:37.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a48b1c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a440bad90 0x7f1a44147800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a48b1c700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4414b7e0 con 0x7f1a440be2f0 2021-01-06T22:03:37.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a48b1c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a440bb3b0 con 0x7f1a440be6d0 2021-01-06T22:03:37.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a48b1c700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4414a8c0 con 0x7f1a440bad90 2021-01-06T22:03:37.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a42ffd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f1a440be6d0 0x7f1a440c1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47022/0 (socket says 172.21.15.203:47022) 2021-01-06T22:03:37.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a42ffd700 1 -- 172.21.15.203:0/2823407200 learned_addr learned my addr 172.21.15.203:0/2823407200 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:37.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 -- 172.21.15.203:0/2823407200 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1094262495 0 0) 0x7f1a440bb3b0 con 0x7f1a440be6d0 2021-01-06T22:03:37.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a43fff700 1 --2- 172.21.15.203:0/2823407200 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a440bad90 0x7f1a44147800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 -- 172.21.15.203:0/2823407200 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a30003230 con 0x7f1a440be6d0 2021-01-06T22:03:37.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 -- 172.21.15.203:0/2823407200 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3561171040 0 0) 0x7f1a30003230 con 0x7f1a440be6d0 2021-01-06T22:03:37.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 -- 172.21.15.203:0/2823407200 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a440bad90 msgr2=0x7f1a44147800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:37.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 --2- 172.21.15.203:0/2823407200 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a440bad90 0x7f1a44147800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 -- 172.21.15.203:0/2823407200 >> v1:172.21.15.91:6789/0 conn(0x7f1a440be2f0 legacy=0x7f1a44065560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 -- 172.21.15.203:0/2823407200 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4414ab60 con 0x7f1a440be6d0 2021-01-06T22:03:37.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 -- 172.21.15.203:0/2823407200 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f1a38001860 con 0x7f1a440be6d0 2021-01-06T22:03:37.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 -- 172.21.15.203:0/2823407200 >> v1:172.21.15.203:6789/0 conn(0x7f1a440be6d0 legacy=0x7f1a440c1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 --2- 172.21.15.203:0/2823407200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a30003720 0x7f1a30003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 --2- 172.21.15.203:0/2823407200 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a300044d0 0x7f1a30004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 --2- 172.21.15.203:0/2823407200 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a440bad90 0x7f1a30004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 -- 172.21.15.203:0/2823407200 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4414a8c0 con 0x7f1a300044d0 2021-01-06T22:03:37.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 -- 172.21.15.203:0/2823407200 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a30004290 con 0x7f1a30003720 2021-01-06T22:03:37.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 -- 172.21.15.203:0/2823407200 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3000d930 con 0x7f1a440bad90 2021-01-06T22:03:37.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a437fe700 1 --2- 172.21.15.203:0/2823407200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a30003720 0x7f1a30003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a43fff700 1 --2- 172.21.15.203:0/2823407200 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a440bad90 0x7f1a30004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a42ffd700 1 --2- 172.21.15.203:0/2823407200 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a300044d0 0x7f1a30004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a437fe700 1 -- 172.21.15.203:0/2823407200 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a440bad90 msgr2=0x7f1a30004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:37.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a437fe700 1 --2- 172.21.15.203:0/2823407200 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a440bad90 0x7f1a30004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a437fe700 1 -- 172.21.15.203:0/2823407200 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a300044d0 msgr2=0x7f1a30004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:37.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a437fe700 1 --2- 172.21.15.203:0/2823407200 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a300044d0 0x7f1a30004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a437fe700 1 -- 172.21.15.203:0/2823407200 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a30003230 con 0x7f1a30003720 2021-01-06T22:03:37.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a42ffd700 1 --2- 172.21.15.203:0/2823407200 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a300044d0 0x7f1a30004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:37.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a437fe700 1 --2- 172.21.15.203:0/2823407200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a30003720 0x7f1a30003b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f1a34001920 tx=0x7f1a34007b80).ready entity=mon.1 client_cookie=ccf99f86fccfa20 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:37.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 -- 172.21.15.203:0/2823407200 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34007020 con 0x7f1a30003720 2021-01-06T22:03:37.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 -- 172.21.15.203:0/2823407200 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a34007180 con 0x7f1a30003720 2021-01-06T22:03:37.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.631+0000 7f1a427fc700 1 -- 172.21.15.203:0/2823407200 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3400c830 con 0x7f1a30003720 2021-01-06T22:03:37.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.635+0000 7f1a48b1c700 1 -- 172.21.15.203:0/2823407200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a30003720 msgr2=0x7f1a30003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.635+0000 7f1a48b1c700 1 --2- 172.21.15.203:0/2823407200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a30003720 0x7f1a30003b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f1a34001920 tx=0x7f1a34007b80).stop 2021-01-06T22:03:37.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.635+0000 7f1a43fff700 1 -- 172.21.15.203:0/2823407200 reap_dead start 2021-01-06T22:03:37.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.635+0000 7f1a48b1c700 1 -- 172.21.15.203:0/2823407200 shutdown_connections 2021-01-06T22:03:37.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.635+0000 7f1a48b1c700 1 -- 172.21.15.203:0/2823407200 >> 172.21.15.203:0/2823407200 conn(0x7f1a44067eb0 msgr2=0x7f1a440682c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:37.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.635+0000 7f1a48b1c700 1 -- 172.21.15.203:0/2823407200 shutdown_connections 2021-01-06T22:03:37.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.635+0000 7f1a48b1c700 1 -- 172.21.15.203:0/2823407200 wait complete. 2021-01-06T22:03:37.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.635+0000 7f1a48b1c700 1 Processor -- start 2021-01-06T22:03:37.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.635+0000 7f1a48b1c700 1 -- start start 2021-01-06T22:03:37.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.635+0000 7f1a48b1c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a30003720 0x7f1a44141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a48b1c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a44060dc0 0x7f1a440bfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.649 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a48b1c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a44060720 0x7f1a440c0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.649 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a48b1c700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a44143850 con 0x7f1a44060720 2021-01-06T22:03:37.649 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a48b1c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a441592c0 con 0x7f1a44060dc0 2021-01-06T22:03:37.649 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a48b1c700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a44159560 con 0x7f1a30003720 2021-01-06T22:03:37.650 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a42ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a44060dc0 0x7f1a440bfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.650 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a42ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a44060dc0 0x7f1a440bfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48442/0 (socket says 172.21.15.203:48442) 2021-01-06T22:03:37.650 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a42ffd700 1 -- 172.21.15.203:0/2193343877 learned_addr learned my addr 172.21.15.203:0/2193343877 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:37.650 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a437fe700 1 --2- 172.21.15.203:0/2193343877 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a30003720 0x7f1a44141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.651 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a43fff700 1 --2- 172.21.15.203:0/2193343877 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a44060720 0x7f1a440c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.651 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a42ffd700 1 -- 172.21.15.203:0/2193343877 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a30003720 msgr2=0x7f1a44141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.651 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a42ffd700 1 --2- 172.21.15.203:0/2193343877 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a30003720 0x7f1a44141cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.651 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a42ffd700 1 -- 172.21.15.203:0/2193343877 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a44060720 msgr2=0x7f1a440c0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:37.652 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a42ffd700 1 --2- 172.21.15.203:0/2193343877 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a44060720 0x7f1a440c0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:37.652 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a42ffd700 1 -- 172.21.15.203:0/2193343877 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a44159800 con 0x7f1a44060dc0 2021-01-06T22:03:37.652 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a42ffd700 1 --2- 172.21.15.203:0/2193343877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a44060dc0 0x7f1a440bfb40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f1a38001150 tx=0x7f1a380010e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:37.652 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a2b7fe700 1 -- 172.21.15.203:0/2193343877 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3800eb20 con 0x7f1a44060dc0 2021-01-06T22:03:37.652 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a2b7fe700 1 -- 172.21.15.203:0/2193343877 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3800ec80 con 0x7f1a44060dc0 2021-01-06T22:03:37.653 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a48b1c700 1 -- 172.21.15.203:0/2193343877 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a44159aa0 con 0x7f1a44060dc0 2021-01-06T22:03:37.653 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a2b7fe700 1 -- 172.21.15.203:0/2193343877 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38014430 con 0x7f1a44060dc0 2021-01-06T22:03:37.653 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a48b1c700 1 -- 172.21.15.203:0/2193343877 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a44149a40 con 0x7f1a44060dc0 2021-01-06T22:03:37.653 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a48b1c700 1 -- 172.21.15.203:0/2193343877 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a4415a0e0 con 0x7f1a44060dc0 2021-01-06T22:03:37.654 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.639+0000 7f1a2b7fe700 1 -- 172.21.15.203:0/2193343877 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f1a38014820 con 0x7f1a44060dc0 2021-01-06T22:03:37.654 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.643+0000 7f1a2b7fe700 1 --2- 172.21.15.203:0/2193343877 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1a240563d0 0x7f1a24058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:37.654 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.643+0000 7f1a2b7fe700 1 -- 172.21.15.203:0/2193343877 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f1a3806f300 con 0x7f1a44060dc0 2021-01-06T22:03:37.654 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.643+0000 7f1a437fe700 1 --2- 172.21.15.203:0/2193343877 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1a240563d0 0x7f1a24058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:37.655 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.643+0000 7f1a437fe700 1 --2- 172.21.15.203:0/2193343877 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1a240563d0 0x7f1a24058890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f1a34000fd0 tx=0x7f1a34015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:37.656 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.651+0000 7f1a2b7fe700 1 -- 172.21.15.203:0/2193343877 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f1a4415a0e0 con 0x7f1a44060dc0 2021-01-06T22:03:37.800 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:37.799+0000 7f1a48b1c700 1 -- 172.21.15.203:0/2193343877 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f1a4415a0e0 con 0x7f1a44060dc0 2021-01-06T22:03:38.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.347+0000 7f1a2b7fe700 1 -- 172.21.15.203:0/2193343877 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v20) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1a4415a0e0 con 0x7f1a44060dc0 2021-01-06T22:03:38.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.351+0000 7f1a48b1c700 1 -- 172.21.15.203:0/2193343877 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1a240563d0 msgr2=0x7f1a24058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:38.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.351+0000 7f1a48b1c700 1 --2- 172.21.15.203:0/2193343877 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1a240563d0 0x7f1a24058890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f1a34000fd0 tx=0x7f1a34015000).stop 2021-01-06T22:03:38.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.351+0000 7f1a48b1c700 1 -- 172.21.15.203:0/2193343877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a44060dc0 msgr2=0x7f1a440bfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:38.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.351+0000 7f1a48b1c700 1 --2- 172.21.15.203:0/2193343877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a44060dc0 0x7f1a440bfb40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f1a38001150 tx=0x7f1a380010e0).stop 2021-01-06T22:03:38.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.351+0000 7f1a48b1c700 1 -- 172.21.15.203:0/2193343877 shutdown_connections 2021-01-06T22:03:38.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.351+0000 7f1a48b1c700 1 --2- 172.21.15.203:0/2193343877 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1a240563d0 0x7f1a24058890 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:38.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.351+0000 7f1a48b1c700 1 --2- 172.21.15.203:0/2193343877 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a44060720 0x7f1a440c0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:38.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.351+0000 7f1a48b1c700 1 --2- 172.21.15.203:0/2193343877 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a30003720 0x7f1a44141cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:38.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.351+0000 7f1a48b1c700 1 --2- 172.21.15.203:0/2193343877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a44060dc0 0x7f1a440bfb40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:38.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.351+0000 7f1a48b1c700 1 -- 172.21.15.203:0/2193343877 >> 172.21.15.203:0/2193343877 conn(0x7f1a44067eb0 msgr2=0x7f1a440bc860 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:38.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.351+0000 7f1a48b1c700 1 -- 172.21.15.203:0/2193343877 shutdown_connections 2021-01-06T22:03:38.359 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.351+0000 7f1a48b1c700 1 -- 172.21.15.203:0/2193343877 wait complete. 2021-01-06T22:03:38.359 INFO:teuthology.orchestra.run.smithi203.stderr:pool 'cephfs_metadata' created 2021-01-06T22:03:38.371 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-01-06T22:03:38.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.507+0000 7f41a8a22700 1 Processor -- start 2021-01-06T22:03:38.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.507+0000 7f41a8a22700 1 -- start start 2021-01-06T22:03:38.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.507+0000 7f41a8a22700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41a40bab60 0x7f41a4060b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:38.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.507+0000 7f41a8a22700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f41a4066ad0 0x7f41a4155640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:38.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.507+0000 7f41a8a22700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a4067a50 con 0x7f41a40baf40 2021-01-06T22:03:38.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.507+0000 7f41a8a22700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a40c02f0 con 0x7f41a4066ad0 2021-01-06T22:03:38.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.507+0000 7f41a8a22700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a40c0590 con 0x7f41a40bab60 2021-01-06T22:03:38.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.507+0000 7f41a259c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f41a40baf40 0x7f41a40b7460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47044/0 (socket says 172.21.15.203:47044) 2021-01-06T22:03:38.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.507+0000 7f41a259c700 1 -- 172.21.15.203:0/13397167 learned_addr learned my addr 172.21.15.203:0/13397167 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:38.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a359e700 1 --2- 172.21.15.203:0/13397167 >> v2:172.21.15.91:3300/0 conn(0x7f41a4066ad0 0x7f41a4155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:38.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 -- 172.21.15.203:0/13397167 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395218626 0 0) 0x7f41a4067a50 con 0x7f41a40baf40 2021-01-06T22:03:38.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a2d9d700 1 --2- 172.21.15.203:0/13397167 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41a40bab60 0x7f41a4060b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:38.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 -- 172.21.15.203:0/13397167 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4190003230 con 0x7f41a40baf40 2021-01-06T22:03:38.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 -- 172.21.15.203:0/13397167 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4211948048 0 0) 0x7f4190003230 con 0x7f41a40baf40 2021-01-06T22:03:38.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 -- 172.21.15.203:0/13397167 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41a40bab60 msgr2=0x7f41a4060b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:38.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 --2- 172.21.15.203:0/13397167 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41a40bab60 0x7f41a4060b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:38.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 -- 172.21.15.203:0/13397167 >> v2:172.21.15.91:3300/0 conn(0x7f41a4066ad0 msgr2=0x7f41a4155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:38.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 --2- 172.21.15.203:0/13397167 >> v2:172.21.15.91:3300/0 conn(0x7f41a4066ad0 0x7f41a4155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:38.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 -- 172.21.15.203:0/13397167 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a40c0830 con 0x7f41a40baf40 2021-01-06T22:03:38.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 -- 172.21.15.203:0/13397167 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f41980014e0 con 0x7f41a40baf40 2021-01-06T22:03:38.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 -- 172.21.15.203:0/13397167 >> v1:172.21.15.203:6789/0 conn(0x7f41a40baf40 legacy=0x7f41a40b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:38.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 --2- 172.21.15.203:0/13397167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4190003720 0x7f4190003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:38.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 --2- 172.21.15.203:0/13397167 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41900044d0 0x7f4190004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:38.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 --2- 172.21.15.203:0/13397167 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41a40bab60 0x7f4190004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:38.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 -- 172.21.15.203:0/13397167 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a40c02f0 con 0x7f41900044d0 2021-01-06T22:03:38.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 -- 172.21.15.203:0/13397167 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a40c0590 con 0x7f4190003720 2021-01-06T22:03:38.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 -- 172.21.15.203:0/13397167 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4190004260 con 0x7f41a40bab60 2021-01-06T22:03:38.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a2d9d700 1 --2- 172.21.15.203:0/13397167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4190003720 0x7f4190003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:38.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a2d9d700 1 -- 172.21.15.203:0/13397167 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41a40bab60 msgr2=0x7f4190004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:38.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a359e700 1 --2- 172.21.15.203:0/13397167 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41a40bab60 0x7f4190004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:38.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a2d9d700 1 --2- 172.21.15.203:0/13397167 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41a40bab60 0x7f4190004e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:38.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a2d9d700 1 -- 172.21.15.203:0/13397167 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41900044d0 msgr2=0x7f4190004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:38.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a2d9d700 1 --2- 172.21.15.203:0/13397167 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41900044d0 0x7f4190004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:03:38.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a2d9d700 1 -- 172.21.15.203:0/13397167 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4190003230 con 0x7f4190003720 2021-01-06T22:03:38.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a2d9d700 1 --2- 172.21.15.203:0/13397167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4190003720 0x7f4190003b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f419400bb90 tx=0x7f4194008040).ready entity=mon.1 client_cookie=f0066099a54a4136 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:38.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 -- 172.21.15.203:0/13397167 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419400b2b0 con 0x7f4190003720 2021-01-06T22:03:38.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 -- 172.21.15.203:0/13397167 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419400ea90 con 0x7f4190003720 2021-01-06T22:03:38.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a1d9b700 1 -- 172.21.15.203:0/13397167 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419400ee30 con 0x7f4190003720 2021-01-06T22:03:38.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a8a22700 1 -- 172.21.15.203:0/13397167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4190003720 msgr2=0x7f4190003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:38.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a8a22700 1 --2- 172.21.15.203:0/13397167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4190003720 0x7f4190003b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f419400bb90 tx=0x7f4194008040).stop 2021-01-06T22:03:38.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a359e700 1 -- 172.21.15.203:0/13397167 reap_dead start 2021-01-06T22:03:38.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a8a22700 1 -- 172.21.15.203:0/13397167 shutdown_connections 2021-01-06T22:03:38.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a8a22700 1 -- 172.21.15.203:0/13397167 >> 172.21.15.203:0/13397167 conn(0x7f41a40626e0 msgr2=0x7f41a4062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:38.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a8a22700 1 -- 172.21.15.203:0/13397167 shutdown_connections 2021-01-06T22:03:38.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a8a22700 1 -- 172.21.15.203:0/13397167 wait complete. 2021-01-06T22:03:38.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a8a22700 1 Processor -- start 2021-01-06T22:03:38.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.511+0000 7f41a8a22700 1 -- start start 2021-01-06T22:03:38.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a8a22700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4190003720 0x7f41a4151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:38.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a8a22700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a414fe80 0x7f41a41498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:38.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a8a22700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41a4150300 0x7f41a4149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:38.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a8a22700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a414c980 con 0x7f41a4150300 2021-01-06T22:03:38.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a8a22700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a4152540 con 0x7f41a414fe80 2021-01-06T22:03:38.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a8a22700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a41527e0 con 0x7f4190003720 2021-01-06T22:03:38.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a259c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a414fe80 0x7f41a41498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:38.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a259c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a414fe80 0x7f41a41498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48464/0 (socket says 172.21.15.203:48464) 2021-01-06T22:03:38.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a259c700 1 -- 172.21.15.203:0/695928817 learned_addr learned my addr 172.21.15.203:0/695928817 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:38.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a2d9d700 1 --2- 172.21.15.203:0/695928817 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4190003720 0x7f41a4151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:38.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a359e700 1 --2- 172.21.15.203:0/695928817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41a4150300 0x7f41a4149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:38.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a259c700 1 -- 172.21.15.203:0/695928817 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4190003720 msgr2=0x7f41a4151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:38.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a259c700 1 --2- 172.21.15.203:0/695928817 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4190003720 0x7f41a4151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:38.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a259c700 1 -- 172.21.15.203:0/695928817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41a4150300 msgr2=0x7f41a4149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:38.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a259c700 1 --2- 172.21.15.203:0/695928817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41a4150300 0x7f41a4149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:38.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a259c700 1 -- 172.21.15.203:0/695928817 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a4152a80 con 0x7f41a414fe80 2021-01-06T22:03:38.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a259c700 1 --2- 172.21.15.203:0/695928817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a414fe80 0x7f41a41498b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f419800ca60 tx=0x7f41a40bc6d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:38.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f418affd700 1 -- 172.21.15.203:0/695928817 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419800ce30 con 0x7f41a414fe80 2021-01-06T22:03:38.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f418affd700 1 -- 172.21.15.203:0/695928817 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4198007170 con 0x7f41a414fe80 2021-01-06T22:03:38.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f418affd700 1 -- 172.21.15.203:0/695928817 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419800a7a0 con 0x7f41a414fe80 2021-01-06T22:03:38.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a8a22700 1 -- 172.21.15.203:0/695928817 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a40b9d10 con 0x7f41a414fe80 2021-01-06T22:03:38.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a8a22700 1 -- 172.21.15.203:0/695928817 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a40ba250 con 0x7f41a414fe80 2021-01-06T22:03:38.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f418affd700 1 -- 172.21.15.203:0/695928817 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f4198010070 con 0x7f41a414fe80 2021-01-06T22:03:38.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a8a22700 1 -- 172.21.15.203:0/695928817 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a40450a0 con 0x7f41a414fe80 2021-01-06T22:03:38.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f418affd700 1 --2- 172.21.15.203:0/695928817 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41840563d0 0x7f4184058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:38.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f418affd700 1 -- 172.21.15.203:0/695928817 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f419806ebc0 con 0x7f41a414fe80 2021-01-06T22:03:38.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.515+0000 7f41a2d9d700 1 --2- 172.21.15.203:0/695928817 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41840563d0 0x7f4184058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:38.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.519+0000 7f41a2d9d700 1 --2- 172.21.15.203:0/695928817 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41840563d0 0x7f4184058890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f419400b430 tx=0x7f419401a000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:38.539 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.523+0000 7f418affd700 1 -- 172.21.15.203:0/695928817 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f41980445b0 con 0x7f41a414fe80 2021-01-06T22:03:38.675 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:38.671+0000 7f41a8a22700 1 -- 172.21.15.203:0/695928817 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f41a405ca20 con 0x7f41a414fe80 2021-01-06T22:03:39.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.359+0000 7f418affd700 1 -- 172.21.15.203:0/695928817 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v21) v1 ==== 125+0+0 (secure 0 0 0) 0x7f4198047bd0 con 0x7f41a414fe80 2021-01-06T22:03:39.368 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.367+0000 7f41a8a22700 1 -- 172.21.15.203:0/695928817 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41840563d0 msgr2=0x7f4184058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:39.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.367+0000 7f41a8a22700 1 --2- 172.21.15.203:0/695928817 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41840563d0 0x7f4184058890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f419400b430 tx=0x7f419401a000).stop 2021-01-06T22:03:39.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.367+0000 7f41a8a22700 1 -- 172.21.15.203:0/695928817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a414fe80 msgr2=0x7f41a41498b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:39.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.367+0000 7f41a8a22700 1 --2- 172.21.15.203:0/695928817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a414fe80 0x7f41a41498b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f419800ca60 tx=0x7f41a40bc6d0).stop 2021-01-06T22:03:39.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.367+0000 7f41a8a22700 1 -- 172.21.15.203:0/695928817 shutdown_connections 2021-01-06T22:03:39.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.367+0000 7f41a8a22700 1 --2- 172.21.15.203:0/695928817 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41840563d0 0x7f4184058890 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.367+0000 7f41a8a22700 1 --2- 172.21.15.203:0/695928817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41a4150300 0x7f41a4149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.367+0000 7f41a8a22700 1 --2- 172.21.15.203:0/695928817 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4190003720 0x7f41a4151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.367+0000 7f41a8a22700 1 --2- 172.21.15.203:0/695928817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a414fe80 0x7f41a41498b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.367+0000 7f41a8a22700 1 -- 172.21.15.203:0/695928817 >> 172.21.15.203:0/695928817 conn(0x7f41a40626e0 msgr2=0x7f41a41542d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:39.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.367+0000 7f41a8a22700 1 -- 172.21.15.203:0/695928817 shutdown_connections 2021-01-06T22:03:39.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.367+0000 7f41a8a22700 1 -- 172.21.15.203:0/695928817 wait complete. 2021-01-06T22:03:39.371 INFO:teuthology.orchestra.run.smithi203.stderr:pool 'cephfs_data' created 2021-01-06T22:03:39.383 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2021-01-06T22:03:39.513 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.511+0000 7ff90342e700 1 Processor -- start 2021-01-06T22:03:39.513 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.511+0000 7ff90342e700 1 -- start start 2021-01-06T22:03:39.516 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90342e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff8fc067340 0x7ff8fc068110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:39.517 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90342e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff8fc148080 0x7ff8fc0602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:39.517 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90342e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8fc0bba10 con 0x7ff8fc067720 2021-01-06T22:03:39.517 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90342e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8fc0bb230 con 0x7ff8fc148080 2021-01-06T22:03:39.517 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90342e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8fc0bb4d0 con 0x7ff8fc067340 2021-01-06T22:03:39.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90242c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff8fc067340 0x7ff8fc068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:39.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90242c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff8fc067340 0x7ff8fc068110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48468/0 (socket says 172.21.15.203:48468) 2021-01-06T22:03:39.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90242c700 1 -- 172.21.15.203:0/343610524 learned_addr learned my addr 172.21.15.203:0/343610524 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:39.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff902c2d700 1 --2- 172.21.15.203:0/343610524 >> v2:172.21.15.91:3300/0 conn(0x7ff8fc148080 0x7ff8fc0602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:39.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90242c700 1 -- 172.21.15.203:0/343610524 >> v2:172.21.15.91:3300/0 conn(0x7ff8fc148080 msgr2=0x7ff8fc0602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:39.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90242c700 1 --2- 172.21.15.203:0/343610524 >> v2:172.21.15.91:3300/0 conn(0x7ff8fc148080 0x7ff8fc0602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90242c700 1 -- 172.21.15.203:0/343610524 >> v1:172.21.15.91:6789/0 conn(0x7ff8fc067720 legacy=0x7ff8fc147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:39.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90242c700 1 -- 172.21.15.203:0/343610524 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc0bb770 con 0x7ff8fc067340 2021-01-06T22:03:39.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90242c700 1 --2- 172.21.15.203:0/343610524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8fc067340 0x7ff8fc068110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7ff8fc061ee0 tx=0x7ff8f8006ea0).ready entity=mon.1 client_cookie=5104189b83c80926 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:39.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90142a700 1 -- 172.21.15.203:0/343610524 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f80098a0 con 0x7ff8fc067340 2021-01-06T22:03:39.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90142a700 1 -- 172.21.15.203:0/343610524 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f8009a00 con 0x7ff8fc067340 2021-01-06T22:03:39.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90142a700 1 -- 172.21.15.203:0/343610524 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8009d80 con 0x7ff8fc067340 2021-01-06T22:03:39.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90342e700 1 -- 172.21.15.203:0/343610524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8fc067340 msgr2=0x7ff8fc068110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:39.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90342e700 1 --2- 172.21.15.203:0/343610524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8fc067340 0x7ff8fc068110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7ff8fc061ee0 tx=0x7ff8f8006ea0).stop 2021-01-06T22:03:39.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90342e700 1 -- 172.21.15.203:0/343610524 shutdown_connections 2021-01-06T22:03:39.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90342e700 1 --2- 172.21.15.203:0/343610524 >> v2:172.21.15.91:3300/0 conn(0x7ff8fc148080 0x7ff8fc0602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90342e700 1 --2- 172.21.15.203:0/343610524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8fc067340 0x7ff8fc068110 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90342e700 1 -- 172.21.15.203:0/343610524 >> 172.21.15.203:0/343610524 conn(0x7ff8fc0667b0 msgr2=0x7ff8fc0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:39.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90342e700 1 -- 172.21.15.203:0/343610524 shutdown_connections 2021-01-06T22:03:39.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.515+0000 7ff90342e700 1 -- 172.21.15.203:0/343610524 wait complete. 2021-01-06T22:03:39.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.519+0000 7ff90342e700 1 Processor -- start 2021-01-06T22:03:39.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.519+0000 7ff90342e700 1 -- start start 2021-01-06T22:03:39.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff90342e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff8fc067340 0x7ff8fc144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:39.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff90342e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8fc067720 0x7ff8fc15ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:39.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff90342e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8fc148080 0x7ff8fc140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:39.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff90342e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0bc950 con 0x7ff8fc148080 2021-01-06T22:03:39.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff90342e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0bbc80 con 0x7ff8fc067720 2021-01-06T22:03:39.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff90342e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc148a40 con 0x7ff8fc067340 2021-01-06T22:03:39.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff901c2b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8fc067720 0x7ff8fc15ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:39.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff901c2b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8fc067720 0x7ff8fc15ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48476/0 (socket says 172.21.15.203:48476) 2021-01-06T22:03:39.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff901c2b700 1 -- 172.21.15.203:0/398172579 learned_addr learned my addr 172.21.15.203:0/398172579 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:39.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff902c2d700 1 --2- 172.21.15.203:0/398172579 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8fc148080 0x7ff8fc140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:39.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff90242c700 1 --2- 172.21.15.203:0/398172579 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff8fc067340 0x7ff8fc144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:39.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff901c2b700 1 -- 172.21.15.203:0/398172579 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff8fc067340 msgr2=0x7ff8fc144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:39.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff901c2b700 1 --2- 172.21.15.203:0/398172579 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff8fc067340 0x7ff8fc144750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff901c2b700 1 -- 172.21.15.203:0/398172579 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8fc148080 msgr2=0x7ff8fc140040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:39.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff901c2b700 1 --2- 172.21.15.203:0/398172579 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8fc148080 0x7ff8fc140040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff901c2b700 1 -- 172.21.15.203:0/398172579 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc0bcb40 con 0x7ff8fc067720 2021-01-06T22:03:39.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff901c2b700 1 --2- 172.21.15.203:0/398172579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8fc067720 0x7ff8fc15ec30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7ff8f000c740 tx=0x7ff8f0007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:39.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff8ee7fc700 1 -- 172.21.15.203:0/398172579 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0007020 con 0x7ff8fc067720 2021-01-06T22:03:39.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff90342e700 1 -- 172.21.15.203:0/398172579 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8fc0bbeb0 con 0x7ff8fc067720 2021-01-06T22:03:39.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff8ee7fc700 1 -- 172.21.15.203:0/398172579 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f0007180 con 0x7ff8fc067720 2021-01-06T22:03:39.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff8ee7fc700 1 -- 172.21.15.203:0/398172579 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f000e560 con 0x7ff8fc067720 2021-01-06T22:03:39.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff90342e700 1 -- 172.21.15.203:0/398172579 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8fc14d440 con 0x7ff8fc067720 2021-01-06T22:03:39.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.523+0000 7ff90342e700 1 -- 172.21.15.203:0/398172579 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8fc0bac60 con 0x7ff8fc067720 2021-01-06T22:03:39.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.527+0000 7ff8ee7fc700 1 -- 172.21.15.203:0/398172579 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff8f0010030 con 0x7ff8fc067720 2021-01-06T22:03:39.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.527+0000 7ff8ee7fc700 1 --2- 172.21.15.203:0/398172579 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff8dc0563d0 0x7ff8dc058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:39.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.527+0000 7ff8ee7fc700 1 -- 172.21.15.203:0/398172579 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5957+0+0 (secure 0 0 0) 0x7ff8f006ed40 con 0x7ff8fc067720 2021-01-06T22:03:39.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.527+0000 7ff90242c700 1 --2- 172.21.15.203:0/398172579 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff8dc0563d0 0x7ff8dc058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:39.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.527+0000 7ff90242c700 1 --2- 172.21.15.203:0/398172579 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff8dc0563d0 0x7ff8dc058890 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff8f800b190 tx=0x7ff8f8016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:39.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.531+0000 7ff8ee7fc700 1 -- 172.21.15.203:0/398172579 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff8f0044630 con 0x7ff8fc067720 2021-01-06T22:03:39.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.735+0000 7ff90342e700 1 -- 172.21.15.203:0/398172579 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff8fc05ca60 con 0x7ff8fc067720 2021-01-06T22:03:39.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.747+0000 7ff8ee7fc700 1 -- 172.21.15.203:0/398172579 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff8f0047c50 con 0x7ff8fc067720 2021-01-06T22:03:39.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.751+0000 7ff90342e700 1 -- 172.21.15.203:0/398172579 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff8dc0563d0 msgr2=0x7ff8dc058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:39.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.751+0000 7ff90342e700 1 --2- 172.21.15.203:0/398172579 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff8dc0563d0 0x7ff8dc058890 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff8f800b190 tx=0x7ff8f8016000).stop 2021-01-06T22:03:39.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.751+0000 7ff90342e700 1 -- 172.21.15.203:0/398172579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8fc067720 msgr2=0x7ff8fc15ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:39.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.751+0000 7ff90342e700 1 --2- 172.21.15.203:0/398172579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8fc067720 0x7ff8fc15ec30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7ff8f000c740 tx=0x7ff8f0007c40).stop 2021-01-06T22:03:39.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.751+0000 7ff90342e700 1 -- 172.21.15.203:0/398172579 shutdown_connections 2021-01-06T22:03:39.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.751+0000 7ff90342e700 1 --2- 172.21.15.203:0/398172579 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff8dc0563d0 0x7ff8dc058890 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.751+0000 7ff90342e700 1 --2- 172.21.15.203:0/398172579 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8fc148080 0x7ff8fc140040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.751+0000 7ff90342e700 1 --2- 172.21.15.203:0/398172579 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff8fc067340 0x7ff8fc144750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.751+0000 7ff90342e700 1 --2- 172.21.15.203:0/398172579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8fc067720 0x7ff8fc15ec30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.751+0000 7ff90342e700 1 -- 172.21.15.203:0/398172579 >> 172.21.15.203:0/398172579 conn(0x7ff8fc0667b0 msgr2=0x7ff8fc1423d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:39.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.751+0000 7ff90342e700 1 -- 172.21.15.203:0/398172579 shutdown_connections 2021-01-06T22:03:39.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.751+0000 7ff90342e700 1 -- 172.21.15.203:0/398172579 wait complete. 2021-01-06T22:03:39.757 INFO:teuthology.orchestra.run.smithi203.stderr:new fs with metadata pool 3 and data pool 4 2021-01-06T22:03:39.772 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-06T22:03:39.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f36454c6700 1 Processor -- start 2021-01-06T22:03:39.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f36454c6700 1 -- start start 2021-01-06T22:03:39.886 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f36454c6700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f36400a5e50 0x7f36400ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:39.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f36454c6700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f36400bb7d0 0x7f364014eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:39.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f36454c6700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36400646b0 con 0x7f36400bb3f0 2021-01-06T22:03:39.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f36454c6700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36400670c0 con 0x7f36400bb7d0 2021-01-06T22:03:39.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f36454c6700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3640064450 con 0x7f36400a5e50 2021-01-06T22:03:39.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f363f7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f36400a5e50 0x7f36400ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:39.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f363ffff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f36400bb3f0 0x7f36400669b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47078/0 (socket says 172.21.15.203:47078) 2021-01-06T22:03:39.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f363ffff700 1 -- 172.21.15.203:0/3283426204 learned_addr learned my addr 172.21.15.203:0/3283426204 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:39.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f3644cc5700 1 --2- 172.21.15.203:0/3283426204 >> v2:172.21.15.91:3300/0 conn(0x7f36400bb7d0 0x7f364014eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:39.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f363f7fe700 1 -- 172.21.15.203:0/3283426204 >> v2:172.21.15.91:3300/0 conn(0x7f36400bb7d0 msgr2=0x7f364014eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:39.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f363f7fe700 1 --2- 172.21.15.203:0/3283426204 >> v2:172.21.15.91:3300/0 conn(0x7f36400bb7d0 0x7f364014eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f363f7fe700 1 -- 172.21.15.203:0/3283426204 >> v1:172.21.15.203:6789/0 conn(0x7f36400bb3f0 legacy=0x7f36400669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:39.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f363effd700 1 -- 172.21.15.203:0/3283426204 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134030266 0 0) 0x7f36400646b0 con 0x7f36400bb3f0 2021-01-06T22:03:39.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f363f7fe700 1 -- 172.21.15.203:0/3283426204 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3640152cc0 con 0x7f36400a5e50 2021-01-06T22:03:39.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f363f7fe700 1 --2- 172.21.15.203:0/3283426204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36400a5e50 0x7f36400ba9d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f3640064c50 tx=0x7f36280091d0).ready entity=mon.1 client_cookie=40ce9f32f732252b server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:39.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f363effd700 1 -- 172.21.15.203:0/3283426204 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362800bbc0 con 0x7f36400a5e50 2021-01-06T22:03:39.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f363effd700 1 -- 172.21.15.203:0/3283426204 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362800bd20 con 0x7f36400a5e50 2021-01-06T22:03:39.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.883+0000 7f363effd700 1 -- 172.21.15.203:0/3283426204 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628011610 con 0x7f36400a5e50 2021-01-06T22:03:39.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f36454c6700 1 -- 172.21.15.203:0/3283426204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36400a5e50 msgr2=0x7f36400ba9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:39.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f36454c6700 1 --2- 172.21.15.203:0/3283426204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36400a5e50 0x7f36400ba9d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f3640064c50 tx=0x7f36280091d0).stop 2021-01-06T22:03:39.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f36454c6700 1 -- 172.21.15.203:0/3283426204 shutdown_connections 2021-01-06T22:03:39.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f36454c6700 1 --2- 172.21.15.203:0/3283426204 >> v2:172.21.15.91:3300/0 conn(0x7f36400bb7d0 0x7f364014eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f36454c6700 1 --2- 172.21.15.203:0/3283426204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36400a5e50 0x7f36400ba9d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f36454c6700 1 -- 172.21.15.203:0/3283426204 >> 172.21.15.203:0/3283426204 conn(0x7f3640061530 msgr2=0x7f3640063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:39.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f36454c6700 1 -- 172.21.15.203:0/3283426204 shutdown_connections 2021-01-06T22:03:39.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f36454c6700 1 -- 172.21.15.203:0/3283426204 wait complete. 2021-01-06T22:03:39.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f36454c6700 1 Processor -- start 2021-01-06T22:03:39.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f36454c6700 1 -- start start 2021-01-06T22:03:39.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f36454c6700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36400a5e50 0x7f36401622d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:39.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f36454c6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36400bb3f0 0x7f36400be1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:39.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f36454c6700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f36400bb7d0 0x7f36400b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:39.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f36454c6700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36400b85a0 con 0x7f36400a5e50 2021-01-06T22:03:39.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f36454c6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3640153a50 con 0x7f36400bb3f0 2021-01-06T22:03:39.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f36454c6700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36400a1be0 con 0x7f36400bb7d0 2021-01-06T22:03:39.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f363f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36400bb3f0 0x7f36400be1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:39.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f363f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36400bb3f0 0x7f36400be1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48490/0 (socket says 172.21.15.203:48490) 2021-01-06T22:03:39.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f363f7fe700 1 -- 172.21.15.203:0/1620520385 learned_addr learned my addr 172.21.15.203:0/1620520385 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:39.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f3644cc5700 1 --2- 172.21.15.203:0/1620520385 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f36400bb7d0 0x7f36400b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:39.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f363ffff700 1 --2- 172.21.15.203:0/1620520385 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36400a5e50 0x7f36401622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:39.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f363f7fe700 1 -- 172.21.15.203:0/1620520385 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f36400bb7d0 msgr2=0x7f36400b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:39.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f363f7fe700 1 --2- 172.21.15.203:0/1620520385 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f36400bb7d0 0x7f36400b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f363f7fe700 1 -- 172.21.15.203:0/1620520385 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36400a5e50 msgr2=0x7f36401622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:39.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f363f7fe700 1 --2- 172.21.15.203:0/1620520385 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36400a5e50 0x7f36401622d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:39.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f363f7fe700 1 -- 172.21.15.203:0/1620520385 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36400a1e80 con 0x7f36400bb3f0 2021-01-06T22:03:39.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f363ffff700 1 --2- 172.21.15.203:0/1620520385 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36400a5e50 0x7f36401622d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:03:39.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f363f7fe700 1 --2- 172.21.15.203:0/1620520385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36400bb3f0 0x7f36400be1e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f3628011000 tx=0x7f3628017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:39.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.887+0000 7f3637fff700 1 -- 172.21.15.203:0/1620520385 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362801a640 con 0x7f36400bb3f0 2021-01-06T22:03:39.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.891+0000 7f3637fff700 1 -- 172.21.15.203:0/1620520385 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362801a7a0 con 0x7f36400bb3f0 2021-01-06T22:03:39.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.891+0000 7f3637fff700 1 -- 172.21.15.203:0/1620520385 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362801a930 con 0x7f36400bb3f0 2021-01-06T22:03:39.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.891+0000 7f36454c6700 1 -- 172.21.15.203:0/1620520385 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36400a2120 con 0x7f36400bb3f0 2021-01-06T22:03:39.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.891+0000 7f36454c6700 1 -- 172.21.15.203:0/1620520385 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36400be6d0 con 0x7f36400bb3f0 2021-01-06T22:03:39.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.891+0000 7f36454c6700 1 -- 172.21.15.203:0/1620520385 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36400450a0 con 0x7f36400bb3f0 2021-01-06T22:03:39.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.891+0000 7f3637fff700 1 -- 172.21.15.203:0/1620520385 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f362800f070 con 0x7f36400bb3f0 2021-01-06T22:03:39.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.891+0000 7f3637fff700 1 --2- 172.21.15.203:0/1620520385 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f36180563d0 0x7f3618058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:39.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.891+0000 7f3637fff700 1 -- 172.21.15.203:0/1620520385 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3628071530 con 0x7f36400bb3f0 2021-01-06T22:03:39.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.891+0000 7f363ffff700 1 --2- 172.21.15.203:0/1620520385 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f36180563d0 0x7f3618058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:39.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.891+0000 7f363ffff700 1 -- 172.21.15.203:0/1620520385 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f36180563d0 msgr2=0x7f3618058890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2021-01-06T22:03:39.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.891+0000 7f363ffff700 1 -- 172.21.15.203:0/1620520385 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f36180563d0 msgr2=0x7f3618058890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2021-01-06T22:03:39.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.891+0000 7f363ffff700 1 --2- 172.21.15.203:0/1620520385 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f36180563d0 0x7f3618058890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:03:39.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.891+0000 7f363ffff700 1 --2- 172.21.15.203:0/1620520385 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f36180563d0 0x7f3618058890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-01-06T22:03:39.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:39.899+0000 7f3637fff700 1 -- 172.21.15.203:0/1620520385 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f3628046db0 con 0x7f36400bb3f0 2021-01-06T22:03:40.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.055+0000 7f36454c6700 1 -- 172.21.15.203:0/1620520385 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f364005ca60 con 0x7f36400bb3f0 2021-01-06T22:03:40.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.055+0000 7f3637fff700 1 -- 172.21.15.203:0/1620520385 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16839 (secure 0 0 0) 0x7f362804a3d0 con 0x7f36400bb3f0 2021-01-06T22:03:40.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.059+0000 7f36454c6700 1 -- 172.21.15.203:0/1620520385 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f36180563d0 msgr2=0x7f3618058890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2021-01-06T22:03:40.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.059+0000 7f36454c6700 1 --2- 172.21.15.203:0/1620520385 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f36180563d0 0x7f3618058890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.059+0000 7f36454c6700 1 -- 172.21.15.203:0/1620520385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36400bb3f0 msgr2=0x7f36400be1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:40.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.059+0000 7f36454c6700 1 --2- 172.21.15.203:0/1620520385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36400bb3f0 0x7f36400be1e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f3628011000 tx=0x7f3628017000).stop 2021-01-06T22:03:40.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.059+0000 7f36454c6700 1 -- 172.21.15.203:0/1620520385 shutdown_connections 2021-01-06T22:03:40.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.059+0000 7f36454c6700 1 --2- 172.21.15.203:0/1620520385 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f36180563d0 0x7f3618058890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.059+0000 7f36454c6700 1 --2- 172.21.15.203:0/1620520385 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f36400bb7d0 0x7f36400b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.059+0000 7f36454c6700 1 --2- 172.21.15.203:0/1620520385 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36400a5e50 0x7f36401622d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.059+0000 7f36454c6700 1 --2- 172.21.15.203:0/1620520385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36400bb3f0 0x7f36400be1e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.059+0000 7f36454c6700 1 -- 172.21.15.203:0/1620520385 >> 172.21.15.203:0/1620520385 conn(0x7f3640061530 msgr2=0x7f36401557c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:40.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.059+0000 7f36454c6700 1 -- 172.21.15.203:0/1620520385 shutdown_connections 2021-01-06T22:03:40.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.059+0000 7f36454c6700 1 -- 172.21.15.203:0/1620520385 wait complete. 2021-01-06T22:03:40.063 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:40.063 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":22,"fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","created":"2021-01-06T22:03:21.322671+0000","modified":"2021-01-06T22:03:39.740894+0000","last_up_change":"2021-01-06T22:03:31.115611+0000","last_in_change":"2021-01-06T22:03:31.115611+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-06T22:03:25.161465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-06T22:03:35.070364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-01-06T22:03:37.862354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-01-06T22:03:38.738416+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e0b35c09-147c-4ccc-b566-c48b81e25412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":18347},{"type":"v1","addr":"172.21.15.203:6815","nonce":18347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":18347},{"type":"v1","addr":"172.21.15.203:6818","nonce":18347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":18347},{"type":"v1","addr":"172.21.15.203:6831","nonce":18347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6821","nonce":18347},{"type":"v1","addr":"172.21.15.203:6827","nonce":18347}]},"public_addr":"172.21.15.203:6815/18347","cluster_addr":"172.21.15.203:6818/18347","heartbeat_back_addr":"172.21.15.203:6831/18347","heartbeat_front_addr":"172.21.15.203:6827/18347","state":["exists","up"]},{"osd":1,"uuid":"8add59a0-2d58-4c1b-a8d1-62049ee9d85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":18337},{"type":"v1","addr":"172.21.15.203:6803","nonce":18337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":18337},{"type":"v1","addr":"172.21.15.203:6805","nonce":18337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":18337},{"type":"v1","addr":"172.21.15.203:6809","nonce":18337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":18337},{"type":"v1","addr":"172.21.15.203:6807","nonce":18337}]},"public_addr":"172.21.15.203:6803/18337","cluster_addr":"172.21.15.203:6805/18337","heartbeat_back_addr":"172.21.15.203:6809/18337","heartbeat_front_addr":"172.21.15.203:6807/18337","state":["exists","up"]},{"osd":2,"uuid":"b294a411-c7c9-477e-8eae-3b7bd0563ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":18354},{"type":"v1","addr":"172.21.15.203:6811","nonce":18354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":18354},{"type":"v1","addr":"172.21.15.203:6819","nonce":18354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6825","nonce":18354},{"type":"v1","addr":"172.21.15.203:6828","nonce":18354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6823","nonce":18354},{"type":"v1","addr":"172.21.15.203:6824","nonce":18354}]},"public_addr":"172.21.15.203:6811/18354","cluster_addr":"172.21.15.203:6819/18354","heartbeat_back_addr":"172.21.15.203:6828/18354","heartbeat_front_addr":"172.21.15.203:6824/18354","state":["exists","up"]},{"osd":3,"uuid":"5878a8f7-1a42-41c3-9810-0ac41a54dd6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":18346},{"type":"v1","addr":"172.21.15.203:6817","nonce":18346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":18346},{"type":"v1","addr":"172.21.15.203:6822","nonce":18346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":18346},{"type":"v1","addr":"172.21.15.203:6833","nonce":18346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":18346},{"type":"v1","addr":"172.21.15.203:6829","nonce":18346}]},"public_addr":"172.21.15.203:6817/18346","cluster_addr":"172.21.15.203:6822/18346","heartbeat_back_addr":"172.21.15.203:6833/18346","heartbeat_front_addr":"172.21.15.203:6829/18346","state":["exists","up"]},{"osd":4,"uuid":"f641a345-81e4-4400-a44c-d2f82fef7f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":18370},{"type":"v1","addr":"172.21.15.91:6809","nonce":18370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":18370},{"type":"v1","addr":"172.21.15.91:6811","nonce":18370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":18370},{"type":"v1","addr":"172.21.15.91:6816","nonce":18370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":18370},{"type":"v1","addr":"172.21.15.91:6813","nonce":18370}]},"public_addr":"172.21.15.91:6809/18370","cluster_addr":"172.21.15.91:6811/18370","heartbeat_back_addr":"172.21.15.91:6816/18370","heartbeat_front_addr":"172.21.15.91:6813/18370","state":["exists","up"]},{"osd":5,"uuid":"c1ff150d-68c6-4e6b-abd4-76af27bb2103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":18372},{"type":"v1","addr":"172.21.15.91:6801","nonce":18372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":18372},{"type":"v1","addr":"172.21.15.91:6803","nonce":18372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":18372},{"type":"v1","addr":"172.21.15.91:6807","nonce":18372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":18372},{"type":"v1","addr":"172.21.15.91:6805","nonce":18372}]},"public_addr":"172.21.15.91:6801/18372","cluster_addr":"172.21.15.91:6803/18372","heartbeat_back_addr":"172.21.15.91:6807/18372","heartbeat_front_addr":"172.21.15.91:6805/18372","state":["exists","up"]},{"osd":6,"uuid":"35a58e1a-0a0a-4f02-a477-adc029038d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6815","nonce":18371},{"type":"v1","addr":"172.21.15.91:6817","nonce":18371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":18371},{"type":"v1","addr":"172.21.15.91:6819","nonce":18371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":18371},{"type":"v1","addr":"172.21.15.91:6823","nonce":18371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":18371},{"type":"v1","addr":"172.21.15.91:6821","nonce":18371}]},"public_addr":"172.21.15.91:6817/18371","cluster_addr":"172.21.15.91:6819/18371","heartbeat_back_addr":"172.21.15.91:6823/18371","heartbeat_front_addr":"172.21.15.91:6821/18371","state":["exists","up"]},{"osd":7,"uuid":"809a2d89-4303-482a-8558-aef907814234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":18373},{"type":"v1","addr":"172.21.15.91:6825","nonce":18373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":18373},{"type":"v1","addr":"172.21.15.91:6827","nonce":18373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":18373},{"type":"v1","addr":"172.21.15.91:6831","nonce":18373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":18373},{"type":"v1","addr":"172.21.15.91:6829","nonce":18373}]},"public_addr":"172.21.15.91:6825/18373","cluster_addr":"172.21.15.91:6827/18373","heartbeat_back_addr":"172.21.15.91:6831/18373","heartbeat_front_addr":"172.21.15.91:6829/18373","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.875052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.103604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.179225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.158512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.206073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.894840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.229708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.248192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-06T22:03:40.073 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-06T22:03:40.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.191+0000 7f29d5f69700 1 Processor -- start 2021-01-06T22:03:40.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.191+0000 7f29d5f69700 1 -- start start 2021-01-06T22:03:40.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.191+0000 7f29d5f69700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29d00bbcc0 0x7f29d00bc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:40.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.191+0000 7f29d5f69700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f29d00bdbf0 0x7f29d0151220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:40.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29d5f69700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d0151f90 con 0x7f29d00bd790 2021-01-06T22:03:40.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29d5f69700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d00602b0 con 0x7f29d00bdbf0 2021-01-06T22:03:40.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29d5f69700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d00604a0 con 0x7f29d00bbcc0 2021-01-06T22:03:40.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29cffff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f29d00bdbf0 0x7f29d0151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:40.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29ceffd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f29d00bd790 0x7f29d014ad00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47090/0 (socket says 172.21.15.203:47090) 2021-01-06T22:03:40.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29ceffd700 1 -- 172.21.15.203:0/468413629 learned_addr learned my addr 172.21.15.203:0/468413629 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:40.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29cf7fe700 1 --2- 172.21.15.203:0/468413629 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29d00bbcc0 0x7f29d00bc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:40.199 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29cffff700 1 -- 172.21.15.203:0/468413629 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29d00bbcc0 msgr2=0x7f29d00bc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:40.199 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29cffff700 1 --2- 172.21.15.203:0/468413629 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29d00bbcc0 0x7f29d00bc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.200 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29cffff700 1 -- 172.21.15.203:0/468413629 >> v1:172.21.15.203:6789/0 conn(0x7f29d00bd790 legacy=0x7f29d014ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:40.200 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29ce7fc700 1 -- 172.21.15.203:0/468413629 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1322636294 0 0) 0x7f29d0151f90 con 0x7f29d00bd790 2021-01-06T22:03:40.200 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29cffff700 1 -- 172.21.15.203:0/468413629 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d0060740 con 0x7f29d00bdbf0 2021-01-06T22:03:40.201 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29cffff700 1 --2- 172.21.15.203:0/468413629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29d00bdbf0 0x7f29d0151220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f29c8009c90 tx=0x7f29c80091d0).ready entity=mon.1 client_cookie=41e16179600af2b server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:40.201 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29ce7fc700 1 -- 172.21.15.203:0/468413629 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c800ba70 con 0x7f29d00bdbf0 2021-01-06T22:03:40.201 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29ce7fc700 1 -- 172.21.15.203:0/468413629 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c800bbd0 con 0x7f29d00bdbf0 2021-01-06T22:03:40.201 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29ce7fc700 1 -- 172.21.15.203:0/468413629 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c8011400 con 0x7f29d00bdbf0 2021-01-06T22:03:40.202 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29d5f69700 1 -- 172.21.15.203:0/468413629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29d00bdbf0 msgr2=0x7f29d0151220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:40.202 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29d5f69700 1 --2- 172.21.15.203:0/468413629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29d00bdbf0 0x7f29d0151220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f29c8009c90 tx=0x7f29c80091d0).stop 2021-01-06T22:03:40.202 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29d5f69700 1 -- 172.21.15.203:0/468413629 shutdown_connections 2021-01-06T22:03:40.202 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29d5f69700 1 --2- 172.21.15.203:0/468413629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29d00bdbf0 0x7f29d0151220 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.203 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29d5f69700 1 --2- 172.21.15.203:0/468413629 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29d00bbcc0 0x7f29d00bc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.203 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29d5f69700 1 -- 172.21.15.203:0/468413629 >> 172.21.15.203:0/468413629 conn(0x7f29d0064dc0 msgr2=0x7f29d00651d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:40.203 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29d5f69700 1 -- 172.21.15.203:0/468413629 shutdown_connections 2021-01-06T22:03:40.203 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.195+0000 7f29d5f69700 1 -- 172.21.15.203:0/468413629 wait complete. 2021-01-06T22:03:40.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29d5f69700 1 Processor -- start 2021-01-06T22:03:40.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29d5f69700 1 -- start start 2021-01-06T22:03:40.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29d5f69700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29d00bbcc0 0x7f29d00b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:40.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29d5f69700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29d00bd790 0x7f29d00b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:40.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29d5f69700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29d00bdbf0 0x7f29d014c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:40.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29d5f69700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d00c1b60 con 0x7f29d00bd790 2021-01-06T22:03:40.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29d5f69700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d015df00 con 0x7f29d00bdbf0 2021-01-06T22:03:40.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29d5f69700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d014bcb0 con 0x7f29d00bbcc0 2021-01-06T22:03:40.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29cffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29d00bdbf0 0x7f29d014c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:40.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29cffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29d00bdbf0 0x7f29d014c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48506/0 (socket says 172.21.15.203:48506) 2021-01-06T22:03:40.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29cffff700 1 -- 172.21.15.203:0/1036408368 learned_addr learned my addr 172.21.15.203:0/1036408368 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:40.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29ceffd700 1 --2- 172.21.15.203:0/1036408368 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29d00bd790 0x7f29d00b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:40.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29cf7fe700 1 --2- 172.21.15.203:0/1036408368 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29d00bbcc0 0x7f29d00b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:40.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29cffff700 1 -- 172.21.15.203:0/1036408368 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29d00bbcc0 msgr2=0x7f29d00b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:40.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29cffff700 1 --2- 172.21.15.203:0/1036408368 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29d00bbcc0 0x7f29d00b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29cffff700 1 -- 172.21.15.203:0/1036408368 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29d00bd790 msgr2=0x7f29d00b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:40.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29cffff700 1 --2- 172.21.15.203:0/1036408368 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29d00bd790 0x7f29d00b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29cffff700 1 -- 172.21.15.203:0/1036408368 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d014bf50 con 0x7f29d00bdbf0 2021-01-06T22:03:40.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29cffff700 1 --2- 172.21.15.203:0/1036408368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29d00bdbf0 0x7f29d014c930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f29c8009210 tx=0x7f29c8017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:40.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29b77fe700 1 -- 172.21.15.203:0/1036408368 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c801c700 con 0x7f29d00bdbf0 2021-01-06T22:03:40.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29b77fe700 1 -- 172.21.15.203:0/1036408368 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c801c860 con 0x7f29d00bdbf0 2021-01-06T22:03:40.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29b77fe700 1 -- 172.21.15.203:0/1036408368 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c801cbe0 con 0x7f29d00bdbf0 2021-01-06T22:03:40.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29d5f69700 1 -- 172.21.15.203:0/1036408368 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d014c1f0 con 0x7f29d00bdbf0 2021-01-06T22:03:40.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29d5f69700 1 -- 172.21.15.203:0/1036408368 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d015db80 con 0x7f29d00bdbf0 2021-01-06T22:03:40.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.199+0000 7f29d5f69700 1 -- 172.21.15.203:0/1036408368 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d00450a0 con 0x7f29d00bdbf0 2021-01-06T22:03:40.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.203+0000 7f29b77fe700 1 -- 172.21.15.203:0/1036408368 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f29c800f070 con 0x7f29d00bdbf0 2021-01-06T22:03:40.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.203+0000 7f29b77fe700 1 --2- 172.21.15.203:0/1036408368 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f29b00563d0 0x7f29b0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:40.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.203+0000 7f29b77fe700 1 -- 172.21.15.203:0/1036408368 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f29c8071610 con 0x7f29d00bdbf0 2021-01-06T22:03:40.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.203+0000 7f29cf7fe700 1 --2- 172.21.15.203:0/1036408368 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f29b00563d0 0x7f29b0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:40.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.203+0000 7f29cf7fe700 1 --2- 172.21.15.203:0/1036408368 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f29b00563d0 0x7f29b0058890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f29c000bb50 tx=0x7f29c0006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:40.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.211+0000 7f29b77fe700 1 -- 172.21.15.203:0/1036408368 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f29c8046d80 con 0x7f29d00bdbf0 2021-01-06T22:03:40.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.367+0000 7f29d5f69700 1 -- 172.21.15.203:0/1036408368 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29d0061ee0 con 0x7f29d00bdbf0 2021-01-06T22:03:40.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.371+0000 7f29b77fe700 1 -- 172.21.15.203:0/1036408368 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16839 (secure 0 0 0) 0x7f29c804a3a0 con 0x7f29d00bdbf0 2021-01-06T22:03:40.375 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:40.376 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":22,"fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","created":"2021-01-06T22:03:21.322671+0000","modified":"2021-01-06T22:03:39.740894+0000","last_up_change":"2021-01-06T22:03:31.115611+0000","last_in_change":"2021-01-06T22:03:31.115611+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-06T22:03:25.161465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-06T22:03:35.070364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-01-06T22:03:37.862354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-01-06T22:03:38.738416+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e0b35c09-147c-4ccc-b566-c48b81e25412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":18347},{"type":"v1","addr":"172.21.15.203:6815","nonce":18347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":18347},{"type":"v1","addr":"172.21.15.203:6818","nonce":18347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":18347},{"type":"v1","addr":"172.21.15.203:6831","nonce":18347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6821","nonce":18347},{"type":"v1","addr":"172.21.15.203:6827","nonce":18347}]},"public_addr":"172.21.15.203:6815/18347","cluster_addr":"172.21.15.203:6818/18347","heartbeat_back_addr":"172.21.15.203:6831/18347","heartbeat_front_addr":"172.21.15.203:6827/18347","state":["exists","up"]},{"osd":1,"uuid":"8add59a0-2d58-4c1b-a8d1-62049ee9d85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":18337},{"type":"v1","addr":"172.21.15.203:6803","nonce":18337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":18337},{"type":"v1","addr":"172.21.15.203:6805","nonce":18337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":18337},{"type":"v1","addr":"172.21.15.203:6809","nonce":18337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":18337},{"type":"v1","addr":"172.21.15.203:6807","nonce":18337}]},"public_addr":"172.21.15.203:6803/18337","cluster_addr":"172.21.15.203:6805/18337","heartbeat_back_addr":"172.21.15.203:6809/18337","heartbeat_front_addr":"172.21.15.203:6807/18337","state":["exists","up"]},{"osd":2,"uuid":"b294a411-c7c9-477e-8eae-3b7bd0563ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":18354},{"type":"v1","addr":"172.21.15.203:6811","nonce":18354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":18354},{"type":"v1","addr":"172.21.15.203:6819","nonce":18354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6825","nonce":18354},{"type":"v1","addr":"172.21.15.203:6828","nonce":18354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6823","nonce":18354},{"type":"v1","addr":"172.21.15.203:6824","nonce":18354}]},"public_addr":"172.21.15.203:6811/18354","cluster_addr":"172.21.15.203:6819/18354","heartbeat_back_addr":"172.21.15.203:6828/18354","heartbeat_front_addr":"172.21.15.203:6824/18354","state":["exists","up"]},{"osd":3,"uuid":"5878a8f7-1a42-41c3-9810-0ac41a54dd6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":18346},{"type":"v1","addr":"172.21.15.203:6817","nonce":18346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":18346},{"type":"v1","addr":"172.21.15.203:6822","nonce":18346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":18346},{"type":"v1","addr":"172.21.15.203:6833","nonce":18346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":18346},{"type":"v1","addr":"172.21.15.203:6829","nonce":18346}]},"public_addr":"172.21.15.203:6817/18346","cluster_addr":"172.21.15.203:6822/18346","heartbeat_back_addr":"172.21.15.203:6833/18346","heartbeat_front_addr":"172.21.15.203:6829/18346","state":["exists","up"]},{"osd":4,"uuid":"f641a345-81e4-4400-a44c-d2f82fef7f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":18370},{"type":"v1","addr":"172.21.15.91:6809","nonce":18370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":18370},{"type":"v1","addr":"172.21.15.91:6811","nonce":18370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":18370},{"type":"v1","addr":"172.21.15.91:6816","nonce":18370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":18370},{"type":"v1","addr":"172.21.15.91:6813","nonce":18370}]},"public_addr":"172.21.15.91:6809/18370","cluster_addr":"172.21.15.91:6811/18370","heartbeat_back_addr":"172.21.15.91:6816/18370","heartbeat_front_addr":"172.21.15.91:6813/18370","state":["exists","up"]},{"osd":5,"uuid":"c1ff150d-68c6-4e6b-abd4-76af27bb2103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":18372},{"type":"v1","addr":"172.21.15.91:6801","nonce":18372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":18372},{"type":"v1","addr":"172.21.15.91:6803","nonce":18372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":18372},{"type":"v1","addr":"172.21.15.91:6807","nonce":18372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":18372},{"type":"v1","addr":"172.21.15.91:6805","nonce":18372}]},"public_addr":"172.21.15.91:6801/18372","cluster_addr":"172.21.15.91:6803/18372","heartbeat_back_addr":"172.21.15.91:6807/18372","heartbeat_front_addr":"172.21.15.91:6805/18372","state":["exists","up"]},{"osd":6,"uuid":"35a58e1a-0a0a-4f02-a477-adc029038d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6815","nonce":18371},{"type":"v1","addr":"172.21.15.91:6817","nonce":18371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":18371},{"type":"v1","addr":"172.21.15.91:6819","nonce":18371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":18371},{"type":"v1","addr":"172.21.15.91:6823","nonce":18371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":18371},{"type":"v1","addr":"172.21.15.91:6821","nonce":18371}]},"public_addr":"172.21.15.91:6817/18371","cluster_addr":"172.21.15.91:6819/18371","heartbeat_back_addr":"172.21.15.91:6823/18371","heartbeat_front_addr":"172.21.15.91:6821/18371","state":["exists","up"]},{"osd":7,"uuid":"809a2d89-4303-482a-8558-aef907814234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":18373},{"type":"v1","addr":"172.21.15.91:6825","nonce":18373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":18373},{"type":"v1","addr":"172.21.15.91:6827","nonce":18373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":18373},{"type":"v1","addr":"172.21.15.91:6831","nonce":18373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":18373},{"type":"v1","addr":"172.21.15.91:6829","nonce":18373}]},"public_addr":"172.21.15.91:6825/18373","cluster_addr":"172.21.15.91:6827/18373","heartbeat_back_addr":"172.21.15.91:6831/18373","heartbeat_front_addr":"172.21.15.91:6829/18373","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.875052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.103604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.179225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.158512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.206073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.894840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.229708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.248192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-06T22:03:40.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.375+0000 7f29d5f69700 1 -- 172.21.15.203:0/1036408368 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f29b00563d0 msgr2=0x7f29b0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:40.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.375+0000 7f29d5f69700 1 --2- 172.21.15.203:0/1036408368 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f29b00563d0 0x7f29b0058890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f29c000bb50 tx=0x7f29c0006de0).stop 2021-01-06T22:03:40.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.375+0000 7f29d5f69700 1 -- 172.21.15.203:0/1036408368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29d00bdbf0 msgr2=0x7f29d014c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:40.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.375+0000 7f29d5f69700 1 --2- 172.21.15.203:0/1036408368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29d00bdbf0 0x7f29d014c930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f29c8009210 tx=0x7f29c8017000).stop 2021-01-06T22:03:40.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.375+0000 7f29d5f69700 1 -- 172.21.15.203:0/1036408368 shutdown_connections 2021-01-06T22:03:40.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.375+0000 7f29d5f69700 1 --2- 172.21.15.203:0/1036408368 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f29b00563d0 0x7f29b0058890 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.375+0000 7f29d5f69700 1 --2- 172.21.15.203:0/1036408368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29d00bdbf0 0x7f29d014c930 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.375+0000 7f29d5f69700 1 --2- 172.21.15.203:0/1036408368 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29d00bbcc0 0x7f29d00b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.375+0000 7f29d5f69700 1 --2- 172.21.15.203:0/1036408368 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29d00bd790 0x7f29d00b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.375+0000 7f29d5f69700 1 -- 172.21.15.203:0/1036408368 >> 172.21.15.203:0/1036408368 conn(0x7f29d0064dc0 msgr2=0x7f29d0151760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:40.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.375+0000 7f29d5f69700 1 -- 172.21.15.203:0/1036408368 shutdown_connections 2021-01-06T22:03:40.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.375+0000 7f29d5f69700 1 -- 172.21.15.203:0/1036408368 wait complete. 2021-01-06T22:03:40.394 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-01-06T22:03:40.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.519+0000 7f9699351700 1 Processor -- start 2021-01-06T22:03:40.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.519+0000 7f9699351700 1 -- start start 2021-01-06T22:03:40.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.519+0000 7f9699351700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96940bf4e0 0x7f9694067d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:40.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.519+0000 7f9699351700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f96940bf8c0 0x7f9694158b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:40.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.519+0000 7f9699351700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96940676a0 con 0x7f96940609c0 2021-01-06T22:03:40.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.519+0000 7f9699351700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96940bbef0 con 0x7f96940bf8c0 2021-01-06T22:03:40.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.519+0000 7f9699351700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9694057510 con 0x7f96940bf4e0 2021-01-06T22:03:40.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.519+0000 7f96927fc700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f96940609c0 0x7f969414b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47104/0 (socket says 172.21.15.203:47104) 2021-01-06T22:03:40.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.519+0000 7f96927fc700 1 -- 172.21.15.203:0/36174251 learned_addr learned my addr 172.21.15.203:0/36174251 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:40.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9692ffd700 1 --2- 172.21.15.203:0/36174251 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96940bf4e0 0x7f9694067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:40.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f96937fe700 1 --2- 172.21.15.203:0/36174251 >> v2:172.21.15.91:3300/0 conn(0x7f96940bf8c0 0x7f9694158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:40.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9691ffb700 1 -- 172.21.15.203:0/36174251 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1872480134 0 0) 0x7f96940676a0 con 0x7f96940609c0 2021-01-06T22:03:40.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9691ffb700 1 -- 172.21.15.203:0/36174251 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9674003230 con 0x7f96940609c0 2021-01-06T22:03:40.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9692ffd700 1 -- 172.21.15.203:0/36174251 >> v2:172.21.15.91:3300/0 conn(0x7f96940bf8c0 msgr2=0x7f9694158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:40.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9691ffb700 1 -- 172.21.15.203:0/36174251 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2756375212 0 0) 0x7f9674003230 con 0x7f96940609c0 2021-01-06T22:03:40.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9692ffd700 1 --2- 172.21.15.203:0/36174251 >> v2:172.21.15.91:3300/0 conn(0x7f96940bf8c0 0x7f9694158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9692ffd700 1 -- 172.21.15.203:0/36174251 >> v1:172.21.15.203:6789/0 conn(0x7f96940609c0 legacy=0x7f969414b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:40.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9692ffd700 1 -- 172.21.15.203:0/36174251 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96940bc150 con 0x7f96940bf4e0 2021-01-06T22:03:40.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f96937fe700 1 --2- 172.21.15.203:0/36174251 >> v2:172.21.15.91:3300/0 conn(0x7f96940bf8c0 0x7f9694158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:40.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9692ffd700 1 --2- 172.21.15.203:0/36174251 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96940bf4e0 0x7f9694067d70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f968400b980 tx=0x7f9684006de0).ready entity=mon.2 client_cookie=d2b215c408a0ff44 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:40.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9691ffb700 1 -- 172.21.15.203:0/36174251 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c0014e0 con 0x7f96940bf4e0 2021-01-06T22:03:40.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9691ffb700 1 -- 172.21.15.203:0/36174251 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9684011400 con 0x7f96940bf4e0 2021-01-06T22:03:40.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9691ffb700 1 -- 172.21.15.203:0/36174251 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684011780 con 0x7f96940bf4e0 2021-01-06T22:03:40.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9699351700 1 -- 172.21.15.203:0/36174251 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96940bf4e0 msgr2=0x7f9694067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:40.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9699351700 1 --2- 172.21.15.203:0/36174251 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96940bf4e0 0x7f9694067d70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f968400b980 tx=0x7f9684006de0).stop 2021-01-06T22:03:40.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9699351700 1 -- 172.21.15.203:0/36174251 shutdown_connections 2021-01-06T22:03:40.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9699351700 1 --2- 172.21.15.203:0/36174251 >> v2:172.21.15.91:3300/0 conn(0x7f96940bf8c0 0x7f9694158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9699351700 1 --2- 172.21.15.203:0/36174251 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96940bf4e0 0x7f9694067d70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9699351700 1 -- 172.21.15.203:0/36174251 >> 172.21.15.203:0/36174251 conn(0x7f9694056cd0 msgr2=0x7f96940602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:40.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9699351700 1 -- 172.21.15.203:0/36174251 shutdown_connections 2021-01-06T22:03:40.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.523+0000 7f9699351700 1 -- 172.21.15.203:0/36174251 wait complete. 2021-01-06T22:03:40.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f9699351700 1 Processor -- start 2021-01-06T22:03:40.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f9699351700 1 -- start start 2021-01-06T22:03:40.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f9699351700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96940609c0 0x7f969414cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:40.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f9699351700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96940bf4e0 0x7f96941539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:40.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f9699351700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96940bf8c0 0x7f969414f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:40.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f9699351700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969414d590 con 0x7f96940609c0 2021-01-06T22:03:40.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f9699351700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969414e560 con 0x7f96940bf4e0 2021-01-06T22:03:40.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f9699351700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969414e800 con 0x7f96940bf8c0 2021-01-06T22:03:40.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f96927fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96940bf4e0 0x7f96941539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:40.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f96927fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96940bf4e0 0x7f96941539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48518/0 (socket says 172.21.15.203:48518) 2021-01-06T22:03:40.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f96927fc700 1 -- 172.21.15.203:0/3961571134 learned_addr learned my addr 172.21.15.203:0/3961571134 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:40.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f9692ffd700 1 --2- 172.21.15.203:0/3961571134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96940609c0 0x7f969414cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:40.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f96937fe700 1 --2- 172.21.15.203:0/3961571134 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96940bf8c0 0x7f969414f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:40.539 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f96927fc700 1 -- 172.21.15.203:0/3961571134 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96940bf8c0 msgr2=0x7f969414f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:40.539 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f96927fc700 1 --2- 172.21.15.203:0/3961571134 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96940bf8c0 0x7f969414f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.539 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f96927fc700 1 -- 172.21.15.203:0/3961571134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96940609c0 msgr2=0x7f969414cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:40.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f96927fc700 1 --2- 172.21.15.203:0/3961571134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96940609c0 0x7f969414cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:40.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f96927fc700 1 -- 172.21.15.203:0/3961571134 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969414eaa0 con 0x7f96940bf4e0 2021-01-06T22:03:40.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f96927fc700 1 --2- 172.21.15.203:0/3961571134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96940bf4e0 0x7f96941539b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f967c0011a0 tx=0x7f967c0011f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:40.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f968affd700 1 -- 172.21.15.203:0/3961571134 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c007320 con 0x7f96940bf4e0 2021-01-06T22:03:40.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f968affd700 1 -- 172.21.15.203:0/3961571134 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c007480 con 0x7f96940bf4e0 2021-01-06T22:03:40.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f9699351700 1 -- 172.21.15.203:0/3961571134 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9694154280 con 0x7f96940bf4e0 2021-01-06T22:03:40.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f968affd700 1 -- 172.21.15.203:0/3961571134 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c0057f0 con 0x7f96940bf4e0 2021-01-06T22:03:40.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.527+0000 7f9699351700 1 -- 172.21.15.203:0/3961571134 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96940c3580 con 0x7f96940bf4e0 2021-01-06T22:03:40.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.531+0000 7f9699351700 1 -- 172.21.15.203:0/3961571134 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969405cfe0 con 0x7f96940bf4e0 2021-01-06T22:03:40.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.531+0000 7f968affd700 1 -- 172.21.15.203:0/3961571134 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f967c010030 con 0x7f96940bf4e0 2021-01-06T22:03:40.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.531+0000 7f968affd700 1 --2- 172.21.15.203:0/3961571134 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9678056400 0x7f96780588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:40.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.531+0000 7f968affd700 1 -- 172.21.15.203:0/3961571134 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f967c06f300 con 0x7f96940bf4e0 2021-01-06T22:03:40.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.531+0000 7f9692ffd700 1 --2- 172.21.15.203:0/3961571134 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9678056400 0x7f96780588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:40.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.531+0000 7f9692ffd700 1 --2- 172.21.15.203:0/3961571134 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9678056400 0x7f96780588c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f968400b130 tx=0x7f9684016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:40.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.539+0000 7f968affd700 1 -- 172.21.15.203:0/3961571134 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f967c044b80 con 0x7f96940bf4e0 2021-01-06T22:03:40.742 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:40.739+0000 7f9699351700 1 -- 172.21.15.203:0/3961571134 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f969405da20 con 0x7f96940bf4e0 2021-01-06T22:03:40.783 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:03:40.785 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:03:41.766 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.763+0000 7f968affd700 1 -- 172.21.15.203:0/3961571134 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f967c0481a0 con 0x7f96940bf4e0 2021-01-06T22:03:41.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.767+0000 7f9699351700 1 -- 172.21.15.203:0/3961571134 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9678056400 msgr2=0x7f96780588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:41.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.767+0000 7f9699351700 1 --2- 172.21.15.203:0/3961571134 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9678056400 0x7f96780588c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f968400b130 tx=0x7f9684016000).stop 2021-01-06T22:03:41.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.767+0000 7f9699351700 1 -- 172.21.15.203:0/3961571134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96940bf4e0 msgr2=0x7f96941539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:41.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.767+0000 7f9699351700 1 --2- 172.21.15.203:0/3961571134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96940bf4e0 0x7f96941539b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f967c0011a0 tx=0x7f967c0011f0).stop 2021-01-06T22:03:41.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.767+0000 7f9699351700 1 -- 172.21.15.203:0/3961571134 shutdown_connections 2021-01-06T22:03:41.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.767+0000 7f9699351700 1 --2- 172.21.15.203:0/3961571134 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9678056400 0x7f96780588c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:41.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.767+0000 7f9699351700 1 --2- 172.21.15.203:0/3961571134 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96940bf8c0 0x7f969414f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:41.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.767+0000 7f9699351700 1 --2- 172.21.15.203:0/3961571134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96940609c0 0x7f969414cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:41.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.767+0000 7f9699351700 1 --2- 172.21.15.203:0/3961571134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96940bf4e0 0x7f96941539b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:41.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.767+0000 7f9699351700 1 -- 172.21.15.203:0/3961571134 >> 172.21.15.203:0/3961571134 conn(0x7f9694056cd0 msgr2=0x7f9694151ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:41.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.767+0000 7f9699351700 1 -- 172.21.15.203:0/3961571134 shutdown_connections 2021-01-06T22:03:41.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.767+0000 7f9699351700 1 -- 172.21.15.203:0/3961571134 wait complete. 2021-01-06T22:03:41.789 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2021-01-06T22:03:41.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.927+0000 7f8fbf51f700 1 Processor -- start 2021-01-06T22:03:41.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.927+0000 7f8fbf51f700 1 -- start start 2021-01-06T22:03:41.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbf51f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8fb814c110 0x7f8fb80657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:41.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbf51f700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb814ee80 con 0x7f8fb80be730 2021-01-06T22:03:41.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbf51f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb814f120 con 0x7f8fb80be350 2021-01-06T22:03:41.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbf51f700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb814f3c0 con 0x7f8fb814c110 2021-01-06T22:03:41.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbed1e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8fb814c110 0x7f8fb80657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:41.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbe51d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f8fb80be350 0x7f8fb80ba350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47124/0 (socket says 172.21.15.203:47124) 2021-01-06T22:03:41.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbe51d700 1 -- 172.21.15.203:0/3857768555 learned_addr learned my addr 172.21.15.203:0/3857768555 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:41.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbed1e700 1 -- 172.21.15.203:0/3857768555 >> v1:172.21.15.203:6789/0 conn(0x7f8fb80be350 legacy=0x7f8fb80ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:41.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbed1e700 1 -- 172.21.15.203:0/3857768555 >> v1:172.21.15.91:6789/0 conn(0x7f8fb80be730 legacy=0x7f8fb814ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:41.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbed1e700 1 -- 172.21.15.203:0/3857768555 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb814f660 con 0x7f8fb814c110 2021-01-06T22:03:41.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbed1e700 1 --2- 172.21.15.203:0/3857768555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb814c110 0x7f8fb80657c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f8fb0009c90 tx=0x7f8fb00091d0).ready entity=mon.1 client_cookie=2a5f61878e7b366e server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:41.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbd51b700 1 -- 172.21.15.203:0/3857768555 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb000bbc0 con 0x7f8fb814c110 2021-01-06T22:03:41.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbd51b700 1 -- 172.21.15.203:0/3857768555 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb000bd20 con 0x7f8fb814c110 2021-01-06T22:03:41.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbd51b700 1 -- 172.21.15.203:0/3857768555 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0011610 con 0x7f8fb814c110 2021-01-06T22:03:41.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbf51f700 1 -- 172.21.15.203:0/3857768555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb814c110 msgr2=0x7f8fb80657c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:41.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbf51f700 1 --2- 172.21.15.203:0/3857768555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb814c110 0x7f8fb80657c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f8fb0009c90 tx=0x7f8fb00091d0).stop 2021-01-06T22:03:41.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbf51f700 1 -- 172.21.15.203:0/3857768555 shutdown_connections 2021-01-06T22:03:41.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbf51f700 1 --2- 172.21.15.203:0/3857768555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb814c110 0x7f8fb80657c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:41.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbf51f700 1 -- 172.21.15.203:0/3857768555 >> 172.21.15.203:0/3857768555 conn(0x7f8fb8067e20 msgr2=0x7f8fb8068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:41.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbf51f700 1 -- 172.21.15.203:0/3857768555 shutdown_connections 2021-01-06T22:03:41.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbf51f700 1 -- 172.21.15.203:0/3857768555 wait complete. 2021-01-06T22:03:41.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.931+0000 7f8fbf51f700 1 Processor -- start 2021-01-06T22:03:41.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbf51f700 1 -- start start 2021-01-06T22:03:41.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbf51f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb80be350 0x7f8fb8149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:41.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbf51f700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fb80be730 0x7f8fb81427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:41.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbf51f700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8fb814c110 0x7f8fb815e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:41.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbf51f700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb814a590 con 0x7f8fb80be730 2021-01-06T22:03:41.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbf51f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb8146bd0 con 0x7f8fb80be350 2021-01-06T22:03:41.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbf51f700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb8146e70 con 0x7f8fb814c110 2021-01-06T22:03:41.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbe51d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb80be350 0x7f8fb8149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:41.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbe51d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb80be350 0x7f8fb8149de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48538/0 (socket says 172.21.15.203:48538) 2021-01-06T22:03:41.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbe51d700 1 -- 172.21.15.203:0/4101264670 learned_addr learned my addr 172.21.15.203:0/4101264670 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:41.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbdd1c700 1 --2- 172.21.15.203:0/4101264670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fb80be730 0x7f8fb81427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:41.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbed1e700 1 --2- 172.21.15.203:0/4101264670 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8fb814c110 0x7f8fb815e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:41.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbe51d700 1 -- 172.21.15.203:0/4101264670 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8fb814c110 msgr2=0x7f8fb815e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:41.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbe51d700 1 --2- 172.21.15.203:0/4101264670 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8fb814c110 0x7f8fb815e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:41.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbe51d700 1 -- 172.21.15.203:0/4101264670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fb80be730 msgr2=0x7f8fb81427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:41.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbe51d700 1 --2- 172.21.15.203:0/4101264670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fb80be730 0x7f8fb81427c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:41.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbe51d700 1 -- 172.21.15.203:0/4101264670 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb8147110 con 0x7f8fb80be350 2021-01-06T22:03:41.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbed1e700 1 --2- 172.21.15.203:0/4101264670 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8fb814c110 0x7f8fb815e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:41.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbe51d700 1 --2- 172.21.15.203:0/4101264670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb80be350 0x7f8fb8149de0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f8fac00c710 tx=0x7f8fac007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:41.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8faa7fc700 1 -- 172.21.15.203:0/4101264670 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac007320 con 0x7f8fb80be350 2021-01-06T22:03:41.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8faa7fc700 1 -- 172.21.15.203:0/4101264670 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac007480 con 0x7f8fb80be350 2021-01-06T22:03:41.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbf51f700 1 -- 172.21.15.203:0/4101264670 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb81473b0 con 0x7f8fb80be350 2021-01-06T22:03:41.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8faa7fc700 1 -- 172.21.15.203:0/4101264670 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac0057f0 con 0x7f8fb80be350 2021-01-06T22:03:41.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbf51f700 1 -- 172.21.15.203:0/4101264670 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb814a2d0 con 0x7f8fb80be350 2021-01-06T22:03:41.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.935+0000 7f8fbf51f700 1 -- 172.21.15.203:0/4101264670 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb815e680 con 0x7f8fb80be350 2021-01-06T22:03:41.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.939+0000 7f8faa7fc700 1 -- 172.21.15.203:0/4101264670 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f8fac010030 con 0x7f8fb80be350 2021-01-06T22:03:41.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.939+0000 7f8faa7fc700 1 --2- 172.21.15.203:0/4101264670 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8f980563d0 0x7f8f98058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:41.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.939+0000 7f8faa7fc700 1 -- 172.21.15.203:0/4101264670 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8fac06f210 con 0x7f8fb80be350 2021-01-06T22:03:41.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.939+0000 7f8fbdd1c700 1 --2- 172.21.15.203:0/4101264670 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8f980563d0 0x7f8f98058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:41.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.939+0000 7f8fbdd1c700 1 --2- 172.21.15.203:0/4101264670 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8f980563d0 0x7f8f98058890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f8fb8146160 tx=0x7f8fb4007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:41.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:41.947+0000 7f8faa7fc700 1 -- 172.21.15.203:0/4101264670 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f8fac044a30 con 0x7f8fb80be350 2021-01-06T22:03:42.153 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.151+0000 7f8fbf51f700 1 -- 172.21.15.203:0/4101264670 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f8fb80450a0 con 0x7f8fb80be350 2021-01-06T22:03:42.329 INFO:tasks.ceph.mds.f.smithi091.stderr:2021-01-06T22:03:42.330+0000 7f318add3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2021-01-06T22:03:42.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.767+0000 7f8faa7fc700 1 -- 172.21.15.203:0/4101264670 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v8) v1 ==== 105+0+0 (secure 0 0 0) 0x7f8fac048050 con 0x7f8fb80be350 2021-01-06T22:03:42.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.771+0000 7f8fbf51f700 1 -- 172.21.15.203:0/4101264670 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8f980563d0 msgr2=0x7f8f98058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:42.774 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.771+0000 7f8fbf51f700 1 --2- 172.21.15.203:0/4101264670 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8f980563d0 0x7f8f98058890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f8fb8146160 tx=0x7f8fb4007b80).stop 2021-01-06T22:03:42.774 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.771+0000 7f8fbf51f700 1 -- 172.21.15.203:0/4101264670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb80be350 msgr2=0x7f8fb8149de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:42.774 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.771+0000 7f8fbf51f700 1 --2- 172.21.15.203:0/4101264670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb80be350 0x7f8fb8149de0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f8fac00c710 tx=0x7f8fac007c30).stop 2021-01-06T22:03:42.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.771+0000 7f8fbf51f700 1 -- 172.21.15.203:0/4101264670 shutdown_connections 2021-01-06T22:03:42.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.771+0000 7f8fbf51f700 1 --2- 172.21.15.203:0/4101264670 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8f980563d0 0x7f8f98058890 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:42.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.771+0000 7f8fbf51f700 1 --2- 172.21.15.203:0/4101264670 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8fb814c110 0x7f8fb815e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:42.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.771+0000 7f8fbf51f700 1 --2- 172.21.15.203:0/4101264670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb80be350 0x7f8fb8149de0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:42.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.771+0000 7f8fbf51f700 1 --2- 172.21.15.203:0/4101264670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fb80be730 0x7f8fb81427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:42.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.771+0000 7f8fbf51f700 1 -- 172.21.15.203:0/4101264670 >> 172.21.15.203:0/4101264670 conn(0x7f8fb8067e20 msgr2=0x7f8fb8064f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:42.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.771+0000 7f8fbf51f700 1 -- 172.21.15.203:0/4101264670 shutdown_connections 2021-01-06T22:03:42.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.771+0000 7f8fbf51f700 1 -- 172.21.15.203:0/4101264670 wait complete. 2021-01-06T22:03:42.795 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2021-01-06T22:03:42.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f213dde9700 1 Processor -- start 2021-01-06T22:03:42.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f213dde9700 1 -- start start 2021-01-06T22:03:42.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f213dde9700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21380b9ce0 0x7f21381523b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:42.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f213dde9700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2138067c00 0x7f21380636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:42.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f213dde9700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2138064df0 con 0x7f2138066c60 2021-01-06T22:03:42.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f213dde9700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2138063cc0 con 0x7f2138067c00 2021-01-06T22:03:42.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f213dde9700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2138063f60 con 0x7f21380b9ce0 2021-01-06T22:03:42.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f2137fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2138067c00 0x7f21380636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:42.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f2137fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2138067c00 0x7f21380636a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48566/0 (socket says 172.21.15.203:48566) 2021-01-06T22:03:42.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f2137fff700 1 -- 172.21.15.203:0/1129248200 learned_addr learned my addr 172.21.15.203:0/1129248200 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:42.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f21377fe700 1 --2- 172.21.15.203:0/1129248200 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21380b9ce0 0x7f21381523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:42.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f2137fff700 1 -- 172.21.15.203:0/1129248200 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21380b9ce0 msgr2=0x7f21381523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:42.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f2137fff700 1 --2- 172.21.15.203:0/1129248200 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21380b9ce0 0x7f21381523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:42.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f2137fff700 1 -- 172.21.15.203:0/1129248200 >> v1:172.21.15.91:6789/0 conn(0x7f2138066c60 legacy=0x7f21380674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:42.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f2137fff700 1 -- 172.21.15.203:0/1129248200 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2138064200 con 0x7f2138067c00 2021-01-06T22:03:42.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f2137fff700 1 --2- 172.21.15.203:0/1129248200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2138067c00 0x7f21380636a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f2130009d50 tx=0x7f2130009290).ready entity=mon.1 client_cookie=b833fa06d7faa529 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:42.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f21367fc700 1 -- 172.21.15.203:0/1129248200 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213000bbc0 con 0x7f2138067c00 2021-01-06T22:03:42.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f21367fc700 1 -- 172.21.15.203:0/1129248200 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213000bd20 con 0x7f2138067c00 2021-01-06T22:03:42.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.919+0000 7f21367fc700 1 -- 172.21.15.203:0/1129248200 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130011610 con 0x7f2138067c00 2021-01-06T22:03:42.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.923+0000 7f213dde9700 1 -- 172.21.15.203:0/1129248200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2138067c00 msgr2=0x7f21380636a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:42.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.923+0000 7f213dde9700 1 --2- 172.21.15.203:0/1129248200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2138067c00 0x7f21380636a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f2130009d50 tx=0x7f2130009290).stop 2021-01-06T22:03:42.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.923+0000 7f213dde9700 1 -- 172.21.15.203:0/1129248200 shutdown_connections 2021-01-06T22:03:42.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.923+0000 7f213dde9700 1 --2- 172.21.15.203:0/1129248200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2138067c00 0x7f21380636a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:42.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.923+0000 7f213dde9700 1 --2- 172.21.15.203:0/1129248200 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21380b9ce0 0x7f21381523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:42.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.923+0000 7f213dde9700 1 -- 172.21.15.203:0/1129248200 >> 172.21.15.203:0/1129248200 conn(0x7f21380602b0 msgr2=0x7f21380606c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:42.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.923+0000 7f213dde9700 1 -- 172.21.15.203:0/1129248200 shutdown_connections 2021-01-06T22:03:42.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.923+0000 7f213dde9700 1 -- 172.21.15.203:0/1129248200 wait complete. 2021-01-06T22:03:42.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.923+0000 7f213dde9700 1 Processor -- start 2021-01-06T22:03:42.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.923+0000 7f213dde9700 1 -- start start 2021-01-06T22:03:42.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.923+0000 7f213dde9700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2138066c60 0x7f2138152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:42.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f213dde9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2138067c00 0x7f2138152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:42.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f213dde9700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21380b9ce0 0x7f2138148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:42.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f213dde9700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21380beb70 con 0x7f21380b9ce0 2021-01-06T22:03:42.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f213dde9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2138152d00 con 0x7f2138067c00 2021-01-06T22:03:42.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f213dde9700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2138082240 con 0x7f2138066c60 2021-01-06T22:03:42.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f21377fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2138066c60 0x7f2138152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:42.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f2136ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2138067c00 0x7f2138152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:42.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f21377fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2138066c60 0x7f2138152320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:53650/0 (socket says 172.21.15.203:53650) 2021-01-06T22:03:42.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f21377fe700 1 -- 172.21.15.203:0/3099051050 learned_addr learned my addr 172.21.15.203:0/3099051050 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:42.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f2137fff700 1 --2- 172.21.15.203:0/3099051050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21380b9ce0 0x7f2138148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:42.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f21377fe700 1 -- 172.21.15.203:0/3099051050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2138067c00 msgr2=0x7f2138152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:42.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f21377fe700 1 --2- 172.21.15.203:0/3099051050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2138067c00 0x7f2138152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:42.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f21377fe700 1 -- 172.21.15.203:0/3099051050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21380b9ce0 msgr2=0x7f2138148e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:42.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f21377fe700 1 --2- 172.21.15.203:0/3099051050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21380b9ce0 0x7f2138148e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:42.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f21377fe700 1 -- 172.21.15.203:0/3099051050 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21380bbcb0 con 0x7f2138066c60 2021-01-06T22:03:42.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f21377fe700 1 --2- 172.21.15.203:0/3099051050 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2138066c60 0x7f2138152320 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f212000ec00 tx=0x7f2120006de0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:42.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f212f7fe700 1 -- 172.21.15.203:0/3099051050 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212000b960 con 0x7f2138066c60 2021-01-06T22:03:42.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f212f7fe700 1 -- 172.21.15.203:0/3099051050 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212000bac0 con 0x7f2138066c60 2021-01-06T22:03:42.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f212f7fe700 1 -- 172.21.15.203:0/3099051050 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212000be40 con 0x7f2138066c60 2021-01-06T22:03:42.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f213dde9700 1 -- 172.21.15.203:0/3099051050 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2138082470 con 0x7f2138066c60 2021-01-06T22:03:42.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f213dde9700 1 -- 172.21.15.203:0/3099051050 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21380829b0 con 0x7f2138066c60 2021-01-06T22:03:42.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f213dde9700 1 -- 172.21.15.203:0/3099051050 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f213805d600 con 0x7f2138066c60 2021-01-06T22:03:42.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.927+0000 7f212f7fe700 1 -- 172.21.15.203:0/3099051050 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f21200339c0 con 0x7f2138066c60 2021-01-06T22:03:42.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.931+0000 7f212f7fe700 1 --2- 172.21.15.203:0/3099051050 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f211c0563d0 0x7f211c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:42.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.931+0000 7f212f7fe700 1 -- 172.21.15.203:0/3099051050 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f212006fa50 con 0x7f2138066c60 2021-01-06T22:03:42.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.931+0000 7f2136ffd700 1 --2- 172.21.15.203:0/3099051050 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f211c0563d0 0x7f211c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:42.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.931+0000 7f2136ffd700 1 --2- 172.21.15.203:0/3099051050 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f211c0563d0 0x7f211c058890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f2128007fd0 tx=0x7f2128000b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:42.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:42.939+0000 7f212f7fe700 1 -- 172.21.15.203:0/3099051050 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f2120045350 con 0x7f2138066c60 2021-01-06T22:03:43.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.139+0000 7f213dde9700 1 -- 172.21.15.203:0/3099051050 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f21380c13a0 con 0x7f2138066c60 2021-01-06T22:03:43.787 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.787+0000 7f212f7fe700 1 -- 172.21.15.203:0/3099051050 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v10) v1 ==== 115+0+0 (secure 0 0 0) 0x7f2120048970 con 0x7f2138066c60 2021-01-06T22:03:43.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.787+0000 7f213dde9700 1 -- 172.21.15.203:0/3099051050 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f211c0563d0 msgr2=0x7f211c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:43.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.787+0000 7f213dde9700 1 --2- 172.21.15.203:0/3099051050 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f211c0563d0 0x7f211c058890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f2128007fd0 tx=0x7f2128000b80).stop 2021-01-06T22:03:43.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.787+0000 7f213dde9700 1 -- 172.21.15.203:0/3099051050 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2138066c60 msgr2=0x7f2138152320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:43.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.787+0000 7f213dde9700 1 --2- 172.21.15.203:0/3099051050 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2138066c60 0x7f2138152320 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f212000ec00 tx=0x7f2120006de0).stop 2021-01-06T22:03:43.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.787+0000 7f213dde9700 1 -- 172.21.15.203:0/3099051050 shutdown_connections 2021-01-06T22:03:43.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.787+0000 7f213dde9700 1 --2- 172.21.15.203:0/3099051050 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f211c0563d0 0x7f211c058890 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:43.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.787+0000 7f213dde9700 1 --2- 172.21.15.203:0/3099051050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21380b9ce0 0x7f2138148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:43.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.787+0000 7f213dde9700 1 --2- 172.21.15.203:0/3099051050 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2138066c60 0x7f2138152320 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:43.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.787+0000 7f213dde9700 1 --2- 172.21.15.203:0/3099051050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2138067c00 0x7f2138152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:43.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.787+0000 7f213dde9700 1 -- 172.21.15.203:0/3099051050 >> 172.21.15.203:0/3099051050 conn(0x7f21380602b0 msgr2=0x7f21380c4760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:43.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.791+0000 7f213dde9700 1 -- 172.21.15.203:0/3099051050 shutdown_connections 2021-01-06T22:03:43.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.791+0000 7f213dde9700 1 -- 172.21.15.203:0/3099051050 wait complete. 2021-01-06T22:03:43.811 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:03:43.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.927+0000 7f0de0ec8700 1 Processor -- start 2021-01-06T22:03:43.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.927+0000 7f0de0ec8700 1 -- start start 2021-01-06T22:03:43.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0de0ec8700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ddc148d70 0x7f0ddc0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:43.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0de0ec8700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ddc14f0a0 con 0x7f0ddc149150 2021-01-06T22:03:43.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0de0ec8700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ddc14c3c0 con 0x7f0ddc0602b0 2021-01-06T22:03:43.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0de0ec8700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc149600 con 0x7f0ddc148d70 2021-01-06T22:03:43.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0ddaffd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f0ddc0602b0 0x7f0ddc14fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47184/0 (socket says 172.21.15.203:47184) 2021-01-06T22:03:43.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0ddaffd700 1 -- 172.21.15.203:0/2928324924 learned_addr learned my addr 172.21.15.203:0/2928324924 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:43.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0ddb7fe700 1 --2- 172.21.15.203:0/2928324924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ddc148d70 0x7f0ddc0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:43.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 -- 172.21.15.203:0/2928324924 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790017843 0 0) 0x7f0ddc14c3c0 con 0x7f0ddc0602b0 2021-01-06T22:03:43.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 -- 172.21.15.203:0/2928324924 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dbc003230 con 0x7f0ddc0602b0 2021-01-06T22:03:43.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 -- 172.21.15.203:0/2928324924 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895201769 0 0) 0x7f0ddc14f0a0 con 0x7f0ddc149150 2021-01-06T22:03:43.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 -- 172.21.15.203:0/2928324924 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ddc14c3c0 con 0x7f0ddc149150 2021-01-06T22:03:43.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 -- 172.21.15.203:0/2928324924 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1263764891 0 0) 0x7f0dbc003230 con 0x7f0ddc0602b0 2021-01-06T22:03:43.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 -- 172.21.15.203:0/2928324924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ddc148d70 msgr2=0x7f0ddc0c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:43.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 --2- 172.21.15.203:0/2928324924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ddc148d70 0x7f0ddc0c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:43.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 -- 172.21.15.203:0/2928324924 >> v1:172.21.15.91:6789/0 conn(0x7f0ddc149150 legacy=0x7f0ddc14bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:43.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 -- 172.21.15.203:0/2928324924 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc14c660 con 0x7f0ddc0602b0 2021-01-06T22:03:43.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0ddb7fe700 1 --2- 172.21.15.203:0/2928324924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ddc148d70 0x7f0ddc0c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:43.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 -- 172.21.15.203:0/2928324924 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f0dc40014e0 con 0x7f0ddc0602b0 2021-01-06T22:03:43.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 -- 172.21.15.203:0/2928324924 >> v1:172.21.15.203:6789/0 conn(0x7f0ddc0602b0 legacy=0x7f0ddc14fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:43.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 --2- 172.21.15.203:0/2928324924 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0dbc0038b0 0x7f0dbc003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:43.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 --2- 172.21.15.203:0/2928324924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dbc0044d0 0x7f0dbc004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:43.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 --2- 172.21.15.203:0/2928324924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ddc148d70 0x7f0dbc004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:43.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 -- 172.21.15.203:0/2928324924 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc149600 con 0x7f0dbc0038b0 2021-01-06T22:03:43.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 -- 172.21.15.203:0/2928324924 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc005330 con 0x7f0dbc0044d0 2021-01-06T22:03:43.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 -- 172.21.15.203:0/2928324924 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc00b720 con 0x7f0ddc148d70 2021-01-06T22:03:43.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0ddaffd700 1 --2- 172.21.15.203:0/2928324924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dbc0044d0 0x7f0dbc004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:43.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0ddb7fe700 1 --2- 172.21.15.203:0/2928324924 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0dbc0038b0 0x7f0dbc003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:43.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0ddbfff700 1 --2- 172.21.15.203:0/2928324924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ddc148d70 0x7f0dbc004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:43.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0ddaffd700 1 -- 172.21.15.203:0/2928324924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ddc148d70 msgr2=0x7f0dbc004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:43.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0ddaffd700 1 --2- 172.21.15.203:0/2928324924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ddc148d70 0x7f0dbc004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:43.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0ddaffd700 1 -- 172.21.15.203:0/2928324924 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0dbc0038b0 msgr2=0x7f0dbc003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:43.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0ddaffd700 1 --2- 172.21.15.203:0/2928324924 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0dbc0038b0 0x7f0dbc003cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:43.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0ddaffd700 1 -- 172.21.15.203:0/2928324924 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc003230 con 0x7f0dbc0044d0 2021-01-06T22:03:43.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0ddaffd700 1 --2- 172.21.15.203:0/2928324924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dbc0044d0 0x7f0dbc004950 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f0ddc068a90 tx=0x7f0ddc14c550).ready entity=mon.1 client_cookie=b59efb885aae5672 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:43.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 -- 172.21.15.203:0/2928324924 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40070d0 con 0x7f0dbc0044d0 2021-01-06T22:03:43.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 -- 172.21.15.203:0/2928324924 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc4007230 con 0x7f0dbc0044d0 2021-01-06T22:03:43.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0dda7fc700 1 -- 172.21.15.203:0/2928324924 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40055e0 con 0x7f0dbc0044d0 2021-01-06T22:03:43.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0de0ec8700 1 -- 172.21.15.203:0/2928324924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dbc0044d0 msgr2=0x7f0dbc004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:43.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0de0ec8700 1 --2- 172.21.15.203:0/2928324924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dbc0044d0 0x7f0dbc004950 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f0ddc068a90 tx=0x7f0ddc14c550).stop 2021-01-06T22:03:43.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0ddbfff700 1 -- 172.21.15.203:0/2928324924 reap_dead start 2021-01-06T22:03:43.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0de0ec8700 1 -- 172.21.15.203:0/2928324924 shutdown_connections 2021-01-06T22:03:43.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.931+0000 7f0de0ec8700 1 -- 172.21.15.203:0/2928324924 >> 172.21.15.203:0/2928324924 conn(0x7f0ddc0667b0 msgr2=0x7f0ddc0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:43.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.935+0000 7f0de0ec8700 1 -- 172.21.15.203:0/2928324924 shutdown_connections 2021-01-06T22:03:43.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.935+0000 7f0de0ec8700 1 -- 172.21.15.203:0/2928324924 wait complete. 2021-01-06T22:03:43.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.935+0000 7f0de0ec8700 1 Processor -- start 2021-01-06T22:03:43.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.935+0000 7f0de0ec8700 1 -- start start 2021-01-06T22:03:43.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.935+0000 7f0de0ec8700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ddc1467f0 0x7f0ddc15c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:43.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.935+0000 7f0de0ec8700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ddc151230 0x7f0ddc15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:43.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.935+0000 7f0de0ec8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ddc146bd0 0x7f0ddc143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:43.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.935+0000 7f0de0ec8700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0ad9a0 con 0x7f0ddc151230 2021-01-06T22:03:43.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.935+0000 7f0de0ec8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0adc40 con 0x7f0ddc146bd0 2021-01-06T22:03:43.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.935+0000 7f0de0ec8700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc15bb20 con 0x7f0ddc1467f0 2021-01-06T22:03:43.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.935+0000 7f0ddbfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ddc146bd0 0x7f0ddc143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:43.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.935+0000 7f0ddb7fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ddc1467f0 0x7f0ddc15c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:43.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.935+0000 7f0ddbfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ddc146bd0 0x7f0ddc143240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48604/0 (socket says 172.21.15.203:48604) 2021-01-06T22:03:43.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.935+0000 7f0ddbfff700 1 -- 172.21.15.203:0/3020175214 learned_addr learned my addr 172.21.15.203:0/3020175214 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:43.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.935+0000 7f0ddaffd700 1 --2- 172.21.15.203:0/3020175214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ddc151230 0x7f0ddc15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:43.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0ddbfff700 1 -- 172.21.15.203:0/3020175214 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ddc1467f0 msgr2=0x7f0ddc15c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:43.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0ddbfff700 1 --2- 172.21.15.203:0/3020175214 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ddc1467f0 0x7f0ddc15c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:43.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0ddbfff700 1 -- 172.21.15.203:0/3020175214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ddc151230 msgr2=0x7f0ddc15ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:43.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0ddbfff700 1 --2- 172.21.15.203:0/3020175214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ddc151230 0x7f0ddc15ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:43.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0ddbfff700 1 -- 172.21.15.203:0/3020175214 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc15bdc0 con 0x7f0ddc146bd0 2021-01-06T22:03:43.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0ddbfff700 1 --2- 172.21.15.203:0/3020175214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ddc146bd0 0x7f0ddc143240 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f0ddc0c18e0 tx=0x7f0dcc009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:43.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0dd37fe700 1 -- 172.21.15.203:0/3020175214 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc003070 con 0x7f0ddc146bd0 2021-01-06T22:03:43.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0dd37fe700 1 -- 172.21.15.203:0/3020175214 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc0031d0 con 0x7f0ddc146bd0 2021-01-06T22:03:43.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0dd37fe700 1 -- 172.21.15.203:0/3020175214 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc003550 con 0x7f0ddc146bd0 2021-01-06T22:03:43.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0de0ec8700 1 -- 172.21.15.203:0/3020175214 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ddc15c060 con 0x7f0ddc146bd0 2021-01-06T22:03:43.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0de0ec8700 1 -- 172.21.15.203:0/3020175214 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ddc151770 con 0x7f0ddc146bd0 2021-01-06T22:03:43.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0de0ec8700 1 -- 172.21.15.203:0/3020175214 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ddc0b7750 con 0x7f0ddc146bd0 2021-01-06T22:03:43.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0dd37fe700 1 -- 172.21.15.203:0/3020175214 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f0dcc0018b0 con 0x7f0ddc146bd0 2021-01-06T22:03:43.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0dd37fe700 1 --2- 172.21.15.203:0/3020175214 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0dc0056400 0x7f0dc00588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:43.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0dd37fe700 1 -- 172.21.15.203:0/3020175214 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0dcc06eea0 con 0x7f0ddc146bd0 2021-01-06T22:03:43.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0ddb7fe700 1 --2- 172.21.15.203:0/3020175214 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0dc0056400 0x7f0dc00588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:43.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.939+0000 7f0ddb7fe700 1 --2- 172.21.15.203:0/3020175214 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0dc0056400 0x7f0dc00588c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f0dd4006a80 tx=0x7f0dd4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:43.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:43.947+0000 7f0dd37fe700 1 -- 172.21.15.203:0/3020175214 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f0dcc044720 con 0x7f0ddc146bd0 2021-01-06T22:03:44.149 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.147+0000 7f0de0ec8700 1 -- 172.21.15.203:0/3020175214 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ddc14f1f0 con 0x7f0ddc146bd0 2021-01-06T22:03:44.150 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.147+0000 7f0dd37fe700 1 -- 172.21.15.203:0/3020175214 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+3576 (secure 0 0 0) 0x7f0dcc022e10 con 0x7f0ddc146bd0 2021-01-06T22:03:44.151 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:44.151 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":5}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:03:43.790556+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"c","rank":2,"incarnation":11,"state":"up:creating","state_seq":1,"addr":"172.21.15.203:6835/1116012025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1116012025},{"type":"v1","addr":"172.21.15.203:6835","nonce":1116012025}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:03:44.153 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.151+0000 7f0de0ec8700 1 -- 172.21.15.203:0/3020175214 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0dc0056400 msgr2=0x7f0dc00588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.153 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.151+0000 7f0de0ec8700 1 --2- 172.21.15.203:0/3020175214 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0dc0056400 0x7f0dc00588c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f0dd4006a80 tx=0x7f0dd4008040).stop 2021-01-06T22:03:44.153 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.151+0000 7f0de0ec8700 1 -- 172.21.15.203:0/3020175214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ddc146bd0 msgr2=0x7f0ddc143240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.153 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.151+0000 7f0de0ec8700 1 --2- 172.21.15.203:0/3020175214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ddc146bd0 0x7f0ddc143240 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f0ddc0c18e0 tx=0x7f0dcc009040).stop 2021-01-06T22:03:44.154 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.151+0000 7f0de0ec8700 1 -- 172.21.15.203:0/3020175214 shutdown_connections 2021-01-06T22:03:44.154 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.151+0000 7f0de0ec8700 1 --2- 172.21.15.203:0/3020175214 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0dc0056400 0x7f0dc00588c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.154 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.151+0000 7f0de0ec8700 1 --2- 172.21.15.203:0/3020175214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ddc146bd0 0x7f0ddc143240 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.154 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.151+0000 7f0de0ec8700 1 --2- 172.21.15.203:0/3020175214 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ddc1467f0 0x7f0ddc15c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.155 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.151+0000 7f0de0ec8700 1 --2- 172.21.15.203:0/3020175214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ddc151230 0x7f0ddc15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.155 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.151+0000 7f0de0ec8700 1 -- 172.21.15.203:0/3020175214 >> 172.21.15.203:0/3020175214 conn(0x7f0ddc0667b0 msgr2=0x7f0ddc0bd880 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:44.155 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.151+0000 7f0de0ec8700 1 -- 172.21.15.203:0/3020175214 shutdown_connections 2021-01-06T22:03:44.155 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.151+0000 7f0de0ec8700 1 -- 172.21.15.203:0/3020175214 wait complete. 2021-01-06T22:03:44.156 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 11 2021-01-06T22:03:44.166 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-06T22:03:44.292 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cb11c6700 1 Processor -- start 2021-01-06T22:03:44.293 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cb11c6700 1 -- start start 2021-01-06T22:03:44.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cb11c6700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2cac0b97a0 0x7f2cac0b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cb11c6700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cac0be410 con 0x7f2cac0bd420 2021-01-06T22:03:44.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cb11c6700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cac066510 con 0x7f2cac0bed80 2021-01-06T22:03:44.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cb11c6700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cac0667b0 con 0x7f2cac0b97a0 2021-01-06T22:03:44.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cb09c5700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f2cac0bed80 0x7f2cac14c030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47206/0 (socket says 172.21.15.203:47206) 2021-01-06T22:03:44.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cb09c5700 1 -- 172.21.15.203:0/1256307366 learned_addr learned my addr 172.21.15.203:0/1256307366 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:44.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cabfff700 1 --2- 172.21.15.203:0/1256307366 >> v2:172.21.15.91:3300/0 conn(0x7f2cac0b97a0 0x7f2cac0b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cab7fe700 1 -- 172.21.15.203:0/1256307366 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161807847 0 0) 0x7f2cac066510 con 0x7f2cac0bed80 2021-01-06T22:03:44.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cab7fe700 1 -- 172.21.15.203:0/1256307366 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c8c003230 con 0x7f2cac0bed80 2021-01-06T22:03:44.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cab7fe700 1 -- 172.21.15.203:0/1256307366 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3892532076 0 0) 0x7f2c8c003230 con 0x7f2cac0bed80 2021-01-06T22:03:44.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cab7fe700 1 -- 172.21.15.203:0/1256307366 >> v2:172.21.15.91:3300/0 conn(0x7f2cac0b97a0 msgr2=0x7f2cac0b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:44.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cab7fe700 1 --2- 172.21.15.203:0/1256307366 >> v2:172.21.15.91:3300/0 conn(0x7f2cac0b97a0 0x7f2cac0b9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cab7fe700 1 -- 172.21.15.203:0/1256307366 >> v1:172.21.15.91:6789/0 conn(0x7f2cac0bd420 legacy=0x7f2cac065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cab7fe700 1 -- 172.21.15.203:0/1256307366 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac066a50 con 0x7f2cac0bed80 2021-01-06T22:03:44.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cab7fe700 1 -- 172.21.15.203:0/1256307366 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f2ca4001860 con 0x7f2cac0bed80 2021-01-06T22:03:44.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cab7fe700 1 -- 172.21.15.203:0/1256307366 >> v1:172.21.15.203:6789/0 conn(0x7f2cac0bed80 legacy=0x7f2cac14c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cab7fe700 1 --2- 172.21.15.203:0/1256307366 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c8c003720 0x7f2c8c003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cab7fe700 1 --2- 172.21.15.203:0/1256307366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c8c0044d0 0x7f2c8c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cab7fe700 1 --2- 172.21.15.203:0/1256307366 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c004e40 0x7f2c8c00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cab7fe700 1 -- 172.21.15.203:0/1256307366 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac0667b0 con 0x7f2c8c003720 2021-01-06T22:03:44.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cab7fe700 1 -- 172.21.15.203:0/1256307366 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c0041c0 con 0x7f2c8c0044d0 2021-01-06T22:03:44.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cab7fe700 1 -- 172.21.15.203:0/1256307366 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c00bc10 con 0x7f2c8c004e40 2021-01-06T22:03:44.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2cabfff700 1 --2- 172.21.15.203:0/1256307366 >> v2:172.21.15.91:3300/0 conn(0x7f2cac0b97a0 0x7f2cac0b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:44.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.291+0000 7f2ca3fff700 1 --2- 172.21.15.203:0/1256307366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c8c0044d0 0x7f2c8c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2cb09c5700 1 --2- 172.21.15.203:0/1256307366 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c004e40 0x7f2c8c00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2cabfff700 1 --2- 172.21.15.203:0/1256307366 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c8c003720 0x7f2c8c003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2ca3fff700 1 -- 172.21.15.203:0/1256307366 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c004e40 msgr2=0x7f2c8c00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:44.304 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2ca3fff700 1 --2- 172.21.15.203:0/1256307366 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c004e40 0x7f2c8c00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.304 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2ca3fff700 1 -- 172.21.15.203:0/1256307366 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c8c003720 msgr2=0x7f2c8c003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:44.304 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2ca3fff700 1 --2- 172.21.15.203:0/1256307366 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c8c003720 0x7f2c8c003b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.304 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2cb09c5700 1 -- 172.21.15.203:0/1256307366 reap_dead start 2021-01-06T22:03:44.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2ca3fff700 1 -- 172.21.15.203:0/1256307366 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c003230 con 0x7f2c8c0044d0 2021-01-06T22:03:44.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2cabfff700 1 --2- 172.21.15.203:0/1256307366 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c8c003720 0x7f2c8c003b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:44.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2ca3fff700 1 --2- 172.21.15.203:0/1256307366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c8c0044d0 0x7f2c8c004950 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f2c9c001920 tx=0x7f2c9c007b80).ready entity=mon.1 client_cookie=5f4ab202c4b06636 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:44.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2cab7fe700 1 -- 172.21.15.203:0/1256307366 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c007020 con 0x7f2c8c0044d0 2021-01-06T22:03:44.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2cab7fe700 1 -- 172.21.15.203:0/1256307366 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9c007180 con 0x7f2c8c0044d0 2021-01-06T22:03:44.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2cab7fe700 1 -- 172.21.15.203:0/1256307366 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c00c830 con 0x7f2c8c0044d0 2021-01-06T22:03:44.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2cb11c6700 1 -- 172.21.15.203:0/1256307366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c8c0044d0 msgr2=0x7f2c8c004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2cb11c6700 1 --2- 172.21.15.203:0/1256307366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c8c0044d0 0x7f2c8c004950 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f2c9c001920 tx=0x7f2c9c007b80).stop 2021-01-06T22:03:44.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2cb11c6700 1 -- 172.21.15.203:0/1256307366 shutdown_connections 2021-01-06T22:03:44.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2cb11c6700 1 --2- 172.21.15.203:0/1256307366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c8c0044d0 0x7f2c8c004950 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2cb11c6700 1 -- 172.21.15.203:0/1256307366 >> 172.21.15.203:0/1256307366 conn(0x7f2cac0648f0 msgr2=0x7f2cac064cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:44.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2cb11c6700 1 -- 172.21.15.203:0/1256307366 shutdown_connections 2021-01-06T22:03:44.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2cb11c6700 1 -- 172.21.15.203:0/1256307366 wait complete. 2021-01-06T22:03:44.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2cb11c6700 1 Processor -- start 2021-01-06T22:03:44.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.295+0000 7f2cb11c6700 1 -- start start 2021-01-06T22:03:44.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2cb11c6700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0044d0 0x7f2cac0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2cb11c6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cac0b6ff0 0x7f2cac15eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2cb11c6700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cac0b7470 0x7f2cac15f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2cb11c6700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac159340 con 0x7f2cac0b7470 2021-01-06T22:03:44.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2cb11c6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac0a2770 con 0x7f2cac0b6ff0 2021-01-06T22:03:44.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2cb11c6700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac0a1f90 con 0x7f2c8c0044d0 2021-01-06T22:03:44.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2ca3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cac0b6ff0 0x7f2cac15eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2ca3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cac0b6ff0 0x7f2cac15eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48622/0 (socket says 172.21.15.203:48622) 2021-01-06T22:03:44.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2ca3fff700 1 -- 172.21.15.203:0/4284247732 learned_addr learned my addr 172.21.15.203:0/4284247732 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:44.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2cb09c5700 1 --2- 172.21.15.203:0/4284247732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cac0b7470 0x7f2cac15f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2cabfff700 1 --2- 172.21.15.203:0/4284247732 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0044d0 0x7f2cac0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2ca3fff700 1 -- 172.21.15.203:0/4284247732 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0044d0 msgr2=0x7f2cac0b8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2ca3fff700 1 --2- 172.21.15.203:0/4284247732 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0044d0 0x7f2cac0b8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2ca3fff700 1 -- 172.21.15.203:0/4284247732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cac0b7470 msgr2=0x7f2cac15f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2ca3fff700 1 --2- 172.21.15.203:0/4284247732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cac0b7470 0x7f2cac15f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2ca3fff700 1 -- 172.21.15.203:0/4284247732 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac0a2230 con 0x7f2cac0b6ff0 2021-01-06T22:03:44.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2ca3fff700 1 --2- 172.21.15.203:0/4284247732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cac0b6ff0 0x7f2cac15eb40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f2c9c000f10 tx=0x7f2c9c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:44.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2ca8ff9700 1 -- 172.21.15.203:0/4284247732 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0067c0 con 0x7f2cac0b6ff0 2021-01-06T22:03:44.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2cb11c6700 1 -- 172.21.15.203:0/4284247732 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac0a24d0 con 0x7f2cac0b6ff0 2021-01-06T22:03:44.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2ca8ff9700 1 -- 172.21.15.203:0/4284247732 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9c006960 con 0x7f2cac0b6ff0 2021-01-06T22:03:44.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2ca8ff9700 1 -- 172.21.15.203:0/4284247732 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c006d20 con 0x7f2cac0b6ff0 2021-01-06T22:03:44.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.299+0000 7f2cb11c6700 1 -- 172.21.15.203:0/4284247732 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac0b6b50 con 0x7f2cac0b6ff0 2021-01-06T22:03:44.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.303+0000 7f2cb11c6700 1 -- 172.21.15.203:0/4284247732 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cac0450a0 con 0x7f2cac0b6ff0 2021-01-06T22:03:44.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.303+0000 7f2ca8ff9700 1 -- 172.21.15.203:0/4284247732 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f2c9c026070 con 0x7f2cac0b6ff0 2021-01-06T22:03:44.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.303+0000 7f2ca8ff9700 1 --2- 172.21.15.203:0/4284247732 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2c900563d0 0x7f2c90058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.303+0000 7f2ca8ff9700 1 -- 172.21.15.203:0/4284247732 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2c9c070110 con 0x7f2cac0b6ff0 2021-01-06T22:03:44.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.303+0000 7f2cabfff700 1 --2- 172.21.15.203:0/4284247732 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2c900563d0 0x7f2c90058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.303+0000 7f2cabfff700 1 --2- 172.21.15.203:0/4284247732 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2c900563d0 0x7f2c90058890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f2c94006cf0 tx=0x7f2c94008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:44.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.307+0000 7f2ca8ff9700 1 -- 172.21.15.203:0/4284247732 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f2c9c045a10 con 0x7f2cac0b6ff0 2021-01-06T22:03:44.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.467+0000 7f2cb11c6700 1 -- 172.21.15.203:0/4284247732 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2cac0b9010 con 0x7f2cac0b6ff0 2021-01-06T22:03:44.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.467+0000 7f2ca8ff9700 1 -- 172.21.15.203:0/4284247732 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f2c9c049030 con 0x7f2cac0b6ff0 2021-01-06T22:03:44.471 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:44.471 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":23,"fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","created":"2021-01-06T22:03:21.322671+0000","modified":"2021-01-06T22:03:40.744826+0000","last_up_change":"2021-01-06T22:03:31.115611+0000","last_in_change":"2021-01-06T22:03:31.115611+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-06T22:03:25.161465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-06T22:03:35.070364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-01-06T22:03:37.862354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-01-06T22:03:38.738416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e0b35c09-147c-4ccc-b566-c48b81e25412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":18347},{"type":"v1","addr":"172.21.15.203:6815","nonce":18347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":18347},{"type":"v1","addr":"172.21.15.203:6818","nonce":18347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":18347},{"type":"v1","addr":"172.21.15.203:6831","nonce":18347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6821","nonce":18347},{"type":"v1","addr":"172.21.15.203:6827","nonce":18347}]},"public_addr":"172.21.15.203:6815/18347","cluster_addr":"172.21.15.203:6818/18347","heartbeat_back_addr":"172.21.15.203:6831/18347","heartbeat_front_addr":"172.21.15.203:6827/18347","state":["exists","up"]},{"osd":1,"uuid":"8add59a0-2d58-4c1b-a8d1-62049ee9d85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":18337},{"type":"v1","addr":"172.21.15.203:6803","nonce":18337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":18337},{"type":"v1","addr":"172.21.15.203:6805","nonce":18337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":18337},{"type":"v1","addr":"172.21.15.203:6809","nonce":18337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":18337},{"type":"v1","addr":"172.21.15.203:6807","nonce":18337}]},"public_addr":"172.21.15.203:6803/18337","cluster_addr":"172.21.15.203:6805/18337","heartbeat_back_addr":"172.21.15.203:6809/18337","heartbeat_front_addr":"172.21.15.203:6807/18337","state":["exists","up"]},{"osd":2,"uuid":"b294a411-c7c9-477e-8eae-3b7bd0563ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":18354},{"type":"v1","addr":"172.21.15.203:6811","nonce":18354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":18354},{"type":"v1","addr":"172.21.15.203:6819","nonce":18354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6825","nonce":18354},{"type":"v1","addr":"172.21.15.203:6828","nonce":18354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6823","nonce":18354},{"type":"v1","addr":"172.21.15.203:6824","nonce":18354}]},"public_addr":"172.21.15.203:6811/18354","cluster_addr":"172.21.15.203:6819/18354","heartbeat_back_addr":"172.21.15.203:6828/18354","heartbeat_front_addr":"172.21.15.203:6824/18354","state":["exists","up"]},{"osd":3,"uuid":"5878a8f7-1a42-41c3-9810-0ac41a54dd6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":18346},{"type":"v1","addr":"172.21.15.203:6817","nonce":18346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":18346},{"type":"v1","addr":"172.21.15.203:6822","nonce":18346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":18346},{"type":"v1","addr":"172.21.15.203:6833","nonce":18346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":18346},{"type":"v1","addr":"172.21.15.203:6829","nonce":18346}]},"public_addr":"172.21.15.203:6817/18346","cluster_addr":"172.21.15.203:6822/18346","heartbeat_back_addr":"172.21.15.203:6833/18346","heartbeat_front_addr":"172.21.15.203:6829/18346","state":["exists","up"]},{"osd":4,"uuid":"f641a345-81e4-4400-a44c-d2f82fef7f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":18370},{"type":"v1","addr":"172.21.15.91:6809","nonce":18370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":18370},{"type":"v1","addr":"172.21.15.91:6811","nonce":18370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":18370},{"type":"v1","addr":"172.21.15.91:6816","nonce":18370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":18370},{"type":"v1","addr":"172.21.15.91:6813","nonce":18370}]},"public_addr":"172.21.15.91:6809/18370","cluster_addr":"172.21.15.91:6811/18370","heartbeat_back_addr":"172.21.15.91:6816/18370","heartbeat_front_addr":"172.21.15.91:6813/18370","state":["exists","up"]},{"osd":5,"uuid":"c1ff150d-68c6-4e6b-abd4-76af27bb2103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":18372},{"type":"v1","addr":"172.21.15.91:6801","nonce":18372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":18372},{"type":"v1","addr":"172.21.15.91:6803","nonce":18372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":18372},{"type":"v1","addr":"172.21.15.91:6807","nonce":18372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":18372},{"type":"v1","addr":"172.21.15.91:6805","nonce":18372}]},"public_addr":"172.21.15.91:6801/18372","cluster_addr":"172.21.15.91:6803/18372","heartbeat_back_addr":"172.21.15.91:6807/18372","heartbeat_front_addr":"172.21.15.91:6805/18372","state":["exists","up"]},{"osd":6,"uuid":"35a58e1a-0a0a-4f02-a477-adc029038d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6815","nonce":18371},{"type":"v1","addr":"172.21.15.91:6817","nonce":18371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":18371},{"type":"v1","addr":"172.21.15.91:6819","nonce":18371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":18371},{"type":"v1","addr":"172.21.15.91:6823","nonce":18371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":18371},{"type":"v1","addr":"172.21.15.91:6821","nonce":18371}]},"public_addr":"172.21.15.91:6817/18371","cluster_addr":"172.21.15.91:6819/18371","heartbeat_back_addr":"172.21.15.91:6823/18371","heartbeat_front_addr":"172.21.15.91:6821/18371","state":["exists","up"]},{"osd":7,"uuid":"809a2d89-4303-482a-8558-aef907814234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":18373},{"type":"v1","addr":"172.21.15.91:6825","nonce":18373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":18373},{"type":"v1","addr":"172.21.15.91:6827","nonce":18373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":18373},{"type":"v1","addr":"172.21.15.91:6831","nonce":18373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":18373},{"type":"v1","addr":"172.21.15.91:6829","nonce":18373}]},"public_addr":"172.21.15.91:6825/18373","cluster_addr":"172.21.15.91:6827/18373","heartbeat_back_addr":"172.21.15.91:6831/18373","heartbeat_front_addr":"172.21.15.91:6829/18373","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.875052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.103604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.179225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.158512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.206073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.894840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.229708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.248192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-06T22:03:44.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.471+0000 7f2cb11c6700 1 -- 172.21.15.203:0/4284247732 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2c900563d0 msgr2=0x7f2c90058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.471+0000 7f2cb11c6700 1 --2- 172.21.15.203:0/4284247732 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2c900563d0 0x7f2c90058890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f2c94006cf0 tx=0x7f2c94008040).stop 2021-01-06T22:03:44.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.471+0000 7f2cb11c6700 1 -- 172.21.15.203:0/4284247732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cac0b6ff0 msgr2=0x7f2cac15eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.471+0000 7f2cb11c6700 1 --2- 172.21.15.203:0/4284247732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cac0b6ff0 0x7f2cac15eb40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f2c9c000f10 tx=0x7f2c9c015000).stop 2021-01-06T22:03:44.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.471+0000 7f2cb11c6700 1 -- 172.21.15.203:0/4284247732 shutdown_connections 2021-01-06T22:03:44.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.471+0000 7f2cb11c6700 1 --2- 172.21.15.203:0/4284247732 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2c900563d0 0x7f2c90058890 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.471+0000 7f2cb11c6700 1 --2- 172.21.15.203:0/4284247732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cac0b7470 0x7f2cac15f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.471+0000 7f2cb11c6700 1 --2- 172.21.15.203:0/4284247732 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0044d0 0x7f2cac0b8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.471+0000 7f2cb11c6700 1 --2- 172.21.15.203:0/4284247732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cac0b6ff0 0x7f2cac15eb40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.471+0000 7f2cb11c6700 1 -- 172.21.15.203:0/4284247732 >> 172.21.15.203:0/4284247732 conn(0x7f2cac0648f0 msgr2=0x7f2cac0c21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:44.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.471+0000 7f2cb11c6700 1 -- 172.21.15.203:0/4284247732 shutdown_connections 2021-01-06T22:03:44.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.471+0000 7f2cb11c6700 1 -- 172.21.15.203:0/4284247732 wait complete. 2021-01-06T22:03:44.491 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-06T22:03:44.492 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2021-01-06T22:03:44.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f9778d34700 1 Processor -- start 2021-01-06T22:03:44.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f9778d34700 1 -- start start 2021-01-06T22:03:44.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f9778d34700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f97740bcbd0 0x7f97740baf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f9778d34700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97740bc450 0x7f97740bb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f9778d34700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97740a4610 con 0x7f97740bcfb0 2021-01-06T22:03:44.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f9778d34700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9774064730 con 0x7f97740bcbd0 2021-01-06T22:03:44.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f9778d34700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97740649d0 con 0x7f97740bc450 2021-01-06T22:03:44.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f9772ffd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f97740bcfb0 0x7f977414e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47226/0 (socket says 172.21.15.203:47226) 2021-01-06T22:03:44.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f9772ffd700 1 -- 172.21.15.203:0/3617138815 learned_addr learned my addr 172.21.15.203:0/3617138815 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:44.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f97737fe700 1 --2- 172.21.15.203:0/3617138815 >> v2:172.21.15.203:3300/0 conn(0x7f97740bcbd0 0x7f97740baf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f9773fff700 1 --2- 172.21.15.203:0/3617138815 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97740bc450 0x7f97740bb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f97727fc700 1 -- 172.21.15.203:0/3617138815 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345201728 0 0) 0x7f97740a4610 con 0x7f97740bcfb0 2021-01-06T22:03:44.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f97727fc700 1 -- 172.21.15.203:0/3617138815 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9760003230 con 0x7f97740bcfb0 2021-01-06T22:03:44.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f97737fe700 1 -- 172.21.15.203:0/3617138815 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97740bc450 msgr2=0x7f97740bb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:44.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f97737fe700 1 --2- 172.21.15.203:0/3617138815 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97740bc450 0x7f97740bb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f97737fe700 1 -- 172.21.15.203:0/3617138815 >> v1:172.21.15.203:6789/0 conn(0x7f97740bcfb0 legacy=0x7f977414e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f97737fe700 1 -- 172.21.15.203:0/3617138815 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97740a3e30 con 0x7f97740bcbd0 2021-01-06T22:03:44.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f9773fff700 1 --2- 172.21.15.203:0/3617138815 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97740bc450 0x7f97740bb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:44.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f97737fe700 1 --2- 172.21.15.203:0/3617138815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97740bcbd0 0x7f97740baf80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f976400b980 tx=0x7f9764006de0).ready entity=mon.1 client_cookie=1d30a3ea53f0c0e1 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:44.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.615+0000 7f97727fc700 1 -- 172.21.15.203:0/3617138815 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97640097b0 con 0x7f97740bcbd0 2021-01-06T22:03:44.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.619+0000 7f97727fc700 1 -- 172.21.15.203:0/3617138815 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9764009910 con 0x7f97740bcbd0 2021-01-06T22:03:44.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.619+0000 7f97727fc700 1 -- 172.21.15.203:0/3617138815 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764009c90 con 0x7f97740bcbd0 2021-01-06T22:03:44.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.619+0000 7f9778d34700 1 -- 172.21.15.203:0/3617138815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97740bcbd0 msgr2=0x7f97740baf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.619+0000 7f9778d34700 1 --2- 172.21.15.203:0/3617138815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97740bcbd0 0x7f97740baf80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f976400b980 tx=0x7f9764006de0).stop 2021-01-06T22:03:44.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.619+0000 7f9778d34700 1 -- 172.21.15.203:0/3617138815 shutdown_connections 2021-01-06T22:03:44.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.619+0000 7f9778d34700 1 --2- 172.21.15.203:0/3617138815 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97740bc450 0x7f97740bb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.619+0000 7f9778d34700 1 --2- 172.21.15.203:0/3617138815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97740bcbd0 0x7f97740baf80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.619+0000 7f9778d34700 1 -- 172.21.15.203:0/3617138815 >> 172.21.15.203:0/3617138815 conn(0x7f9774061530 msgr2=0x7f9774063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:44.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.619+0000 7f9778d34700 1 -- 172.21.15.203:0/3617138815 shutdown_connections 2021-01-06T22:03:44.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.619+0000 7f9778d34700 1 -- 172.21.15.203:0/3617138815 wait complete. 2021-01-06T22:03:44.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.619+0000 7f9778d34700 1 Processor -- start 2021-01-06T22:03:44.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.619+0000 7f9778d34700 1 -- start start 2021-01-06T22:03:44.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.619+0000 7f9778d34700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97740bc450 0x7f9774141040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.619+0000 7f9778d34700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97740bcbd0 0x7f97740bec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.619+0000 7f9778d34700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97740bcfb0 0x7f97740b8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.619+0000 7f9778d34700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97741406d0 con 0x7f97740bcfb0 2021-01-06T22:03:44.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f9778d34700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97740ba510 con 0x7f97740bc450 2021-01-06T22:03:44.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f9778d34700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977409f390 con 0x7f97740bcbd0 2021-01-06T22:03:44.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f97737fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97740bc450 0x7f9774141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f97737fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97740bc450 0x7f9774141040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48640/0 (socket says 172.21.15.203:48640) 2021-01-06T22:03:44.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f97737fe700 1 -- 172.21.15.203:0/1379866760 learned_addr learned my addr 172.21.15.203:0/1379866760 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:44.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f9773fff700 1 --2- 172.21.15.203:0/1379866760 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97740bcfb0 0x7f97740b8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f9772ffd700 1 --2- 172.21.15.203:0/1379866760 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97740bcbd0 0x7f97740bec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f97737fe700 1 -- 172.21.15.203:0/1379866760 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97740bcbd0 msgr2=0x7f97740bec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f97737fe700 1 --2- 172.21.15.203:0/1379866760 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97740bcbd0 0x7f97740bec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f97737fe700 1 -- 172.21.15.203:0/1379866760 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97740bcfb0 msgr2=0x7f97740b8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f97737fe700 1 --2- 172.21.15.203:0/1379866760 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97740bcfb0 0x7f97740b8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f97737fe700 1 -- 172.21.15.203:0/1379866760 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977409f630 con 0x7f97740bc450 2021-01-06T22:03:44.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f9772ffd700 1 --2- 172.21.15.203:0/1379866760 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97740bcbd0 0x7f97740bec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:44.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f97737fe700 1 --2- 172.21.15.203:0/1379866760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97740bc450 0x7f9774141040 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f9774057cc0 tx=0x7f9764016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:44.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f975b7fe700 1 -- 172.21.15.203:0/1379866760 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764012dd0 con 0x7f97740bc450 2021-01-06T22:03:44.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f975b7fe700 1 -- 172.21.15.203:0/1379866760 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976401c420 con 0x7f97740bc450 2021-01-06T22:03:44.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f9778d34700 1 -- 172.21.15.203:0/1379866760 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977409f8d0 con 0x7f97740bc450 2021-01-06T22:03:44.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f975b7fe700 1 -- 172.21.15.203:0/1379866760 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976401c7c0 con 0x7f97740bc450 2021-01-06T22:03:44.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f9778d34700 1 -- 172.21.15.203:0/1379866760 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9774140ba0 con 0x7f97740bc450 2021-01-06T22:03:44.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.623+0000 7f9778d34700 1 -- 172.21.15.203:0/1379866760 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9774140d60 con 0x7f97740bc450 2021-01-06T22:03:44.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.627+0000 7f975b7fe700 1 -- 172.21.15.203:0/1379866760 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f976400f070 con 0x7f97740bc450 2021-01-06T22:03:44.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.627+0000 7f975b7fe700 1 --2- 172.21.15.203:0/1379866760 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f97540563d0 0x7f9754058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.627+0000 7f975b7fe700 1 -- 172.21.15.203:0/1379866760 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9764070ac0 con 0x7f97740bc450 2021-01-06T22:03:44.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.627+0000 7f9772ffd700 1 --2- 172.21.15.203:0/1379866760 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f97540563d0 0x7f9754058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.627+0000 7f9772ffd700 1 --2- 172.21.15.203:0/1379866760 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f97540563d0 0x7f9754058890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f9768007fd0 tx=0x7f9768009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:44.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.631+0000 7f975b7fe700 1 -- 172.21.15.203:0/1379866760 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f9764046340 con 0x7f97740bc450 2021-01-06T22:03:44.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.783+0000 7f9778d34700 1 -- 172.21.15.203:0/1379866760 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f977405ca60 con 0x7f97740bc450 2021-01-06T22:03:44.787 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.787+0000 7f975b7fe700 1 -- 172.21.15.203:0/1379866760 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f9764049960 con 0x7f97740bc450 2021-01-06T22:03:44.788 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:44.788 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":23,"fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","created":"2021-01-06T22:03:21.322671+0000","modified":"2021-01-06T22:03:40.744826+0000","last_up_change":"2021-01-06T22:03:31.115611+0000","last_in_change":"2021-01-06T22:03:31.115611+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-06T22:03:25.161465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-06T22:03:35.070364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-01-06T22:03:37.862354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-01-06T22:03:38.738416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e0b35c09-147c-4ccc-b566-c48b81e25412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":18347},{"type":"v1","addr":"172.21.15.203:6815","nonce":18347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":18347},{"type":"v1","addr":"172.21.15.203:6818","nonce":18347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":18347},{"type":"v1","addr":"172.21.15.203:6831","nonce":18347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6821","nonce":18347},{"type":"v1","addr":"172.21.15.203:6827","nonce":18347}]},"public_addr":"172.21.15.203:6815/18347","cluster_addr":"172.21.15.203:6818/18347","heartbeat_back_addr":"172.21.15.203:6831/18347","heartbeat_front_addr":"172.21.15.203:6827/18347","state":["exists","up"]},{"osd":1,"uuid":"8add59a0-2d58-4c1b-a8d1-62049ee9d85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":18337},{"type":"v1","addr":"172.21.15.203:6803","nonce":18337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":18337},{"type":"v1","addr":"172.21.15.203:6805","nonce":18337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":18337},{"type":"v1","addr":"172.21.15.203:6809","nonce":18337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":18337},{"type":"v1","addr":"172.21.15.203:6807","nonce":18337}]},"public_addr":"172.21.15.203:6803/18337","cluster_addr":"172.21.15.203:6805/18337","heartbeat_back_addr":"172.21.15.203:6809/18337","heartbeat_front_addr":"172.21.15.203:6807/18337","state":["exists","up"]},{"osd":2,"uuid":"b294a411-c7c9-477e-8eae-3b7bd0563ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":18354},{"type":"v1","addr":"172.21.15.203:6811","nonce":18354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":18354},{"type":"v1","addr":"172.21.15.203:6819","nonce":18354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6825","nonce":18354},{"type":"v1","addr":"172.21.15.203:6828","nonce":18354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6823","nonce":18354},{"type":"v1","addr":"172.21.15.203:6824","nonce":18354}]},"public_addr":"172.21.15.203:6811/18354","cluster_addr":"172.21.15.203:6819/18354","heartbeat_back_addr":"172.21.15.203:6828/18354","heartbeat_front_addr":"172.21.15.203:6824/18354","state":["exists","up"]},{"osd":3,"uuid":"5878a8f7-1a42-41c3-9810-0ac41a54dd6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":18346},{"type":"v1","addr":"172.21.15.203:6817","nonce":18346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":18346},{"type":"v1","addr":"172.21.15.203:6822","nonce":18346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":18346},{"type":"v1","addr":"172.21.15.203:6833","nonce":18346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":18346},{"type":"v1","addr":"172.21.15.203:6829","nonce":18346}]},"public_addr":"172.21.15.203:6817/18346","cluster_addr":"172.21.15.203:6822/18346","heartbeat_back_addr":"172.21.15.203:6833/18346","heartbeat_front_addr":"172.21.15.203:6829/18346","state":["exists","up"]},{"osd":4,"uuid":"f641a345-81e4-4400-a44c-d2f82fef7f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":18370},{"type":"v1","addr":"172.21.15.91:6809","nonce":18370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":18370},{"type":"v1","addr":"172.21.15.91:6811","nonce":18370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":18370},{"type":"v1","addr":"172.21.15.91:6816","nonce":18370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":18370},{"type":"v1","addr":"172.21.15.91:6813","nonce":18370}]},"public_addr":"172.21.15.91:6809/18370","cluster_addr":"172.21.15.91:6811/18370","heartbeat_back_addr":"172.21.15.91:6816/18370","heartbeat_front_addr":"172.21.15.91:6813/18370","state":["exists","up"]},{"osd":5,"uuid":"c1ff150d-68c6-4e6b-abd4-76af27bb2103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":18372},{"type":"v1","addr":"172.21.15.91:6801","nonce":18372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":18372},{"type":"v1","addr":"172.21.15.91:6803","nonce":18372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":18372},{"type":"v1","addr":"172.21.15.91:6807","nonce":18372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":18372},{"type":"v1","addr":"172.21.15.91:6805","nonce":18372}]},"public_addr":"172.21.15.91:6801/18372","cluster_addr":"172.21.15.91:6803/18372","heartbeat_back_addr":"172.21.15.91:6807/18372","heartbeat_front_addr":"172.21.15.91:6805/18372","state":["exists","up"]},{"osd":6,"uuid":"35a58e1a-0a0a-4f02-a477-adc029038d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6815","nonce":18371},{"type":"v1","addr":"172.21.15.91:6817","nonce":18371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":18371},{"type":"v1","addr":"172.21.15.91:6819","nonce":18371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":18371},{"type":"v1","addr":"172.21.15.91:6823","nonce":18371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":18371},{"type":"v1","addr":"172.21.15.91:6821","nonce":18371}]},"public_addr":"172.21.15.91:6817/18371","cluster_addr":"172.21.15.91:6819/18371","heartbeat_back_addr":"172.21.15.91:6823/18371","heartbeat_front_addr":"172.21.15.91:6821/18371","state":["exists","up"]},{"osd":7,"uuid":"809a2d89-4303-482a-8558-aef907814234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":18373},{"type":"v1","addr":"172.21.15.91:6825","nonce":18373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":18373},{"type":"v1","addr":"172.21.15.91:6827","nonce":18373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":18373},{"type":"v1","addr":"172.21.15.91:6831","nonce":18373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":18373},{"type":"v1","addr":"172.21.15.91:6829","nonce":18373}]},"public_addr":"172.21.15.91:6825/18373","cluster_addr":"172.21.15.91:6827/18373","heartbeat_back_addr":"172.21.15.91:6831/18373","heartbeat_front_addr":"172.21.15.91:6829/18373","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.875052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.103604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.179225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.158512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.206073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.894840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.229708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.248192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-06T22:03:44.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.787+0000 7f9778d34700 1 -- 172.21.15.203:0/1379866760 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f97540563d0 msgr2=0x7f9754058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.787+0000 7f9778d34700 1 --2- 172.21.15.203:0/1379866760 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f97540563d0 0x7f9754058890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f9768007fd0 tx=0x7f9768009040).stop 2021-01-06T22:03:44.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.787+0000 7f9778d34700 1 -- 172.21.15.203:0/1379866760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97740bc450 msgr2=0x7f9774141040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.787+0000 7f9778d34700 1 --2- 172.21.15.203:0/1379866760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97740bc450 0x7f9774141040 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f9774057cc0 tx=0x7f9764016000).stop 2021-01-06T22:03:44.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.787+0000 7f9778d34700 1 -- 172.21.15.203:0/1379866760 shutdown_connections 2021-01-06T22:03:44.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.787+0000 7f9778d34700 1 --2- 172.21.15.203:0/1379866760 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f97540563d0 0x7f9754058890 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.787+0000 7f9778d34700 1 --2- 172.21.15.203:0/1379866760 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97740bcfb0 0x7f97740b8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.787+0000 7f9778d34700 1 --2- 172.21.15.203:0/1379866760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97740bc450 0x7f9774141040 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.787+0000 7f9778d34700 1 --2- 172.21.15.203:0/1379866760 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97740bcbd0 0x7f97740bec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.787+0000 7f9778d34700 1 -- 172.21.15.203:0/1379866760 >> 172.21.15.203:0/1379866760 conn(0x7f9774061530 msgr2=0x7f97740c07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:44.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.787+0000 7f9778d34700 1 -- 172.21.15.203:0/1379866760 shutdown_connections 2021-01-06T22:03:44.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.787+0000 7f9778d34700 1 -- 172.21.15.203:0/1379866760 wait complete. 2021-01-06T22:03:44.805 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-01-06T22:03:25.161465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-01-06T22:03:35.070364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-01-06T22:03:37.862354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-01-06T22:03:38.738416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-01-06T22:03:44.805 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-01-06T22:03:44.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d26488700 1 Processor -- start 2021-01-06T22:03:44.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d26488700 1 -- start start 2021-01-06T22:03:44.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d26488700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8d200c08a0 0x7f8d200680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d26488700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8d20155b60 0x7f8d20156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d26488700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d201531f0 con 0x7f8d200c04c0 2021-01-06T22:03:44.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d26488700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d201522e0 con 0x7f8d20155b60 2021-01-06T22:03:44.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d26488700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d200645f0 con 0x7f8d200c08a0 2021-01-06T22:03:44.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d1ffff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f8d200c04c0 0x7f8d20063ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47244/0 (socket says 172.21.15.203:47244) 2021-01-06T22:03:44.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d1ffff700 1 -- 172.21.15.203:0/1518981307 learned_addr learned my addr 172.21.15.203:0/1518981307 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:44.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d1f7fe700 1 --2- 172.21.15.203:0/1518981307 >> v2:172.21.15.203:3300/0 conn(0x7f8d200c08a0 0x7f8d200680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d24a25700 1 --2- 172.21.15.203:0/1518981307 >> v2:172.21.15.91:3300/0 conn(0x7f8d20155b60 0x7f8d20156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d1effd700 1 -- 172.21.15.203:0/1518981307 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520175782 0 0) 0x7f8d201531f0 con 0x7f8d200c04c0 2021-01-06T22:03:44.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d1effd700 1 -- 172.21.15.203:0/1518981307 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d0c003230 con 0x7f8d200c04c0 2021-01-06T22:03:44.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d1f7fe700 1 -- 172.21.15.203:0/1518981307 >> v2:172.21.15.91:3300/0 conn(0x7f8d20155b60 msgr2=0x7f8d20156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:44.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d1f7fe700 1 --2- 172.21.15.203:0/1518981307 >> v2:172.21.15.91:3300/0 conn(0x7f8d20155b60 0x7f8d20156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d1f7fe700 1 -- 172.21.15.203:0/1518981307 >> v1:172.21.15.203:6789/0 conn(0x7f8d200c04c0 legacy=0x7f8d20063ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d1f7fe700 1 -- 172.21.15.203:0/1518981307 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d20156500 con 0x7f8d200c08a0 2021-01-06T22:03:44.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d1f7fe700 1 --2- 172.21.15.203:0/1518981307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d200c08a0 0x7f8d200680b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f8d201533a0 tx=0x7f8d14006de0).ready entity=mon.1 client_cookie=d517cdf3c748cf9a server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:44.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.931+0000 7f8d1effd700 1 -- 172.21.15.203:0/1518981307 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d140096d0 con 0x7f8d200c08a0 2021-01-06T22:03:44.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.935+0000 7f8d1effd700 1 -- 172.21.15.203:0/1518981307 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d14009830 con 0x7f8d200c08a0 2021-01-06T22:03:44.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.935+0000 7f8d1effd700 1 -- 172.21.15.203:0/1518981307 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14009bb0 con 0x7f8d200c08a0 2021-01-06T22:03:44.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.935+0000 7f8d26488700 1 -- 172.21.15.203:0/1518981307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d200c08a0 msgr2=0x7f8d200680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.935+0000 7f8d26488700 1 --2- 172.21.15.203:0/1518981307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d200c08a0 0x7f8d200680b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f8d201533a0 tx=0x7f8d14006de0).stop 2021-01-06T22:03:44.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.935+0000 7f8d26488700 1 -- 172.21.15.203:0/1518981307 shutdown_connections 2021-01-06T22:03:44.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.935+0000 7f8d26488700 1 --2- 172.21.15.203:0/1518981307 >> v2:172.21.15.91:3300/0 conn(0x7f8d20155b60 0x7f8d20156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.935+0000 7f8d26488700 1 --2- 172.21.15.203:0/1518981307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d200c08a0 0x7f8d200680b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.935+0000 7f8d26488700 1 -- 172.21.15.203:0/1518981307 >> 172.21.15.203:0/1518981307 conn(0x7f8d20056cd0 msgr2=0x7f8d200626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:44.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.935+0000 7f8d26488700 1 -- 172.21.15.203:0/1518981307 shutdown_connections 2021-01-06T22:03:44.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.935+0000 7f8d26488700 1 -- 172.21.15.203:0/1518981307 wait complete. 2021-01-06T22:03:44.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.935+0000 7f8d26488700 1 Processor -- start 2021-01-06T22:03:44.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.935+0000 7f8d26488700 1 -- start start 2021-01-06T22:03:44.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d26488700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d200c04c0 0x7f8d20165fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d26488700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8d20155b60 0x7f8d20161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d26488700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d200b7d10 0x7f8d200bac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d26488700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d201623a0 con 0x7f8d200b7d10 2021-01-06T22:03:44.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d26488700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200ae2b0 con 0x7f8d200c04c0 2021-01-06T22:03:44.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d26488700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200afaa0 con 0x7f8d20155b60 2021-01-06T22:03:44.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d1ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d200c04c0 0x7f8d20165fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d1ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d200c04c0 0x7f8d20165fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48656/0 (socket says 172.21.15.203:48656) 2021-01-06T22:03:44.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d1ffff700 1 -- 172.21.15.203:0/1180875167 learned_addr learned my addr 172.21.15.203:0/1180875167 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:44.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d24a25700 1 --2- 172.21.15.203:0/1180875167 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d200b7d10 0x7f8d200bac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d1f7fe700 1 --2- 172.21.15.203:0/1180875167 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8d20155b60 0x7f8d20161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d1ffff700 1 -- 172.21.15.203:0/1180875167 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8d20155b60 msgr2=0x7f8d20161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d1ffff700 1 --2- 172.21.15.203:0/1180875167 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8d20155b60 0x7f8d20161eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d1ffff700 1 -- 172.21.15.203:0/1180875167 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d200b7d10 msgr2=0x7f8d200bac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:44.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d1ffff700 1 --2- 172.21.15.203:0/1180875167 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d200b7d10 0x7f8d200bac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:44.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d1ffff700 1 -- 172.21.15.203:0/1180875167 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d200afd40 con 0x7f8d200c04c0 2021-01-06T22:03:44.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d1ffff700 1 --2- 172.21.15.203:0/1180875167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d200c04c0 0x7f8d20165fe0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f8d1000c6e0 tx=0x7f8d10007ab0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:44.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d07fff700 1 -- 172.21.15.203:0/1180875167 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d100072e0 con 0x7f8d200c04c0 2021-01-06T22:03:44.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d07fff700 1 -- 172.21.15.203:0/1180875167 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d10007440 con 0x7f8d200c04c0 2021-01-06T22:03:44.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d07fff700 1 -- 172.21.15.203:0/1180875167 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10005810 con 0x7f8d200c04c0 2021-01-06T22:03:44.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d26488700 1 -- 172.21.15.203:0/1180875167 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d20165600 con 0x7f8d200c04c0 2021-01-06T22:03:44.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d26488700 1 -- 172.21.15.203:0/1180875167 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d20165b40 con 0x7f8d200c04c0 2021-01-06T22:03:44.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.939+0000 7f8d26488700 1 -- 172.21.15.203:0/1180875167 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d20165d00 con 0x7f8d200c04c0 2021-01-06T22:03:44.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.943+0000 7f8d07fff700 1 -- 172.21.15.203:0/1180875167 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f8d10010070 con 0x7f8d200c04c0 2021-01-06T22:03:44.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.943+0000 7f8d07fff700 1 --2- 172.21.15.203:0/1180875167 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8d000563d0 0x7f8d00058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:44.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.943+0000 7f8d07fff700 1 -- 172.21.15.203:0/1180875167 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8d1006ec00 con 0x7f8d200c04c0 2021-01-06T22:03:44.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.943+0000 7f8d1f7fe700 1 --2- 172.21.15.203:0/1180875167 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8d000563d0 0x7f8d00058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:44.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.943+0000 7f8d1f7fe700 1 --2- 172.21.15.203:0/1180875167 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8d000563d0 0x7f8d00058890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f8d1400b000 tx=0x7f8d14016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:44.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:44.947+0000 7f8d07fff700 1 -- 172.21.15.203:0/1180875167 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f8d10074020 con 0x7f8d200c04c0 2021-01-06T22:03:45.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.103+0000 7f8d26488700 1 -- 172.21.15.203:0/1180875167 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8d20064e60 con 0x7f8d200c04c0 2021-01-06T22:03:45.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.103+0000 7f8d07fff700 1 -- 172.21.15.203:0/1180875167 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8d10018030 con 0x7f8d200c04c0 2021-01-06T22:03:45.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.103+0000 7f8d26488700 1 -- 172.21.15.203:0/1180875167 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8d000563d0 msgr2=0x7f8d00058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:45.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.103+0000 7f8d26488700 1 --2- 172.21.15.203:0/1180875167 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8d000563d0 0x7f8d00058890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f8d1400b000 tx=0x7f8d14016000).stop 2021-01-06T22:03:45.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.103+0000 7f8d26488700 1 -- 172.21.15.203:0/1180875167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d200c04c0 msgr2=0x7f8d20165fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:45.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.103+0000 7f8d26488700 1 --2- 172.21.15.203:0/1180875167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d200c04c0 0x7f8d20165fe0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f8d1000c6e0 tx=0x7f8d10007ab0).stop 2021-01-06T22:03:45.348 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.103+0000 7f8d26488700 1 -- 172.21.15.203:0/1180875167 shutdown_connections 2021-01-06T22:03:45.348 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.103+0000 7f8d26488700 1 --2- 172.21.15.203:0/1180875167 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8d000563d0 0x7f8d00058890 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.348 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.103+0000 7f8d26488700 1 --2- 172.21.15.203:0/1180875167 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d200b7d10 0x7f8d200bac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.348 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.103+0000 7f8d26488700 1 --2- 172.21.15.203:0/1180875167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d200c04c0 0x7f8d20165fe0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.348 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.103+0000 7f8d26488700 1 --2- 172.21.15.203:0/1180875167 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8d20155b60 0x7f8d20161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.103+0000 7f8d26488700 1 -- 172.21.15.203:0/1180875167 >> 172.21.15.203:0/1180875167 conn(0x7f8d20056cd0 msgr2=0x7f8d200626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:45.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.107+0000 7f8d26488700 1 -- 172.21.15.203:0/1180875167 shutdown_connections 2021-01-06T22:03:45.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.107+0000 7f8d26488700 1 -- 172.21.15.203:0/1180875167 wait complete. 2021-01-06T22:03:45.349 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 1 2021-01-06T22:03:45.350 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-01-06T22:03:45.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb6759e700 1 Processor -- start 2021-01-06T22:03:45.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb6759e700 1 -- start start 2021-01-06T22:03:45.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb6759e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb680c0fe0 0x7fcb680aa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:45.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb6759e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcb680c03f0 0x7fcb68060d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:45.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb6759e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb680a9ca0 con 0x7fcb680c13c0 2021-01-06T22:03:45.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb6759e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb68066a50 con 0x7fcb680c03f0 2021-01-06T22:03:45.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb6759e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb68066cf0 con 0x7fcb680c0fe0 2021-01-06T22:03:45.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb66d9d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fcb680c13c0 0x7fcb68151320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47262/0 (socket says 172.21.15.203:47262) 2021-01-06T22:03:45.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb5dd9b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcb680c03f0 0x7fcb68060d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:45.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb66d9d700 1 -- 172.21.15.203:0/3972867467 learned_addr learned my addr 172.21.15.203:0/3972867467 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:45.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb6659c700 1 --2- 172.21.15.203:0/3972867467 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb680c0fe0 0x7fcb680aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:45.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb65d9b700 1 -- 172.21.15.203:0/3972867467 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532310719 0 0) 0x7fcb680a9ca0 con 0x7fcb680c13c0 2021-01-06T22:03:45.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb65d9b700 1 -- 172.21.15.203:0/3972867467 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb48003230 con 0x7fcb680c13c0 2021-01-06T22:03:45.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb5dd9b700 1 -- 172.21.15.203:0/3972867467 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb680c0fe0 msgr2=0x7fcb680aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:45.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb5dd9b700 1 --2- 172.21.15.203:0/3972867467 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb680c0fe0 0x7fcb680aa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb5dd9b700 1 -- 172.21.15.203:0/3972867467 >> v1:172.21.15.203:6789/0 conn(0x7fcb680c13c0 legacy=0x7fcb68151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:45.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb5dd9b700 1 -- 172.21.15.203:0/3972867467 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb68068460 con 0x7fcb680c03f0 2021-01-06T22:03:45.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb6659c700 1 --2- 172.21.15.203:0/3972867467 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb680c0fe0 0x7fcb680aa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:45.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb5dd9b700 1 --2- 172.21.15.203:0/3972867467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb680c03f0 0x7fcb68060d90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fcb5000b980 tx=0x7fcb50006de0).ready entity=mon.1 client_cookie=2dd916205713b939 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:45.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb65d9b700 1 -- 172.21.15.203:0/3972867467 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb500097b0 con 0x7fcb680c03f0 2021-01-06T22:03:45.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb65d9b700 1 -- 172.21.15.203:0/3972867467 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb50009910 con 0x7fcb680c03f0 2021-01-06T22:03:45.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.487+0000 7fcb65d9b700 1 -- 172.21.15.203:0/3972867467 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50009c90 con 0x7fcb680c03f0 2021-01-06T22:03:45.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.491+0000 7fcb6759e700 1 -- 172.21.15.203:0/3972867467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb680c03f0 msgr2=0x7fcb68060d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:45.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.491+0000 7fcb6759e700 1 --2- 172.21.15.203:0/3972867467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb680c03f0 0x7fcb68060d90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fcb5000b980 tx=0x7fcb50006de0).stop 2021-01-06T22:03:45.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.491+0000 7fcb6759e700 1 -- 172.21.15.203:0/3972867467 shutdown_connections 2021-01-06T22:03:45.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.491+0000 7fcb6759e700 1 --2- 172.21.15.203:0/3972867467 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb680c0fe0 0x7fcb680aa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.491+0000 7fcb6759e700 1 --2- 172.21.15.203:0/3972867467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb680c03f0 0x7fcb68060d90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.491+0000 7fcb6759e700 1 -- 172.21.15.203:0/3972867467 >> 172.21.15.203:0/3972867467 conn(0x7fcb68061530 msgr2=0x7fcb68061910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:45.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.491+0000 7fcb6759e700 1 -- 172.21.15.203:0/3972867467 shutdown_connections 2021-01-06T22:03:45.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.491+0000 7fcb6759e700 1 -- 172.21.15.203:0/3972867467 wait complete. 2021-01-06T22:03:45.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.491+0000 7fcb6759e700 1 Processor -- start 2021-01-06T22:03:45.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.491+0000 7fcb6759e700 1 -- start start 2021-01-06T22:03:45.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb6759e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb680c03f0 0x7fcb6815deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:45.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb6759e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb680c0fe0 0x7fcb68161240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:45.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb6759e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb680c13c0 0x7fcb6814f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:45.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb6759e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6815e3a0 con 0x7fcb680c03f0 2021-01-06T22:03:45.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb6759e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb68151d50 con 0x7fcb680c0fe0 2021-01-06T22:03:45.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb6759e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb68151fb0 con 0x7fcb680c13c0 2021-01-06T22:03:45.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb5dd9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb680c0fe0 0x7fcb68161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:45.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb5dd9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb680c0fe0 0x7fcb68161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48672/0 (socket says 172.21.15.203:48672) 2021-01-06T22:03:45.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb5dd9b700 1 -- 172.21.15.203:0/373414277 learned_addr learned my addr 172.21.15.203:0/373414277 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:45.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb6659c700 1 --2- 172.21.15.203:0/373414277 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb680c03f0 0x7fcb6815deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:45.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb66d9d700 1 --2- 172.21.15.203:0/373414277 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb680c13c0 0x7fcb6814f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:45.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb5dd9b700 1 -- 172.21.15.203:0/373414277 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb680c13c0 msgr2=0x7fcb6814f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:45.508 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb5dd9b700 1 --2- 172.21.15.203:0/373414277 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb680c13c0 0x7fcb6814f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.508 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb5dd9b700 1 -- 172.21.15.203:0/373414277 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb680c03f0 msgr2=0x7fcb6815deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:45.508 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb5dd9b700 1 --2- 172.21.15.203:0/373414277 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb680c03f0 0x7fcb6815deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.508 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb5dd9b700 1 -- 172.21.15.203:0/373414277 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb68152250 con 0x7fcb680c0fe0 2021-01-06T22:03:45.508 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb5dd9b700 1 --2- 172.21.15.203:0/373414277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb680c0fe0 0x7fcb68161240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fcb68068200 tx=0x7fcb50016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:45.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb5effd700 1 -- 172.21.15.203:0/373414277 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50002f10 con 0x7fcb680c0fe0 2021-01-06T22:03:45.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb5effd700 1 -- 172.21.15.203:0/373414277 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb50012d20 con 0x7fcb680c0fe0 2021-01-06T22:03:45.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb5effd700 1 -- 172.21.15.203:0/373414277 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5001c630 con 0x7fcb680c0fe0 2021-01-06T22:03:45.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb6759e700 1 -- 172.21.15.203:0/373414277 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb68160b00 con 0x7fcb680c0fe0 2021-01-06T22:03:45.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb6759e700 1 -- 172.21.15.203:0/373414277 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb68161730 con 0x7fcb680c0fe0 2021-01-06T22:03:45.510 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb6759e700 1 -- 172.21.15.203:0/373414277 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb68068640 con 0x7fcb680c0fe0 2021-01-06T22:03:45.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.495+0000 7fcb5effd700 1 -- 172.21.15.203:0/373414277 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fcb5000f070 con 0x7fcb680c0fe0 2021-01-06T22:03:45.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.499+0000 7fcb5effd700 1 --2- 172.21.15.203:0/373414277 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fcb4c0563d0 0x7fcb4c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:45.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.499+0000 7fcb6659c700 1 --2- 172.21.15.203:0/373414277 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fcb4c0563d0 0x7fcb4c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:45.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.499+0000 7fcb5effd700 1 -- 172.21.15.203:0/373414277 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcb50070930 con 0x7fcb680c0fe0 2021-01-06T22:03:45.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.499+0000 7fcb6659c700 1 --2- 172.21.15.203:0/373414277 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fcb4c0563d0 0x7fcb4c058890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fcb5800bb10 tx=0x7fcb58008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:45.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.503+0000 7fcb5effd700 1 -- 172.21.15.203:0/373414277 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fcb500461b0 con 0x7fcb680c0fe0 2021-01-06T22:03:45.660 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.659+0000 7fcb6759e700 1 -- 172.21.15.203:0/373414277 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcb681523e0 con 0x7fcb680c0fe0 2021-01-06T22:03:45.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.659+0000 7fcb5effd700 1 -- 172.21.15.203:0/373414277 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcb500497d0 con 0x7fcb680c0fe0 2021-01-06T22:03:45.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.659+0000 7fcb6759e700 1 -- 172.21.15.203:0/373414277 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fcb4c0563d0 msgr2=0x7fcb4c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:45.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.659+0000 7fcb6759e700 1 --2- 172.21.15.203:0/373414277 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fcb4c0563d0 0x7fcb4c058890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fcb5800bb10 tx=0x7fcb58008040).stop 2021-01-06T22:03:45.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.659+0000 7fcb6759e700 1 -- 172.21.15.203:0/373414277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb680c0fe0 msgr2=0x7fcb68161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:45.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.659+0000 7fcb6759e700 1 --2- 172.21.15.203:0/373414277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb680c0fe0 0x7fcb68161240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fcb68068200 tx=0x7fcb50016000).stop 2021-01-06T22:03:45.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.659+0000 7fcb6759e700 1 -- 172.21.15.203:0/373414277 shutdown_connections 2021-01-06T22:03:45.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.659+0000 7fcb6759e700 1 --2- 172.21.15.203:0/373414277 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fcb4c0563d0 0x7fcb4c058890 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.659+0000 7fcb6759e700 1 --2- 172.21.15.203:0/373414277 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb680c13c0 0x7fcb6814f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.659+0000 7fcb6759e700 1 --2- 172.21.15.203:0/373414277 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb680c03f0 0x7fcb6815deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.659+0000 7fcb6759e700 1 --2- 172.21.15.203:0/373414277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb680c0fe0 0x7fcb68161240 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.659+0000 7fcb6759e700 1 -- 172.21.15.203:0/373414277 >> 172.21.15.203:0/373414277 conn(0x7fcb68061530 msgr2=0x7fcb68061910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:45.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.659+0000 7fcb6759e700 1 -- 172.21.15.203:0/373414277 shutdown_connections 2021-01-06T22:03:45.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.659+0000 7fcb6759e700 1 -- 172.21.15.203:0/373414277 wait complete. 2021-01-06T22:03:45.727 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:03:45.727 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-01-06T22:03:45.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.863+0000 7f78c759e700 1 Processor -- start 2021-01-06T22:03:45.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.863+0000 7f78c759e700 1 -- start start 2021-01-06T22:03:45.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.863+0000 7f78c759e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78c8063010 0x7f78c814e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:45.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.863+0000 7f78c759e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c80be990 con 0x7f78c80633f0 2021-01-06T22:03:45.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.863+0000 7f78c759e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c80be6f0 con 0x7f78c80be1b0 2021-01-06T22:03:45.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.863+0000 7f78c759e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c814ec60 con 0x7f78c8063010 2021-01-06T22:03:45.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c6d9d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f78c80be1b0 0x7f78c8147790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47276/0 (socket says 172.21.15.203:47276) 2021-01-06T22:03:45.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c6d9d700 1 -- 172.21.15.203:0/2354658057 learned_addr learned my addr 172.21.15.203:0/2354658057 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:45.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c659c700 1 --2- 172.21.15.203:0/2354658057 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78c8063010 0x7f78c814e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:45.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 -- 172.21.15.203:0/2354658057 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034334231 0 0) 0x7f78c80be6f0 con 0x7f78c80be1b0 2021-01-06T22:03:45.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 -- 172.21.15.203:0/2354658057 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78a8003230 con 0x7f78c80be1b0 2021-01-06T22:03:45.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 -- 172.21.15.203:0/2354658057 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3162517487 0 0) 0x7f78c80be990 con 0x7f78c80633f0 2021-01-06T22:03:45.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 -- 172.21.15.203:0/2354658057 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78c80be6f0 con 0x7f78c80633f0 2021-01-06T22:03:45.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 -- 172.21.15.203:0/2354658057 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 617322247 0 0) 0x7f78a8003230 con 0x7f78c80be1b0 2021-01-06T22:03:45.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 -- 172.21.15.203:0/2354658057 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78c8063010 msgr2=0x7f78c814e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:45.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 --2- 172.21.15.203:0/2354658057 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78c8063010 0x7f78c814e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 -- 172.21.15.203:0/2354658057 >> v1:172.21.15.91:6789/0 conn(0x7f78c80633f0 legacy=0x7f78c8064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:45.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 -- 172.21.15.203:0/2354658057 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c80c1050 con 0x7f78c80be1b0 2021-01-06T22:03:45.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 -- 172.21.15.203:0/2354658057 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f78b8001860 con 0x7f78c80be1b0 2021-01-06T22:03:45.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 -- 172.21.15.203:0/2354658057 >> v1:172.21.15.203:6789/0 conn(0x7f78c80be1b0 legacy=0x7f78c8147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:45.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 --2- 172.21.15.203:0/2354658057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a80038b0 0x7f78a8003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:45.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 --2- 172.21.15.203:0/2354658057 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a80044d0 0x7f78a8004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:45.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 --2- 172.21.15.203:0/2354658057 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78c8063010 0x7f78a8004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:45.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 -- 172.21.15.203:0/2354658057 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c814ec60 con 0x7f78a80044d0 2021-01-06T22:03:45.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 -- 172.21.15.203:0/2354658057 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a8005330 con 0x7f78a80038b0 2021-01-06T22:03:45.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 -- 172.21.15.203:0/2354658057 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a800b720 con 0x7f78c8063010 2021-01-06T22:03:45.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c659c700 1 --2- 172.21.15.203:0/2354658057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a80038b0 0x7f78a8003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:45.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c6d9d700 1 --2- 172.21.15.203:0/2354658057 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78c8063010 0x7f78a8004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:45.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c5d9b700 1 --2- 172.21.15.203:0/2354658057 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a80044d0 0x7f78a8004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:45.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c659c700 1 -- 172.21.15.203:0/2354658057 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78c8063010 msgr2=0x7f78a8004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:45.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c659c700 1 --2- 172.21.15.203:0/2354658057 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78c8063010 0x7f78a8004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c659c700 1 -- 172.21.15.203:0/2354658057 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a80044d0 msgr2=0x7f78a8004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:45.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c659c700 1 --2- 172.21.15.203:0/2354658057 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a80044d0 0x7f78a8004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c659c700 1 -- 172.21.15.203:0/2354658057 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a8003230 con 0x7f78a80038b0 2021-01-06T22:03:45.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c659c700 1 --2- 172.21.15.203:0/2354658057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a80038b0 0x7f78a8003cb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f78c000bbc0 tx=0x7f78c0008040).ready entity=mon.1 client_cookie=b4ecb0cc6ae68763 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:45.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 -- 172.21.15.203:0/2354658057 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c000b2b0 con 0x7f78a80038b0 2021-01-06T22:03:45.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 -- 172.21.15.203:0/2354658057 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c000eac0 con 0x7f78a80038b0 2021-01-06T22:03:45.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c559a700 1 -- 172.21.15.203:0/2354658057 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c000ee60 con 0x7f78a80038b0 2021-01-06T22:03:45.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c759e700 1 -- 172.21.15.203:0/2354658057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a80038b0 msgr2=0x7f78a8003cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:45.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c759e700 1 --2- 172.21.15.203:0/2354658057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a80038b0 0x7f78a8003cb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f78c000bbc0 tx=0x7f78c0008040).stop 2021-01-06T22:03:45.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c6d9d700 1 -- 172.21.15.203:0/2354658057 reap_dead start 2021-01-06T22:03:45.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c759e700 1 -- 172.21.15.203:0/2354658057 shutdown_connections 2021-01-06T22:03:45.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c759e700 1 -- 172.21.15.203:0/2354658057 >> 172.21.15.203:0/2354658057 conn(0x7f78c8067e20 msgr2=0x7f78c8068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:45.878 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c759e700 1 -- 172.21.15.203:0/2354658057 shutdown_connections 2021-01-06T22:03:45.878 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.867+0000 7f78c759e700 1 -- 172.21.15.203:0/2354658057 wait complete. 2021-01-06T22:03:45.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c759e700 1 Processor -- start 2021-01-06T22:03:45.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c759e700 1 -- start start 2021-01-06T22:03:45.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c759e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a80038b0 0x7f78c8146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:45.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c759e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78c8159c00 0x7f78c815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:45.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c759e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78c80acf20 0x7f78c814d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:45.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c759e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c81473a0 con 0x7f78c8159c00 2021-01-06T22:03:45.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c759e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c81488a0 con 0x7f78a80038b0 2021-01-06T22:03:45.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c759e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c8148b40 con 0x7f78c80acf20 2021-01-06T22:03:45.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a80038b0 0x7f78c8146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:45.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a80038b0 0x7f78c8146eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48690/0 (socket says 172.21.15.203:48690) 2021-01-06T22:03:45.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c659c700 1 -- 172.21.15.203:0/2807631439 learned_addr learned my addr 172.21.15.203:0/2807631439 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:45.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c6d9d700 1 --2- 172.21.15.203:0/2807631439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78c80acf20 0x7f78c814d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:45.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c5d9b700 1 --2- 172.21.15.203:0/2807631439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78c8159c00 0x7f78c815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:45.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c659c700 1 -- 172.21.15.203:0/2807631439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78c80acf20 msgr2=0x7f78c814d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:45.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c659c700 1 --2- 172.21.15.203:0/2807631439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78c80acf20 0x7f78c814d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c659c700 1 -- 172.21.15.203:0/2807631439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78c8159c00 msgr2=0x7f78c815a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:45.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c659c700 1 --2- 172.21.15.203:0/2807631439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78c8159c00 0x7f78c815a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:45.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c659c700 1 -- 172.21.15.203:0/2807631439 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c8158f80 con 0x7f78a80038b0 2021-01-06T22:03:45.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c6d9d700 1 --2- 172.21.15.203:0/2807631439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78c80acf20 0x7f78c814d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:45.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c659c700 1 --2- 172.21.15.203:0/2807631439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a80038b0 0x7f78c8146eb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f78c0006cf0 tx=0x7f78c001a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:45.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78be7fc700 1 -- 172.21.15.203:0/2807631439 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c0025440 con 0x7f78a80038b0 2021-01-06T22:03:45.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78be7fc700 1 -- 172.21.15.203:0/2807631439 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c00255a0 con 0x7f78a80038b0 2021-01-06T22:03:45.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c759e700 1 -- 172.21.15.203:0/2807631439 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c8159220 con 0x7f78a80038b0 2021-01-06T22:03:45.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78be7fc700 1 -- 172.21.15.203:0/2807631439 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c0025940 con 0x7f78a80038b0 2021-01-06T22:03:45.886 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78c759e700 1 -- 172.21.15.203:0/2807631439 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c8159760 con 0x7f78a80038b0 2021-01-06T22:03:45.886 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.871+0000 7f78be7fc700 1 -- 172.21.15.203:0/2807631439 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f78c0025d70 con 0x7f78a80038b0 2021-01-06T22:03:45.886 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.875+0000 7f78c759e700 1 -- 172.21.15.203:0/2807631439 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c80450a0 con 0x7f78a80038b0 2021-01-06T22:03:45.886 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.875+0000 7f78be7fc700 1 --2- 172.21.15.203:0/2807631439 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f78ac056400 0x7f78ac0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:45.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.875+0000 7f78be7fc700 1 -- 172.21.15.203:0/2807631439 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f78c00713b0 con 0x7f78a80038b0 2021-01-06T22:03:45.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.875+0000 7f78c5d9b700 1 --2- 172.21.15.203:0/2807631439 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f78ac056400 0x7f78ac0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:45.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.875+0000 7f78c5d9b700 1 --2- 172.21.15.203:0/2807631439 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f78ac056400 0x7f78ac0588c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f78c8065cb0 tx=0x7f78b0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:45.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:45.879+0000 7f78be7fc700 1 -- 172.21.15.203:0/2807631439 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f78c0046ce0 con 0x7f78a80038b0 2021-01-06T22:03:46.033 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.031+0000 7f78c759e700 1 -- 172.21.15.203:0/2807631439 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78c815a8c0 con 0x7f78a80038b0 2021-01-06T22:03:46.034 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.031+0000 7f78be7fc700 1 -- 172.21.15.203:0/2807631439 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f78c0016020 con 0x7f78a80038b0 2021-01-06T22:03:46.034 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:03:46.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.035+0000 7f78c759e700 1 -- 172.21.15.203:0/2807631439 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f78ac056400 msgr2=0x7f78ac0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.035+0000 7f78c759e700 1 --2- 172.21.15.203:0/2807631439 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f78ac056400 0x7f78ac0588c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f78c8065cb0 tx=0x7f78b0009040).stop 2021-01-06T22:03:46.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.035+0000 7f78c759e700 1 -- 172.21.15.203:0/2807631439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a80038b0 msgr2=0x7f78c8146eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.035+0000 7f78c759e700 1 --2- 172.21.15.203:0/2807631439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a80038b0 0x7f78c8146eb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f78c0006cf0 tx=0x7f78c001a000).stop 2021-01-06T22:03:46.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.035+0000 7f78c759e700 1 -- 172.21.15.203:0/2807631439 shutdown_connections 2021-01-06T22:03:46.038 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.035+0000 7f78c759e700 1 --2- 172.21.15.203:0/2807631439 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f78ac056400 0x7f78ac0588c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.038 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.035+0000 7f78c759e700 1 --2- 172.21.15.203:0/2807631439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78c80acf20 0x7f78c814d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.038 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.035+0000 7f78c759e700 1 --2- 172.21.15.203:0/2807631439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a80038b0 0x7f78c8146eb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.038 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.035+0000 7f78c759e700 1 --2- 172.21.15.203:0/2807631439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78c8159c00 0x7f78c815a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.038 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.035+0000 7f78c759e700 1 -- 172.21.15.203:0/2807631439 >> 172.21.15.203:0/2807631439 conn(0x7f78c8067e20 msgr2=0x7f78c80652e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:46.039 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.035+0000 7f78c759e700 1 -- 172.21.15.203:0/2807631439 shutdown_connections 2021-01-06T22:03:46.039 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.035+0000 7f78c759e700 1 -- 172.21.15.203:0/2807631439 wait complete. 2021-01-06T22:03:46.049 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-01-06T22:03:46.176 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72ba821700 1 Processor -- start 2021-01-06T22:03:46.290 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72ba821700 1 -- start start 2021-01-06T22:03:46.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72ba821700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72b4063010 0x7f72b414e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72ba821700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f72b40633f0 0x7f72b4064b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72ba821700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b40be990 con 0x7f72b40be1b0 2021-01-06T22:03:46.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72ba821700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b414ec60 con 0x7f72b40633f0 2021-01-06T22:03:46.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72ba821700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b40be6f0 con 0x7f72b4063010 2021-01-06T22:03:46.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72ba020700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f72b40be1b0 0x7f72b4147790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47296/0 (socket says 172.21.15.203:47296) 2021-01-06T22:03:46.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72ba020700 1 -- 172.21.15.203:0/3039185547 learned_addr learned my addr 172.21.15.203:0/3039185547 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:46.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72b981f700 1 --2- 172.21.15.203:0/3039185547 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72b4063010 0x7f72b414e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.304 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72b901e700 1 --2- 172.21.15.203:0/3039185547 >> v2:172.21.15.91:3300/0 conn(0x7f72b40633f0 0x7f72b4064b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.304 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72b881d700 1 -- 172.21.15.203:0/3039185547 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951636320 0 0) 0x7f72b40be990 con 0x7f72b40be1b0 2021-01-06T22:03:46.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72b881d700 1 -- 172.21.15.203:0/3039185547 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72a0003230 con 0x7f72b40be1b0 2021-01-06T22:03:46.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72b981f700 1 -- 172.21.15.203:0/3039185547 >> v2:172.21.15.91:3300/0 conn(0x7f72b40633f0 msgr2=0x7f72b4064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:46.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72b981f700 1 --2- 172.21.15.203:0/3039185547 >> v2:172.21.15.91:3300/0 conn(0x7f72b40633f0 0x7f72b4064b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72b981f700 1 -- 172.21.15.203:0/3039185547 >> v1:172.21.15.203:6789/0 conn(0x7f72b40be1b0 legacy=0x7f72b4147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72b981f700 1 -- 172.21.15.203:0/3039185547 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b40c1050 con 0x7f72b4063010 2021-01-06T22:03:46.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72b981f700 1 --2- 172.21.15.203:0/3039185547 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72b4063010 0x7f72b414e770 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f72b0009c90 tx=0x7f72b00091d0).ready entity=mon.2 client_cookie=325e6f176225da42 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:46.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72b881d700 1 -- 172.21.15.203:0/3039185547 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b000b9c0 con 0x7f72b4063010 2021-01-06T22:03:46.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72b881d700 1 -- 172.21.15.203:0/3039185547 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b000bb20 con 0x7f72b4063010 2021-01-06T22:03:46.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.175+0000 7f72b881d700 1 -- 172.21.15.203:0/3039185547 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b000bea0 con 0x7f72b4063010 2021-01-06T22:03:46.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba821700 1 -- 172.21.15.203:0/3039185547 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72b4063010 msgr2=0x7f72b414e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba821700 1 --2- 172.21.15.203:0/3039185547 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72b4063010 0x7f72b414e770 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f72b0009c90 tx=0x7f72b00091d0).stop 2021-01-06T22:03:46.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba821700 1 -- 172.21.15.203:0/3039185547 shutdown_connections 2021-01-06T22:03:46.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba821700 1 --2- 172.21.15.203:0/3039185547 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72b4063010 0x7f72b414e770 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba821700 1 --2- 172.21.15.203:0/3039185547 >> v2:172.21.15.91:3300/0 conn(0x7f72b40633f0 0x7f72b4064b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba821700 1 -- 172.21.15.203:0/3039185547 >> 172.21.15.203:0/3039185547 conn(0x7f72b4067e20 msgr2=0x7f72b4068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:46.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba821700 1 -- 172.21.15.203:0/3039185547 shutdown_connections 2021-01-06T22:03:46.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba821700 1 -- 172.21.15.203:0/3039185547 wait complete. 2021-01-06T22:03:46.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba821700 1 Processor -- start 2021-01-06T22:03:46.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba821700 1 -- start start 2021-01-06T22:03:46.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba821700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72b4063010 0x7f72b414b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba821700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72b40633f0 0x7f72b4148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba821700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72b40be1b0 0x7f72b414a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba821700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b4148670 con 0x7f72b40be1b0 2021-01-06T22:03:46.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba821700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b414d420 con 0x7f72b4063010 2021-01-06T22:03:46.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba821700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b414ba70 con 0x7f72b40633f0 2021-01-06T22:03:46.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72b981f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72b4063010 0x7f72b414b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72b981f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72b4063010 0x7f72b414b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48704/0 (socket says 172.21.15.203:48704) 2021-01-06T22:03:46.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72b981f700 1 -- 172.21.15.203:0/998527348 learned_addr learned my addr 172.21.15.203:0/998527348 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:46.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72b901e700 1 --2- 172.21.15.203:0/998527348 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72b40633f0 0x7f72b4148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72ba020700 1 --2- 172.21.15.203:0/998527348 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72b40be1b0 0x7f72b414a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72b981f700 1 -- 172.21.15.203:0/998527348 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72b40633f0 msgr2=0x7f72b4148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72b981f700 1 --2- 172.21.15.203:0/998527348 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72b40633f0 0x7f72b4148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72b981f700 1 -- 172.21.15.203:0/998527348 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72b40be1b0 msgr2=0x7f72b414a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.179+0000 7f72b981f700 1 --2- 172.21.15.203:0/998527348 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72b40be1b0 0x7f72b414a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.183+0000 7f72b981f700 1 -- 172.21.15.203:0/998527348 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b414bd10 con 0x7f72b4063010 2021-01-06T22:03:46.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.183+0000 7f72ba020700 1 --2- 172.21.15.203:0/998527348 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72b40be1b0 0x7f72b414a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:03:46.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.183+0000 7f72b981f700 1 --2- 172.21.15.203:0/998527348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72b4063010 0x7f72b414b2e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f72b0009420 tx=0x7f72b0019000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:46.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.183+0000 7f72a9ffb700 1 -- 172.21.15.203:0/998527348 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0010e90 con 0x7f72b4063010 2021-01-06T22:03:46.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.183+0000 7f72a9ffb700 1 -- 172.21.15.203:0/998527348 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b0019660 con 0x7f72b4063010 2021-01-06T22:03:46.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.183+0000 7f72a9ffb700 1 -- 172.21.15.203:0/998527348 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0019a00 con 0x7f72b4063010 2021-01-06T22:03:46.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.183+0000 7f72ba821700 1 -- 172.21.15.203:0/998527348 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b414d680 con 0x7f72b4063010 2021-01-06T22:03:46.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.183+0000 7f72ba821700 1 -- 172.21.15.203:0/998527348 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b40652e0 con 0x7f72b4063010 2021-01-06T22:03:46.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.183+0000 7f72ba821700 1 -- 172.21.15.203:0/998527348 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b40450a0 con 0x7f72b4063010 2021-01-06T22:03:46.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.183+0000 7f72a9ffb700 1 -- 172.21.15.203:0/998527348 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f72b0016030 con 0x7f72b4063010 2021-01-06T22:03:46.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.183+0000 7f72a9ffb700 1 --2- 172.21.15.203:0/998527348 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72940563d0 0x7f7294058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.183+0000 7f72a9ffb700 1 -- 172.21.15.203:0/998527348 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f72b0070b20 con 0x7f72b4063010 2021-01-06T22:03:46.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.183+0000 7f72b901e700 1 --2- 172.21.15.203:0/998527348 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72940563d0 0x7f7294058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.187+0000 7f72b901e700 1 --2- 172.21.15.203:0/998527348 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72940563d0 0x7f7294058890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f72b414b000 tx=0x7f72ac008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:46.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.191+0000 7f72a9ffb700 1 -- 172.21.15.203:0/998527348 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f72b00463a0 con 0x7f72b4063010 2021-01-06T22:03:46.345 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.343+0000 7f72ba821700 1 -- 172.21.15.203:0/998527348 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f72b4061e70 con 0x7f72b4063010 2021-01-06T22:03:46.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.343+0000 7f72a9ffb700 1 -- 172.21.15.203:0/998527348 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f72b00499c0 con 0x7f72b4063010 2021-01-06T22:03:46.347 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:03:46.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.347+0000 7f72ba821700 1 -- 172.21.15.203:0/998527348 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72940563d0 msgr2=0x7f7294058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.350 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.347+0000 7f72ba821700 1 --2- 172.21.15.203:0/998527348 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72940563d0 0x7f7294058890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f72b414b000 tx=0x7f72ac008040).stop 2021-01-06T22:03:46.350 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.347+0000 7f72ba821700 1 -- 172.21.15.203:0/998527348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72b4063010 msgr2=0x7f72b414b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.350 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.347+0000 7f72ba821700 1 --2- 172.21.15.203:0/998527348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72b4063010 0x7f72b414b2e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f72b0009420 tx=0x7f72b0019000).stop 2021-01-06T22:03:46.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.347+0000 7f72ba821700 1 -- 172.21.15.203:0/998527348 shutdown_connections 2021-01-06T22:03:46.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.347+0000 7f72ba821700 1 --2- 172.21.15.203:0/998527348 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72940563d0 0x7f7294058890 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.347+0000 7f72ba821700 1 --2- 172.21.15.203:0/998527348 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72b40be1b0 0x7f72b414a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.347+0000 7f72ba821700 1 --2- 172.21.15.203:0/998527348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72b4063010 0x7f72b414b2e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.347+0000 7f72ba821700 1 --2- 172.21.15.203:0/998527348 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72b40633f0 0x7f72b4148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.347+0000 7f72ba821700 1 -- 172.21.15.203:0/998527348 >> 172.21.15.203:0/998527348 conn(0x7f72b4067e20 msgr2=0x7f72b40b8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:46.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.347+0000 7f72ba821700 1 -- 172.21.15.203:0/998527348 shutdown_connections 2021-01-06T22:03:46.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.347+0000 7f72ba821700 1 -- 172.21.15.203:0/998527348 wait complete. 2021-01-06T22:03:46.364 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2021-01-06T22:03:46.365 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2021-01-06T22:03:46.366 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2021-01-06T22:03:46.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.487+0000 7f85c0819700 1 Processor -- start 2021-01-06T22:03:46.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.487+0000 7f85c0819700 1 -- start start 2021-01-06T22:03:46.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.487+0000 7f85c0819700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f85bc0be350 0x7f85bc065690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.487+0000 7f85c0819700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f85bc0be730 0x7f85bc152510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.487+0000 7f85c0819700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85bc0bba40 con 0x7f85bc152a00 2021-01-06T22:03:46.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.487+0000 7f85c0819700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc1563d0 con 0x7f85bc0be350 2021-01-06T22:03:46.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.487+0000 7f85c0819700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc0bbca0 con 0x7f85bc0be730 2021-01-06T22:03:46.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.487+0000 7f85b2ffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f85bc0be730 0x7f85bc152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.487+0000 7f85b2ffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f85bc0be730 0x7f85bc152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48712/0 (socket says 172.21.15.203:48712) 2021-01-06T22:03:46.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.487+0000 7f85b2ffd700 1 -- 172.21.15.203:0/3614608345 learned_addr learned my addr 172.21.15.203:0/3614608345 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:46.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85bb7fe700 1 --2- 172.21.15.203:0/3614608345 >> v2:172.21.15.91:3300/0 conn(0x7f85bc0be350 0x7f85bc065690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85baffd700 1 -- 172.21.15.203:0/3614608345 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390639557 0 0) 0x7f85bc0bba40 con 0x7f85bc152a00 2021-01-06T22:03:46.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85baffd700 1 -- 172.21.15.203:0/3614608345 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f859c003230 con 0x7f85bc152a00 2021-01-06T22:03:46.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85b2ffd700 1 -- 172.21.15.203:0/3614608345 >> v2:172.21.15.91:3300/0 conn(0x7f85bc0be350 msgr2=0x7f85bc065690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:46.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85b2ffd700 1 --2- 172.21.15.203:0/3614608345 >> v2:172.21.15.91:3300/0 conn(0x7f85bc0be350 0x7f85bc065690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85b2ffd700 1 -- 172.21.15.203:0/3614608345 >> v1:172.21.15.203:6789/0 conn(0x7f85bc152a00 legacy=0x7f85bc062e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85b2ffd700 1 -- 172.21.15.203:0/3614608345 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc0bbf40 con 0x7f85bc0be730 2021-01-06T22:03:46.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85bb7fe700 1 --2- 172.21.15.203:0/3614608345 >> v2:172.21.15.91:3300/0 conn(0x7f85bc0be350 0x7f85bc065690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:46.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85b2ffd700 1 --2- 172.21.15.203:0/3614608345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85bc0be730 0x7f85bc152510 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f85bc062500 tx=0x7f85a4006ea0).ready entity=mon.1 client_cookie=2d0d62255f1840da server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:46.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85baffd700 1 -- 172.21.15.203:0/3614608345 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a40098a0 con 0x7f85bc0be730 2021-01-06T22:03:46.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85baffd700 1 -- 172.21.15.203:0/3614608345 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a4009a00 con 0x7f85bc0be730 2021-01-06T22:03:46.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85baffd700 1 -- 172.21.15.203:0/3614608345 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4009d80 con 0x7f85bc0be730 2021-01-06T22:03:46.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85c0819700 1 -- 172.21.15.203:0/3614608345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85bc0be730 msgr2=0x7f85bc152510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85c0819700 1 --2- 172.21.15.203:0/3614608345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85bc0be730 0x7f85bc152510 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f85bc062500 tx=0x7f85a4006ea0).stop 2021-01-06T22:03:46.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85c0819700 1 -- 172.21.15.203:0/3614608345 shutdown_connections 2021-01-06T22:03:46.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85c0819700 1 --2- 172.21.15.203:0/3614608345 >> v2:172.21.15.91:3300/0 conn(0x7f85bc0be350 0x7f85bc065690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85c0819700 1 --2- 172.21.15.203:0/3614608345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85bc0be730 0x7f85bc152510 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85c0819700 1 -- 172.21.15.203:0/3614608345 >> 172.21.15.203:0/3614608345 conn(0x7f85bc067e20 msgr2=0x7f85bc068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:46.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85c0819700 1 -- 172.21.15.203:0/3614608345 shutdown_connections 2021-01-06T22:03:46.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85c0819700 1 -- 172.21.15.203:0/3614608345 wait complete. 2021-01-06T22:03:46.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85c0819700 1 Processor -- start 2021-01-06T22:03:46.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85c0819700 1 -- start start 2021-01-06T22:03:46.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85c0819700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f85bc0be350 0x7f85bc0bf590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85c0819700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85bc0be730 0x7f85bc161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85c0819700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85bc152a00 0x7f85bc165460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85c0819700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc1576e0 con 0x7f85bc0be350 2021-01-06T22:03:46.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85c0819700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc157980 con 0x7f85bc152a00 2021-01-06T22:03:46.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85c0819700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc157c20 con 0x7f85bc0be730 2021-01-06T22:03:46.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85bbfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85bc152a00 0x7f85bc165460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85bbfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85bc152a00 0x7f85bc165460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48722/0 (socket says 172.21.15.203:48722) 2021-01-06T22:03:46.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85bbfff700 1 -- 172.21.15.203:0/188416444 learned_addr learned my addr 172.21.15.203:0/188416444 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:46.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85bb7fe700 1 --2- 172.21.15.203:0/188416444 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f85bc0be350 0x7f85bc0bf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.491+0000 7f85b2ffd700 1 --2- 172.21.15.203:0/188416444 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85bc0be730 0x7f85bc161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85bbfff700 1 -- 172.21.15.203:0/188416444 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85bc0be730 msgr2=0x7f85bc161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85bbfff700 1 --2- 172.21.15.203:0/188416444 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85bc0be730 0x7f85bc161eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85bbfff700 1 -- 172.21.15.203:0/188416444 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f85bc0be350 msgr2=0x7f85bc0bf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85bbfff700 1 --2- 172.21.15.203:0/188416444 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f85bc0be350 0x7f85bc0bf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85bbfff700 1 -- 172.21.15.203:0/188416444 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc164a80 con 0x7f85bc152a00 2021-01-06T22:03:46.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85bbfff700 1 --2- 172.21.15.203:0/188416444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85bc152a00 0x7f85bc165460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f85b400ca90 tx=0x7f85b4007d80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:46.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85b3fff700 1 -- 172.21.15.203:0/188416444 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b400ce60 con 0x7f85bc152a00 2021-01-06T22:03:46.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85b3fff700 1 -- 172.21.15.203:0/188416444 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b4002240 con 0x7f85bc152a00 2021-01-06T22:03:46.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85c0819700 1 -- 172.21.15.203:0/188416444 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85bc164d20 con 0x7f85bc152a00 2021-01-06T22:03:46.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85b3fff700 1 -- 172.21.15.203:0/188416444 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4001550 con 0x7f85bc152a00 2021-01-06T22:03:46.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85c0819700 1 -- 172.21.15.203:0/188416444 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85bc063290 con 0x7f85bc152a00 2021-01-06T22:03:46.508 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85b3fff700 1 -- 172.21.15.203:0/188416444 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f85b4019030 con 0x7f85bc152a00 2021-01-06T22:03:46.508 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85c0819700 1 -- 172.21.15.203:0/188416444 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85bc0450a0 con 0x7f85bc152a00 2021-01-06T22:03:46.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85b3fff700 1 --2- 172.21.15.203:0/188416444 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f85a00563d0 0x7f85a0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85b3fff700 1 -- 172.21.15.203:0/188416444 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f85b406ea40 con 0x7f85bc152a00 2021-01-06T22:03:46.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85bb7fe700 1 --2- 172.21.15.203:0/188416444 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f85a00563d0 0x7f85a0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.495+0000 7f85bb7fe700 1 --2- 172.21.15.203:0/188416444 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f85a00563d0 0x7f85a0058890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f85ac00bb10 tx=0x7f85ac008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:46.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.503+0000 7f85b3fff700 1 -- 172.21.15.203:0/188416444 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f85b4044370 con 0x7f85bc152a00 2021-01-06T22:03:46.690 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.687+0000 7f85c0819700 1 -- 172.21.15.203:0/188416444 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f85bc061e70 con 0x7f85bc152a00 2021-01-06T22:03:46.787 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.695+0000 7f85b3fff700 1 -- 172.21.15.203:0/188416444 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+134526 (secure 0 0 0) 0x7f85b4041d90 con 0x7f85bc152a00 2021-01-06T22:03:46.788 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.699+0000 7f85c0819700 1 -- 172.21.15.203:0/188416444 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f85a00563d0 msgr2=0x7f85a0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.788 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.699+0000 7f85c0819700 1 --2- 172.21.15.203:0/188416444 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f85a00563d0 0x7f85a0058890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f85ac00bb10 tx=0x7f85ac008040).stop 2021-01-06T22:03:46.788 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.699+0000 7f85c0819700 1 -- 172.21.15.203:0/188416444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85bc152a00 msgr2=0x7f85bc165460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.788 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.699+0000 7f85c0819700 1 --2- 172.21.15.203:0/188416444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85bc152a00 0x7f85bc165460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f85b400ca90 tx=0x7f85b4007d80).stop 2021-01-06T22:03:46.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.699+0000 7f85c0819700 1 -- 172.21.15.203:0/188416444 shutdown_connections 2021-01-06T22:03:46.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.699+0000 7f85c0819700 1 --2- 172.21.15.203:0/188416444 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f85a00563d0 0x7f85a0058890 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.699+0000 7f85c0819700 1 --2- 172.21.15.203:0/188416444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85bc152a00 0x7f85bc165460 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.699+0000 7f85c0819700 1 --2- 172.21.15.203:0/188416444 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f85bc0be350 0x7f85bc0bf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.699+0000 7f85c0819700 1 --2- 172.21.15.203:0/188416444 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85bc0be730 0x7f85bc161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.699+0000 7f85c0819700 1 -- 172.21.15.203:0/188416444 >> 172.21.15.203:0/188416444 conn(0x7f85bc067e20 msgr2=0x7f85bc0656d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:46.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.699+0000 7f85c0819700 1 -- 172.21.15.203:0/188416444 shutdown_connections 2021-01-06T22:03:46.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.699+0000 7f85c0819700 1 -- 172.21.15.203:0/188416444 wait complete. 2021-01-06T22:03:46.791 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:46.791 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":5,"active_gid":4102,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":18146},{"type":"v1","addr":"172.21.15.203:6801","nonce":18146}]},"active_addr":"172.21.15.203:6801/18146","active_change":"2021-01-06T22:03:25.082596+0000","active_mgr_features":4540138297136906239,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540138297136906239,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":300423415}]},{"addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":2727300991}]},{"addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":3119442073}]}]}} 2021-01-06T22:03:46.795 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2021-01-06T22:03:46.796 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2021-01-06T22:03:46.796 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-06T22:03:46.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.923+0000 7f265b59e700 1 Processor -- start 2021-01-06T22:03:46.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.923+0000 7f265b59e700 1 -- start start 2021-01-06T22:03:46.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265b59e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f265c0bb3f0 0x7f265c0ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265b59e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f265c0bb7d0 0x7f265c14eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265b59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f265c0646b0 con 0x7f265c0a5e50 2021-01-06T22:03:46.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265b59e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265c0670c0 con 0x7f265c0bb3f0 2021-01-06T22:03:46.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265b59e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265c064450 con 0x7f265c0bb7d0 2021-01-06T22:03:46.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265ad9d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f265c0bb7d0 0x7f265c14eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265ad9d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f265c0bb7d0 0x7f265c14eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48728/0 (socket says 172.21.15.203:48728) 2021-01-06T22:03:46.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265ad9d700 1 -- 172.21.15.203:0/1105827556 learned_addr learned my addr 172.21.15.203:0/1105827556 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:46.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265a59c700 1 --2- 172.21.15.203:0/1105827556 >> v2:172.21.15.91:3300/0 conn(0x7f265c0bb3f0 0x7f265c0ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265ad9d700 1 -- 172.21.15.203:0/1105827556 >> v2:172.21.15.91:3300/0 conn(0x7f265c0bb3f0 msgr2=0x7f265c0ba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:46.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265ad9d700 1 --2- 172.21.15.203:0/1105827556 >> v2:172.21.15.91:3300/0 conn(0x7f265c0bb3f0 0x7f265c0ba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265ad9d700 1 -- 172.21.15.203:0/1105827556 >> v1:172.21.15.91:6789/0 conn(0x7f265c0a5e50 legacy=0x7f265c0669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265ad9d700 1 -- 172.21.15.203:0/1105827556 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c152cc0 con 0x7f265c0bb7d0 2021-01-06T22:03:46.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265ad9d700 1 --2- 172.21.15.203:0/1105827556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f265c0bb7d0 0x7f265c14eef0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f265c0bb130 tx=0x7f2654009290).ready entity=mon.1 client_cookie=5640047930ccd6b1 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:46.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265959a700 1 -- 172.21.15.203:0/1105827556 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265400bbc0 con 0x7f265c0bb7d0 2021-01-06T22:03:46.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265959a700 1 -- 172.21.15.203:0/1105827556 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265400bd20 con 0x7f265c0bb7d0 2021-01-06T22:03:46.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265959a700 1 -- 172.21.15.203:0/1105827556 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654011610 con 0x7f265c0bb7d0 2021-01-06T22:03:46.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265b59e700 1 -- 172.21.15.203:0/1105827556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f265c0bb7d0 msgr2=0x7f265c14eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265b59e700 1 --2- 172.21.15.203:0/1105827556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f265c0bb7d0 0x7f265c14eef0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f265c0bb130 tx=0x7f2654009290).stop 2021-01-06T22:03:46.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265b59e700 1 -- 172.21.15.203:0/1105827556 shutdown_connections 2021-01-06T22:03:46.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265b59e700 1 --2- 172.21.15.203:0/1105827556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f265c0bb7d0 0x7f265c14eef0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265b59e700 1 --2- 172.21.15.203:0/1105827556 >> v2:172.21.15.91:3300/0 conn(0x7f265c0bb3f0 0x7f265c0ba9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265b59e700 1 -- 172.21.15.203:0/1105827556 >> 172.21.15.203:0/1105827556 conn(0x7f265c061530 msgr2=0x7f265c063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:46.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265b59e700 1 -- 172.21.15.203:0/1105827556 shutdown_connections 2021-01-06T22:03:46.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265b59e700 1 -- 172.21.15.203:0/1105827556 wait complete. 2021-01-06T22:03:46.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.927+0000 7f265b59e700 1 Processor -- start 2021-01-06T22:03:46.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265b59e700 1 -- start start 2021-01-06T22:03:46.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265b59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f265c0a5e50 0x7f265c162390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265b59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f265c0bb3f0 0x7f265c158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265b59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f265c0bb7d0 0x7f265c1651d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265b59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c0b97c0 con 0x7f265c0bb7d0 2021-01-06T22:03:46.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265b59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c0ba470 con 0x7f265c0a5e50 2021-01-06T22:03:46.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265b59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265c0b99f0 con 0x7f265c0bb3f0 2021-01-06T22:03:46.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265a59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f265c0a5e50 0x7f265c162390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265a59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f265c0a5e50 0x7f265c162390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48738/0 (socket says 172.21.15.203:48738) 2021-01-06T22:03:46.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265a59c700 1 -- 172.21.15.203:0/2073394483 learned_addr learned my addr 172.21.15.203:0/2073394483 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:46.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f2659d9b700 1 --2- 172.21.15.203:0/2073394483 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f265c0bb3f0 0x7f265c158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265ad9d700 1 --2- 172.21.15.203:0/2073394483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f265c0bb7d0 0x7f265c1651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265a59c700 1 -- 172.21.15.203:0/2073394483 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f265c0bb3f0 msgr2=0x7f265c158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265a59c700 1 --2- 172.21.15.203:0/2073394483 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f265c0bb3f0 0x7f265c158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265a59c700 1 -- 172.21.15.203:0/2073394483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f265c0bb7d0 msgr2=0x7f265c1651d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:46.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265a59c700 1 --2- 172.21.15.203:0/2073394483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f265c0bb7d0 0x7f265c1651d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:46.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265a59c700 1 -- 172.21.15.203:0/2073394483 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c0b9c90 con 0x7f265c0a5e50 2021-01-06T22:03:46.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265a59c700 1 --2- 172.21.15.203:0/2073394483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f265c0a5e50 0x7f265c162390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f264c00db60 tx=0x7f264c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:46.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f26527fc700 1 -- 172.21.15.203:0/2073394483 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c00be10 con 0x7f265c0a5e50 2021-01-06T22:03:46.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f26527fc700 1 -- 172.21.15.203:0/2073394483 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264c009790 con 0x7f265c0a5e50 2021-01-06T22:03:46.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265b59e700 1 -- 172.21.15.203:0/2073394483 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265c0b9f30 con 0x7f265c0a5e50 2021-01-06T22:03:46.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f26527fc700 1 -- 172.21.15.203:0/2073394483 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c009b10 con 0x7f265c0a5e50 2021-01-06T22:03:46.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265b59e700 1 -- 172.21.15.203:0/2073394483 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265c09e4c0 con 0x7f265c0a5e50 2021-01-06T22:03:46.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f265b59e700 1 -- 172.21.15.203:0/2073394483 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265c0450a0 con 0x7f265c0a5e50 2021-01-06T22:03:46.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.931+0000 7f26527fc700 1 -- 172.21.15.203:0/2073394483 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f264c013480 con 0x7f265c0a5e50 2021-01-06T22:03:46.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.935+0000 7f26527fc700 1 --2- 172.21.15.203:0/2073394483 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f26400563d0 0x7f2640058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:46.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.935+0000 7f2659d9b700 1 --2- 172.21.15.203:0/2073394483 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f26400563d0 0x7f2640058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:46.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.935+0000 7f26527fc700 1 -- 172.21.15.203:0/2073394483 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f264c06f7a0 con 0x7f265c0a5e50 2021-01-06T22:03:46.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.935+0000 7f2659d9b700 1 --2- 172.21.15.203:0/2073394483 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f26400563d0 0x7f2640058890 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f265c151b30 tx=0x7f2644007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:46.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:46.939+0000 7f26527fc700 1 -- 172.21.15.203:0/2073394483 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f264c044fc0 con 0x7f265c0a5e50 2021-01-06T22:03:47.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.095+0000 7f265b59e700 1 -- 172.21.15.203:0/2073394483 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f265c164ec0 con 0x7f265c0a5e50 2021-01-06T22:03:47.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.095+0000 7f26527fc700 1 -- 172.21.15.203:0/2073394483 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f264c0485e0 con 0x7f265c0a5e50 2021-01-06T22:03:47.097 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:47.097 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":23,"fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","created":"2021-01-06T22:03:21.322671+0000","modified":"2021-01-06T22:03:40.744826+0000","last_up_change":"2021-01-06T22:03:31.115611+0000","last_in_change":"2021-01-06T22:03:31.115611+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-06T22:03:25.161465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-06T22:03:35.070364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-01-06T22:03:37.862354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-01-06T22:03:38.738416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e0b35c09-147c-4ccc-b566-c48b81e25412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":18347},{"type":"v1","addr":"172.21.15.203:6815","nonce":18347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":18347},{"type":"v1","addr":"172.21.15.203:6818","nonce":18347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":18347},{"type":"v1","addr":"172.21.15.203:6831","nonce":18347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6821","nonce":18347},{"type":"v1","addr":"172.21.15.203:6827","nonce":18347}]},"public_addr":"172.21.15.203:6815/18347","cluster_addr":"172.21.15.203:6818/18347","heartbeat_back_addr":"172.21.15.203:6831/18347","heartbeat_front_addr":"172.21.15.203:6827/18347","state":["exists","up"]},{"osd":1,"uuid":"8add59a0-2d58-4c1b-a8d1-62049ee9d85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":18337},{"type":"v1","addr":"172.21.15.203:6803","nonce":18337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":18337},{"type":"v1","addr":"172.21.15.203:6805","nonce":18337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":18337},{"type":"v1","addr":"172.21.15.203:6809","nonce":18337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":18337},{"type":"v1","addr":"172.21.15.203:6807","nonce":18337}]},"public_addr":"172.21.15.203:6803/18337","cluster_addr":"172.21.15.203:6805/18337","heartbeat_back_addr":"172.21.15.203:6809/18337","heartbeat_front_addr":"172.21.15.203:6807/18337","state":["exists","up"]},{"osd":2,"uuid":"b294a411-c7c9-477e-8eae-3b7bd0563ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":18354},{"type":"v1","addr":"172.21.15.203:6811","nonce":18354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":18354},{"type":"v1","addr":"172.21.15.203:6819","nonce":18354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6825","nonce":18354},{"type":"v1","addr":"172.21.15.203:6828","nonce":18354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6823","nonce":18354},{"type":"v1","addr":"172.21.15.203:6824","nonce":18354}]},"public_addr":"172.21.15.203:6811/18354","cluster_addr":"172.21.15.203:6819/18354","heartbeat_back_addr":"172.21.15.203:6828/18354","heartbeat_front_addr":"172.21.15.203:6824/18354","state":["exists","up"]},{"osd":3,"uuid":"5878a8f7-1a42-41c3-9810-0ac41a54dd6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":18346},{"type":"v1","addr":"172.21.15.203:6817","nonce":18346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":18346},{"type":"v1","addr":"172.21.15.203:6822","nonce":18346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":18346},{"type":"v1","addr":"172.21.15.203:6833","nonce":18346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":18346},{"type":"v1","addr":"172.21.15.203:6829","nonce":18346}]},"public_addr":"172.21.15.203:6817/18346","cluster_addr":"172.21.15.203:6822/18346","heartbeat_back_addr":"172.21.15.203:6833/18346","heartbeat_front_addr":"172.21.15.203:6829/18346","state":["exists","up"]},{"osd":4,"uuid":"f641a345-81e4-4400-a44c-d2f82fef7f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":18370},{"type":"v1","addr":"172.21.15.91:6809","nonce":18370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":18370},{"type":"v1","addr":"172.21.15.91:6811","nonce":18370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":18370},{"type":"v1","addr":"172.21.15.91:6816","nonce":18370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":18370},{"type":"v1","addr":"172.21.15.91:6813","nonce":18370}]},"public_addr":"172.21.15.91:6809/18370","cluster_addr":"172.21.15.91:6811/18370","heartbeat_back_addr":"172.21.15.91:6816/18370","heartbeat_front_addr":"172.21.15.91:6813/18370","state":["exists","up"]},{"osd":5,"uuid":"c1ff150d-68c6-4e6b-abd4-76af27bb2103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":18372},{"type":"v1","addr":"172.21.15.91:6801","nonce":18372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":18372},{"type":"v1","addr":"172.21.15.91:6803","nonce":18372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":18372},{"type":"v1","addr":"172.21.15.91:6807","nonce":18372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":18372},{"type":"v1","addr":"172.21.15.91:6805","nonce":18372}]},"public_addr":"172.21.15.91:6801/18372","cluster_addr":"172.21.15.91:6803/18372","heartbeat_back_addr":"172.21.15.91:6807/18372","heartbeat_front_addr":"172.21.15.91:6805/18372","state":["exists","up"]},{"osd":6,"uuid":"35a58e1a-0a0a-4f02-a477-adc029038d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6815","nonce":18371},{"type":"v1","addr":"172.21.15.91:6817","nonce":18371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":18371},{"type":"v1","addr":"172.21.15.91:6819","nonce":18371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":18371},{"type":"v1","addr":"172.21.15.91:6823","nonce":18371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":18371},{"type":"v1","addr":"172.21.15.91:6821","nonce":18371}]},"public_addr":"172.21.15.91:6817/18371","cluster_addr":"172.21.15.91:6819/18371","heartbeat_back_addr":"172.21.15.91:6823/18371","heartbeat_front_addr":"172.21.15.91:6821/18371","state":["exists","up"]},{"osd":7,"uuid":"809a2d89-4303-482a-8558-aef907814234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":18373},{"type":"v1","addr":"172.21.15.91:6825","nonce":18373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":18373},{"type":"v1","addr":"172.21.15.91:6827","nonce":18373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":18373},{"type":"v1","addr":"172.21.15.91:6831","nonce":18373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":18373},{"type":"v1","addr":"172.21.15.91:6829","nonce":18373}]},"public_addr":"172.21.15.91:6825/18373","cluster_addr":"172.21.15.91:6827/18373","heartbeat_back_addr":"172.21.15.91:6831/18373","heartbeat_front_addr":"172.21.15.91:6829/18373","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.875052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.103604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.179225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.158512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.206073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.894840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.229708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.248192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-06T22:03:47.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.095+0000 7f265b59e700 1 -- 172.21.15.203:0/2073394483 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f26400563d0 msgr2=0x7f2640058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.095+0000 7f265b59e700 1 --2- 172.21.15.203:0/2073394483 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f26400563d0 0x7f2640058890 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f265c151b30 tx=0x7f2644007b80).stop 2021-01-06T22:03:47.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.095+0000 7f265b59e700 1 -- 172.21.15.203:0/2073394483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f265c0a5e50 msgr2=0x7f265c162390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.095+0000 7f265b59e700 1 --2- 172.21.15.203:0/2073394483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f265c0a5e50 0x7f265c162390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f264c00db60 tx=0x7f264c008040).stop 2021-01-06T22:03:47.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.095+0000 7f265b59e700 1 -- 172.21.15.203:0/2073394483 shutdown_connections 2021-01-06T22:03:47.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.095+0000 7f265b59e700 1 --2- 172.21.15.203:0/2073394483 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f26400563d0 0x7f2640058890 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.095+0000 7f265b59e700 1 --2- 172.21.15.203:0/2073394483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f265c0bb7d0 0x7f265c1651d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.095+0000 7f265b59e700 1 --2- 172.21.15.203:0/2073394483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f265c0a5e50 0x7f265c162390 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.095+0000 7f265b59e700 1 --2- 172.21.15.203:0/2073394483 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f265c0bb3f0 0x7f265c158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.095+0000 7f265b59e700 1 -- 172.21.15.203:0/2073394483 >> 172.21.15.203:0/2073394483 conn(0x7f265c061530 msgr2=0x7f265c0ba640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:47.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.095+0000 7f265b59e700 1 -- 172.21.15.203:0/2073394483 shutdown_connections 2021-01-06T22:03:47.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.095+0000 7f265b59e700 1 -- 172.21.15.203:0/2073394483 wait complete. 2021-01-06T22:03:47.111 INFO:tasks.ceph.ceph_manager.ceph:all up! 2021-01-06T22:03:47.113 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-06T22:03:47.237 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.235+0000 7f9ddefca700 1 Processor -- start 2021-01-06T22:03:47.238 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.235+0000 7f9ddefca700 1 -- start start 2021-01-06T22:03:47.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.235+0000 7f9ddefca700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9dd8064380 0x7f9dd80c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.235+0000 7f9ddefca700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9dd81478c0 0x7f9dd80bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.235+0000 7f9ddefca700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd80bb2d0 con 0x7f9dd814e310 2021-01-06T22:03:47.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.235+0000 7f9ddefca700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dd814a560 con 0x7f9dd8064380 2021-01-06T22:03:47.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.235+0000 7f9ddefca700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd814a800 con 0x7f9dd81478c0 2021-01-06T22:03:47.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.235+0000 7f9dddfc8700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9dd8064380 0x7f9dd80c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.235+0000 7f9dddfc8700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9dd8064380 0x7f9dd80c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48742/0 (socket says 172.21.15.203:48742) 2021-01-06T22:03:47.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.235+0000 7f9ddd7c7700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f9dd814e310 0x7f9dd81471b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47338/0 (socket says 172.21.15.203:47338) 2021-01-06T22:03:47.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.235+0000 7f9dddfc8700 1 -- 172.21.15.203:0/2839614069 learned_addr learned my addr 172.21.15.203:0/2839614069 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:47.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.235+0000 7f9dde7c9700 1 --2- 172.21.15.203:0/2839614069 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9dd81478c0 0x7f9dd80bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9dddfc8700 1 -- 172.21.15.203:0/2839614069 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9dd81478c0 msgr2=0x7f9dd80bb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:47.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9dddfc8700 1 --2- 172.21.15.203:0/2839614069 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9dd81478c0 0x7f9dd80bb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9dddfc8700 1 -- 172.21.15.203:0/2839614069 >> v1:172.21.15.203:6789/0 conn(0x7f9dd814e310 legacy=0x7f9dd81471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9ddcfc6700 1 -- 172.21.15.203:0/2839614069 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3143423871 0 0) 0x7f9dd80bb2d0 con 0x7f9dd814e310 2021-01-06T22:03:47.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9dddfc8700 1 -- 172.21.15.203:0/2839614069 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd814aaa0 con 0x7f9dd8064380 2021-01-06T22:03:47.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9dddfc8700 1 --2- 172.21.15.203:0/2839614069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd8064380 0x7f9dd80c0fa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f9dd4009d50 tx=0x7f9dd4009290).ready entity=mon.1 client_cookie=2d8523b849c7c45 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:47.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9ddcfc6700 1 -- 172.21.15.203:0/2839614069 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd400bd10 con 0x7f9dd8064380 2021-01-06T22:03:47.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9ddcfc6700 1 -- 172.21.15.203:0/2839614069 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd400be70 con 0x7f9dd8064380 2021-01-06T22:03:47.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9ddcfc6700 1 -- 172.21.15.203:0/2839614069 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4011610 con 0x7f9dd8064380 2021-01-06T22:03:47.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9ddefca700 1 -- 172.21.15.203:0/2839614069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd8064380 msgr2=0x7f9dd80c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9ddefca700 1 --2- 172.21.15.203:0/2839614069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd8064380 0x7f9dd80c0fa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f9dd4009d50 tx=0x7f9dd4009290).stop 2021-01-06T22:03:47.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9ddefca700 1 -- 172.21.15.203:0/2839614069 shutdown_connections 2021-01-06T22:03:47.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9ddefca700 1 --2- 172.21.15.203:0/2839614069 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9dd81478c0 0x7f9dd80bb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9ddefca700 1 --2- 172.21.15.203:0/2839614069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd8064380 0x7f9dd80c0fa0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9ddefca700 1 -- 172.21.15.203:0/2839614069 >> 172.21.15.203:0/2839614069 conn(0x7f9dd80667b0 msgr2=0x7f9dd80c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:47.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9ddefca700 1 -- 172.21.15.203:0/2839614069 shutdown_connections 2021-01-06T22:03:47.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.239+0000 7f9ddefca700 1 -- 172.21.15.203:0/2839614069 wait complete. 2021-01-06T22:03:47.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddefca700 1 Processor -- start 2021-01-06T22:03:47.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddefca700 1 -- start start 2021-01-06T22:03:47.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddefca700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9dd8064380 0x7f9dd8141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddefca700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd81478c0 0x7f9dd80b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddefca700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9dd814e310 0x7f9dd80b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddefca700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd8143790 con 0x7f9dd8064380 2021-01-06T22:03:47.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddefca700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd8060a70 con 0x7f9dd81478c0 2021-01-06T22:03:47.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddefca700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd81439c0 con 0x7f9dd814e310 2021-01-06T22:03:47.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddd7c7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd81478c0 0x7f9dd80b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddd7c7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd81478c0 0x7f9dd80b6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48752/0 (socket says 172.21.15.203:48752) 2021-01-06T22:03:47.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddd7c7700 1 -- 172.21.15.203:0/3848987344 learned_addr learned my addr 172.21.15.203:0/3848987344 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:47.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9dde7c9700 1 --2- 172.21.15.203:0/3848987344 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9dd814e310 0x7f9dd80b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9dddfc8700 1 --2- 172.21.15.203:0/3848987344 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9dd8064380 0x7f9dd8141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddd7c7700 1 -- 172.21.15.203:0/3848987344 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9dd814e310 msgr2=0x7f9dd80b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddd7c7700 1 --2- 172.21.15.203:0/3848987344 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9dd814e310 0x7f9dd80b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddd7c7700 1 -- 172.21.15.203:0/3848987344 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9dd8064380 msgr2=0x7f9dd8141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddd7c7700 1 --2- 172.21.15.203:0/3848987344 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9dd8064380 0x7f9dd8141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddd7c7700 1 -- 172.21.15.203:0/3848987344 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd8143c60 con 0x7f9dd81478c0 2021-01-06T22:03:47.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddd7c7700 1 --2- 172.21.15.203:0/3848987344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd81478c0 0x7f9dd80b6eb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f9dcc00c6e0 tx=0x7f9dcc007aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:47.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9dc9ffb700 1 -- 172.21.15.203:0/3848987344 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc0072e0 con 0x7f9dd81478c0 2021-01-06T22:03:47.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9dc9ffb700 1 -- 172.21.15.203:0/3848987344 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dcc005520 con 0x7f9dd81478c0 2021-01-06T22:03:47.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddefca700 1 -- 172.21.15.203:0/3848987344 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd8143f00 con 0x7f9dd81478c0 2021-01-06T22:03:47.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9dc9ffb700 1 -- 172.21.15.203:0/3848987344 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc0058a0 con 0x7f9dd81478c0 2021-01-06T22:03:47.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.243+0000 7f9ddefca700 1 -- 172.21.15.203:0/3848987344 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd80ba220 con 0x7f9dd81478c0 2021-01-06T22:03:47.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.247+0000 7f9ddefca700 1 -- 172.21.15.203:0/3848987344 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd80450a0 con 0x7f9dd81478c0 2021-01-06T22:03:47.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.247+0000 7f9dc9ffb700 1 -- 172.21.15.203:0/3848987344 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f9dcc01a060 con 0x7f9dd81478c0 2021-01-06T22:03:47.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.247+0000 7f9dc9ffb700 1 --2- 172.21.15.203:0/3848987344 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9db80563d0 0x7f9db8058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.247+0000 7f9dc9ffb700 1 -- 172.21.15.203:0/3848987344 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9dcc06ed40 con 0x7f9dd81478c0 2021-01-06T22:03:47.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.247+0000 7f9dddfc8700 1 --2- 172.21.15.203:0/3848987344 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9db80563d0 0x7f9db8058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.247+0000 7f9dddfc8700 1 --2- 172.21.15.203:0/3848987344 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9db80563d0 0x7f9db8058890 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f9dd40092d0 tx=0x7f9dd4017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:47.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.251+0000 7f9dc9ffb700 1 -- 172.21.15.203:0/3848987344 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f9dcc0444b0 con 0x7f9dd81478c0 2021-01-06T22:03:47.408 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.407+0000 7f9ddefca700 1 -- 172.21.15.203:0/3848987344 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9dd805ca60 con 0x7f9dd81478c0 2021-01-06T22:03:47.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.407+0000 7f9dc9ffb700 1 -- 172.21.15.203:0/3848987344 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f9dcc018030 con 0x7f9dd81478c0 2021-01-06T22:03:47.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.411+0000 7f9ddefca700 1 -- 172.21.15.203:0/3848987344 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9db80563d0 msgr2=0x7f9db8058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.411+0000 7f9ddefca700 1 --2- 172.21.15.203:0/3848987344 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9db80563d0 0x7f9db8058890 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f9dd40092d0 tx=0x7f9dd4017000).stop 2021-01-06T22:03:47.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.411+0000 7f9ddefca700 1 -- 172.21.15.203:0/3848987344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd81478c0 msgr2=0x7f9dd80b6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.411+0000 7f9ddefca700 1 --2- 172.21.15.203:0/3848987344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd81478c0 0x7f9dd80b6eb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f9dcc00c6e0 tx=0x7f9dcc007aa0).stop 2021-01-06T22:03:47.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.411+0000 7f9ddefca700 1 -- 172.21.15.203:0/3848987344 shutdown_connections 2021-01-06T22:03:47.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.411+0000 7f9ddefca700 1 --2- 172.21.15.203:0/3848987344 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9db80563d0 0x7f9db8058890 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.411+0000 7f9ddefca700 1 --2- 172.21.15.203:0/3848987344 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9dd814e310 0x7f9dd80b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.411+0000 7f9ddefca700 1 --2- 172.21.15.203:0/3848987344 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9dd8064380 0x7f9dd8141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.411+0000 7f9ddefca700 1 --2- 172.21.15.203:0/3848987344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd81478c0 0x7f9dd80b6eb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.411+0000 7f9ddefca700 1 -- 172.21.15.203:0/3848987344 >> 172.21.15.203:0/3848987344 conn(0x7f9dd80667b0 msgr2=0x7f9dd8061560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:47.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.411+0000 7f9ddefca700 1 -- 172.21.15.203:0/3848987344 shutdown_connections 2021-01-06T22:03:47.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.411+0000 7f9ddefca700 1 -- 172.21.15.203:0/3848987344 wait complete. 2021-01-06T22:03:47.415 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:47.416 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":23,"fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","created":"2021-01-06T22:03:21.322671+0000","modified":"2021-01-06T22:03:40.744826+0000","last_up_change":"2021-01-06T22:03:31.115611+0000","last_in_change":"2021-01-06T22:03:31.115611+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-06T22:03:25.161465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-06T22:03:35.070364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-01-06T22:03:37.862354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-01-06T22:03:38.738416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e0b35c09-147c-4ccc-b566-c48b81e25412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":18347},{"type":"v1","addr":"172.21.15.203:6815","nonce":18347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":18347},{"type":"v1","addr":"172.21.15.203:6818","nonce":18347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":18347},{"type":"v1","addr":"172.21.15.203:6831","nonce":18347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6821","nonce":18347},{"type":"v1","addr":"172.21.15.203:6827","nonce":18347}]},"public_addr":"172.21.15.203:6815/18347","cluster_addr":"172.21.15.203:6818/18347","heartbeat_back_addr":"172.21.15.203:6831/18347","heartbeat_front_addr":"172.21.15.203:6827/18347","state":["exists","up"]},{"osd":1,"uuid":"8add59a0-2d58-4c1b-a8d1-62049ee9d85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":18337},{"type":"v1","addr":"172.21.15.203:6803","nonce":18337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":18337},{"type":"v1","addr":"172.21.15.203:6805","nonce":18337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":18337},{"type":"v1","addr":"172.21.15.203:6809","nonce":18337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":18337},{"type":"v1","addr":"172.21.15.203:6807","nonce":18337}]},"public_addr":"172.21.15.203:6803/18337","cluster_addr":"172.21.15.203:6805/18337","heartbeat_back_addr":"172.21.15.203:6809/18337","heartbeat_front_addr":"172.21.15.203:6807/18337","state":["exists","up"]},{"osd":2,"uuid":"b294a411-c7c9-477e-8eae-3b7bd0563ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":18354},{"type":"v1","addr":"172.21.15.203:6811","nonce":18354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":18354},{"type":"v1","addr":"172.21.15.203:6819","nonce":18354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6825","nonce":18354},{"type":"v1","addr":"172.21.15.203:6828","nonce":18354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6823","nonce":18354},{"type":"v1","addr":"172.21.15.203:6824","nonce":18354}]},"public_addr":"172.21.15.203:6811/18354","cluster_addr":"172.21.15.203:6819/18354","heartbeat_back_addr":"172.21.15.203:6828/18354","heartbeat_front_addr":"172.21.15.203:6824/18354","state":["exists","up"]},{"osd":3,"uuid":"5878a8f7-1a42-41c3-9810-0ac41a54dd6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":18346},{"type":"v1","addr":"172.21.15.203:6817","nonce":18346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":18346},{"type":"v1","addr":"172.21.15.203:6822","nonce":18346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":18346},{"type":"v1","addr":"172.21.15.203:6833","nonce":18346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":18346},{"type":"v1","addr":"172.21.15.203:6829","nonce":18346}]},"public_addr":"172.21.15.203:6817/18346","cluster_addr":"172.21.15.203:6822/18346","heartbeat_back_addr":"172.21.15.203:6833/18346","heartbeat_front_addr":"172.21.15.203:6829/18346","state":["exists","up"]},{"osd":4,"uuid":"f641a345-81e4-4400-a44c-d2f82fef7f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":18370},{"type":"v1","addr":"172.21.15.91:6809","nonce":18370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":18370},{"type":"v1","addr":"172.21.15.91:6811","nonce":18370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":18370},{"type":"v1","addr":"172.21.15.91:6816","nonce":18370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":18370},{"type":"v1","addr":"172.21.15.91:6813","nonce":18370}]},"public_addr":"172.21.15.91:6809/18370","cluster_addr":"172.21.15.91:6811/18370","heartbeat_back_addr":"172.21.15.91:6816/18370","heartbeat_front_addr":"172.21.15.91:6813/18370","state":["exists","up"]},{"osd":5,"uuid":"c1ff150d-68c6-4e6b-abd4-76af27bb2103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":18372},{"type":"v1","addr":"172.21.15.91:6801","nonce":18372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":18372},{"type":"v1","addr":"172.21.15.91:6803","nonce":18372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":18372},{"type":"v1","addr":"172.21.15.91:6807","nonce":18372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":18372},{"type":"v1","addr":"172.21.15.91:6805","nonce":18372}]},"public_addr":"172.21.15.91:6801/18372","cluster_addr":"172.21.15.91:6803/18372","heartbeat_back_addr":"172.21.15.91:6807/18372","heartbeat_front_addr":"172.21.15.91:6805/18372","state":["exists","up"]},{"osd":6,"uuid":"35a58e1a-0a0a-4f02-a477-adc029038d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6815","nonce":18371},{"type":"v1","addr":"172.21.15.91:6817","nonce":18371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":18371},{"type":"v1","addr":"172.21.15.91:6819","nonce":18371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":18371},{"type":"v1","addr":"172.21.15.91:6823","nonce":18371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":18371},{"type":"v1","addr":"172.21.15.91:6821","nonce":18371}]},"public_addr":"172.21.15.91:6817/18371","cluster_addr":"172.21.15.91:6819/18371","heartbeat_back_addr":"172.21.15.91:6823/18371","heartbeat_front_addr":"172.21.15.91:6821/18371","state":["exists","up"]},{"osd":7,"uuid":"809a2d89-4303-482a-8558-aef907814234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":18373},{"type":"v1","addr":"172.21.15.91:6825","nonce":18373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":18373},{"type":"v1","addr":"172.21.15.91:6827","nonce":18373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":18373},{"type":"v1","addr":"172.21.15.91:6831","nonce":18373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":18373},{"type":"v1","addr":"172.21.15.91:6829","nonce":18373}]},"public_addr":"172.21.15.91:6825/18373","cluster_addr":"172.21.15.91:6827/18373","heartbeat_back_addr":"172.21.15.91:6831/18373","heartbeat_front_addr":"172.21.15.91:6829/18373","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.875052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.103604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.179225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.158512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.206073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.894840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.229708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.248192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-06T22:03:47.425 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2021-01-06T22:03:47.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faedabeb700 1 Processor -- start 2021-01-06T22:03:47.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faedabeb700 1 -- start start 2021-01-06T22:03:47.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faedabeb700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faed414ab60 0x7faed414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faedabeb700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed40bb860 con 0x7faed40c08a0 2021-01-06T22:03:47.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faedabeb700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed414b940 con 0x7faed40c04c0 2021-01-06T22:03:47.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faedabeb700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed40673e0 con 0x7faed414ab60 2021-01-06T22:03:47.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed8987700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7faed40c04c0 0x7faed4066cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47350/0 (socket says 172.21.15.203:47350) 2021-01-06T22:03:47.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed8987700 1 -- 172.21.15.203:0/816682791 learned_addr learned my addr 172.21.15.203:0/816682791 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:47.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed9188700 1 --2- 172.21.15.203:0/816682791 >> v2:172.21.15.91:3300/0 conn(0x7faed414ab60 0x7faed414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.549 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed37fe700 1 -- 172.21.15.203:0/816682791 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1444099391 0 0) 0x7faed414b940 con 0x7faed40c04c0 2021-01-06T22:03:47.549 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed37fe700 1 -- 172.21.15.203:0/816682791 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faeb8003230 con 0x7faed40c04c0 2021-01-06T22:03:47.549 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed37fe700 1 -- 172.21.15.203:0/816682791 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 766338820 0 0) 0x7faeb8003230 con 0x7faed40c04c0 2021-01-06T22:03:47.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed37fe700 1 -- 172.21.15.203:0/816682791 >> v2:172.21.15.91:3300/0 conn(0x7faed414ab60 msgr2=0x7faed414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:47.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed37fe700 1 --2- 172.21.15.203:0/816682791 >> v2:172.21.15.91:3300/0 conn(0x7faed414ab60 0x7faed414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed37fe700 1 -- 172.21.15.203:0/816682791 >> v1:172.21.15.91:6789/0 conn(0x7faed40c08a0 legacy=0x7faed4147cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed37fe700 1 -- 172.21.15.203:0/816682791 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed40bba90 con 0x7faed40c04c0 2021-01-06T22:03:47.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed37fe700 1 -- 172.21.15.203:0/816682791 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7faec80014e0 con 0x7faed40c04c0 2021-01-06T22:03:47.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed37fe700 1 -- 172.21.15.203:0/816682791 >> v1:172.21.15.203:6789/0 conn(0x7faed40c04c0 legacy=0x7faed4066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed37fe700 1 --2- 172.21.15.203:0/816682791 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faeb8003720 0x7faeb8003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed37fe700 1 --2- 172.21.15.203:0/816682791 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faeb80044d0 0x7faeb8004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed37fe700 1 --2- 172.21.15.203:0/816682791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb8004e40 0x7faeb800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed37fe700 1 -- 172.21.15.203:0/816682791 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed40673e0 con 0x7faeb80044d0 2021-01-06T22:03:47.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed37fe700 1 -- 172.21.15.203:0/816682791 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb80041c0 con 0x7faeb8004e40 2021-01-06T22:03:47.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed37fe700 1 -- 172.21.15.203:0/816682791 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb800bc10 con 0x7faeb8003720 2021-01-06T22:03:47.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed9188700 1 --2- 172.21.15.203:0/816682791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb8004e40 0x7faeb800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed8987700 1 --2- 172.21.15.203:0/816682791 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faeb8003720 0x7faeb8003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed3fff700 1 --2- 172.21.15.203:0/816682791 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faeb80044d0 0x7faeb8004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed9188700 1 -- 172.21.15.203:0/816682791 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faeb8003720 msgr2=0x7faeb8003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:47.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed9188700 1 --2- 172.21.15.203:0/816682791 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faeb8003720 0x7faeb8003b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed9188700 1 -- 172.21.15.203:0/816682791 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faeb80044d0 msgr2=0x7faeb8004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:47.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed9188700 1 --2- 172.21.15.203:0/816682791 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faeb80044d0 0x7faeb8004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed9188700 1 -- 172.21.15.203:0/816682791 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb8003230 con 0x7faeb8004e40 2021-01-06T22:03:47.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed3fff700 1 --2- 172.21.15.203:0/816682791 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faeb80044d0 0x7faeb8004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:47.555 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.543+0000 7faed9188700 1 -- 172.21.15.203:0/816682791 reap_dead start 2021-01-06T22:03:47.555 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.547+0000 7faed9188700 1 --2- 172.21.15.203:0/816682791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb8004e40 0x7faeb800b720 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7faecc009e90 tx=0x7faecc0091d0).ready entity=mon.1 client_cookie=bee616c6ec2fcc1 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:47.555 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.547+0000 7faed37fe700 1 -- 172.21.15.203:0/816682791 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc00ecf0 con 0x7faeb8004e40 2021-01-06T22:03:47.555 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.547+0000 7faed37fe700 1 -- 172.21.15.203:0/816682791 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faecc00ee50 con 0x7faeb8004e40 2021-01-06T22:03:47.556 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.547+0000 7faed37fe700 1 -- 172.21.15.203:0/816682791 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc012600 con 0x7faeb8004e40 2021-01-06T22:03:47.556 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.547+0000 7faedabeb700 1 -- 172.21.15.203:0/816682791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb8004e40 msgr2=0x7faeb800b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.556 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.547+0000 7faedabeb700 1 --2- 172.21.15.203:0/816682791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb8004e40 0x7faeb800b720 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7faecc009e90 tx=0x7faecc0091d0).stop 2021-01-06T22:03:47.556 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.547+0000 7faedabeb700 1 -- 172.21.15.203:0/816682791 shutdown_connections 2021-01-06T22:03:47.556 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.547+0000 7faedabeb700 1 --2- 172.21.15.203:0/816682791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb8004e40 0x7faeb800b720 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.547+0000 7faedabeb700 1 -- 172.21.15.203:0/816682791 >> 172.21.15.203:0/816682791 conn(0x7faed4056cd0 msgr2=0x7faed40626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:47.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.547+0000 7faedabeb700 1 -- 172.21.15.203:0/816682791 shutdown_connections 2021-01-06T22:03:47.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.547+0000 7faedabeb700 1 -- 172.21.15.203:0/816682791 wait complete. 2021-01-06T22:03:47.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.547+0000 7faedabeb700 1 Processor -- start 2021-01-06T22:03:47.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.547+0000 7faedabeb700 1 -- start start 2021-01-06T22:03:47.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faedabeb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb8004e40 0x7faed40bee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faedabeb700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faed41489a0 0x7faed414c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faedabeb700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faed414c5f0 0x7faed415eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faedabeb700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed414ca70 con 0x7faed41489a0 2021-01-06T22:03:47.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faedabeb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed40bfbe0 con 0x7faeb8004e40 2021-01-06T22:03:47.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faedabeb700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed40bfe10 con 0x7faed414c5f0 2021-01-06T22:03:47.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faed8987700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb8004e40 0x7faed40bee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faed8987700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb8004e40 0x7faed40bee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48768/0 (socket says 172.21.15.203:48768) 2021-01-06T22:03:47.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faed8987700 1 -- 172.21.15.203:0/2028059195 learned_addr learned my addr 172.21.15.203:0/2028059195 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:47.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faed3fff700 1 --2- 172.21.15.203:0/2028059195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faed41489a0 0x7faed414c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faed9188700 1 --2- 172.21.15.203:0/2028059195 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faed414c5f0 0x7faed415eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faed8987700 1 -- 172.21.15.203:0/2028059195 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faed414c5f0 msgr2=0x7faed415eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faed8987700 1 --2- 172.21.15.203:0/2028059195 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faed414c5f0 0x7faed415eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faed8987700 1 -- 172.21.15.203:0/2028059195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faed41489a0 msgr2=0x7faed414c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faed8987700 1 --2- 172.21.15.203:0/2028059195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faed41489a0 0x7faed414c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faed8987700 1 -- 172.21.15.203:0/2028059195 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed40c00b0 con 0x7faeb8004e40 2021-01-06T22:03:47.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faed9188700 1 --2- 172.21.15.203:0/2028059195 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faed414c5f0 0x7faed415eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:47.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faed8987700 1 --2- 172.21.15.203:0/2028059195 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb8004e40 0x7faed40bee30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7faec8004af0 tx=0x7faec8007740).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:47.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faed0ff9700 1 -- 172.21.15.203:0/2028059195 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec800f8d0 con 0x7faeb8004e40 2021-01-06T22:03:47.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faed0ff9700 1 -- 172.21.15.203:0/2028059195 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec800fa30 con 0x7faeb8004e40 2021-01-06T22:03:47.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faedabeb700 1 -- 172.21.15.203:0/2028059195 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed41315a0 con 0x7faeb8004e40 2021-01-06T22:03:47.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faed0ff9700 1 -- 172.21.15.203:0/2028059195 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec800fdd0 con 0x7faeb8004e40 2021-01-06T22:03:47.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faedabeb700 1 -- 172.21.15.203:0/2028059195 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed4131ae0 con 0x7faeb8004e40 2021-01-06T22:03:47.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faedabeb700 1 -- 172.21.15.203:0/2028059195 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faed405cfe0 con 0x7faeb8004e40 2021-01-06T22:03:47.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.551+0000 7faed0ff9700 1 -- 172.21.15.203:0/2028059195 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7faec801b030 con 0x7faeb8004e40 2021-01-06T22:03:47.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.555+0000 7faed0ff9700 1 --2- 172.21.15.203:0/2028059195 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faeb40563d0 0x7faeb4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.555+0000 7faed0ff9700 1 -- 172.21.15.203:0/2028059195 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faec806ec50 con 0x7faeb8004e40 2021-01-06T22:03:47.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.555+0000 7faed3fff700 1 --2- 172.21.15.203:0/2028059195 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faeb40563d0 0x7faeb4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.555+0000 7faed0ff9700 1 --2- 172.21.15.203:0/2028059195 >> [v2:172.21.15.203:6812/18347,v1:172.21.15.203:6815/18347] conn(0x7faeb405d890 0x7faeb405fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.555+0000 7faed0ff9700 1 -- 172.21.15.203:0/2028059195 --> [v2:172.21.15.203:6812/18347,v1:172.21.15.203:6815/18347] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faeb4060370 con 0x7faeb405d890 2021-01-06T22:03:47.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.555+0000 7faed0ff9700 1 -- 172.21.15.203:0/2028059195 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7faec80712c0 con 0x7faeb8004e40 2021-01-06T22:03:47.568 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.555+0000 7faed9188700 1 --2- 172.21.15.203:0/2028059195 >> [v2:172.21.15.203:6812/18347,v1:172.21.15.203:6815/18347] conn(0x7faeb405d890 0x7faeb405fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.568 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.555+0000 7faed3fff700 1 --2- 172.21.15.203:0/2028059195 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faeb40563d0 0x7faeb4058890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7faed4158490 tx=0x7faec000c040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:47.568 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.555+0000 7faed9188700 1 -- 172.21.15.203:0/2028059195 >> [v2:172.21.15.203:6812/18347,v1:172.21.15.203:6815/18347] conn(0x7faeb405d890 msgr2=0x7faeb405fcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2021-01-06T22:03:47.568 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.555+0000 7faed9188700 1 --2- 172.21.15.203:0/2028059195 >> [v2:172.21.15.203:6812/18347,v1:172.21.15.203:6815/18347] conn(0x7faeb405d890 0x7faeb405fcf0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:47.568 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.555+0000 7faed0ff9700 1 -- 172.21.15.203:0/2028059195 <== osd.0 v2:172.21.15.203:6812/18347 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22884 (crc 0 0 0) 0x7faeb4060370 con 0x7faeb405d890 2021-01-06T22:03:47.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.575+0000 7faedabeb700 1 -- 172.21.15.203:0/2028059195 --> [v2:172.21.15.203:6812/18347,v1:172.21.15.203:6815/18347] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faed40576d0 con 0x7faeb405d890 2021-01-06T22:03:47.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.575+0000 7faed0ff9700 1 -- 172.21.15.203:0/2028059195 <== osd.0 v2:172.21.15.203:6812/18347 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faed40576d0 con 0x7faeb405d890 2021-01-06T22:03:47.576 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.575+0000 7faedabeb700 1 -- 172.21.15.203:0/2028059195 >> [v2:172.21.15.203:6812/18347,v1:172.21.15.203:6815/18347] conn(0x7faeb405d890 msgr2=0x7faeb405fcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.577 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.575+0000 7faedabeb700 1 --2- 172.21.15.203:0/2028059195 >> [v2:172.21.15.203:6812/18347,v1:172.21.15.203:6815/18347] conn(0x7faeb405d890 0x7faeb405fcf0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.577 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.575+0000 7faedabeb700 1 -- 172.21.15.203:0/2028059195 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faeb40563d0 msgr2=0x7faeb4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.577 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.575+0000 7faedabeb700 1 --2- 172.21.15.203:0/2028059195 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faeb40563d0 0x7faeb4058890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7faed4158490 tx=0x7faec000c040).stop 2021-01-06T22:03:47.577 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.575+0000 7faedabeb700 1 -- 172.21.15.203:0/2028059195 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb8004e40 msgr2=0x7faed40bee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.577 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.575+0000 7faedabeb700 1 --2- 172.21.15.203:0/2028059195 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb8004e40 0x7faed40bee30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7faec8004af0 tx=0x7faec8007740).stop 2021-01-06T22:03:47.578 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.575+0000 7faed9188700 1 -- 172.21.15.203:0/2028059195 reap_dead start 2021-01-06T22:03:47.578 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.575+0000 7faedabeb700 1 -- 172.21.15.203:0/2028059195 shutdown_connections 2021-01-06T22:03:47.578 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.575+0000 7faedabeb700 1 -- 172.21.15.203:0/2028059195 >> 172.21.15.203:0/2028059195 conn(0x7faed4056cd0 msgr2=0x7faed40bace0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:47.579 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.575+0000 7faedabeb700 1 -- 172.21.15.203:0/2028059195 shutdown_connections 2021-01-06T22:03:47.579 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.575+0000 7faedabeb700 1 -- 172.21.15.203:0/2028059195 wait complete. 2021-01-06T22:03:47.595 INFO:teuthology.orchestra.run.smithi203.stdout:55834574854 2021-01-06T22:03:47.595 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2021-01-06T22:03:47.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.723+0000 7f26aee76700 1 Processor -- start 2021-01-06T22:03:47.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.723+0000 7f26aee76700 1 -- start start 2021-01-06T22:03:47.728 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.723+0000 7f26aee76700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f26a8148170 0x7f26a80ba520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.728 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.723+0000 7f26aee76700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a80baa10 con 0x7f26a814a990 2021-01-06T22:03:47.728 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.723+0000 7f26aee76700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a80b9070 con 0x7f26a8147d90 2021-01-06T22:03:47.728 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.723+0000 7f26aee76700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a80bac60 con 0x7f26a8148170 2021-01-06T22:03:47.728 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26ad673700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f26a8148170 0x7f26a80ba520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.729 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26ad673700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f26a8148170 0x7f26a80ba520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48778/0 (socket says 172.21.15.203:48778) 2021-01-06T22:03:47.729 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26ad673700 1 -- 172.21.15.203:0/2578717321 learned_addr learned my addr 172.21.15.203:0/2578717321 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:47.729 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26ade74700 1 --1- 172.21.15.203:0/2578717321 >> v1:172.21.15.203:6789/0 conn(0x7f26a8147d90 0x7f26a8067bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47372/0 (socket says 172.21.15.203:47372) 2021-01-06T22:03:47.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26ad673700 1 -- 172.21.15.203:0/2578717321 >> v1:172.21.15.203:6789/0 conn(0x7f26a8147d90 legacy=0x7f26a8067bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:47.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26ad673700 1 -- 172.21.15.203:0/2578717321 >> v1:172.21.15.91:6789/0 conn(0x7f26a814a990 legacy=0x7f26a80b9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:47.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26ad673700 1 -- 172.21.15.203:0/2578717321 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a80b8890 con 0x7f26a8148170 2021-01-06T22:03:47.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26ad673700 1 --2- 172.21.15.203:0/2578717321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a8148170 0x7f26a80ba520 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f269c00b9e0 tx=0x7f269c006ea0).ready entity=mon.1 client_cookie=55bdf656fdf918b2 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:47.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26ace72700 1 -- 172.21.15.203:0/2578717321 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c009890 con 0x7f26a8148170 2021-01-06T22:03:47.731 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26ace72700 1 -- 172.21.15.203:0/2578717321 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269c0099f0 con 0x7f26a8148170 2021-01-06T22:03:47.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26ace72700 1 -- 172.21.15.203:0/2578717321 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c009d70 con 0x7f26a8148170 2021-01-06T22:03:47.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26aee76700 1 -- 172.21.15.203:0/2578717321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a8148170 msgr2=0x7f26a80ba520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26aee76700 1 --2- 172.21.15.203:0/2578717321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a8148170 0x7f26a80ba520 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f269c00b9e0 tx=0x7f269c006ea0).stop 2021-01-06T22:03:47.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26aee76700 1 -- 172.21.15.203:0/2578717321 shutdown_connections 2021-01-06T22:03:47.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26aee76700 1 --2- 172.21.15.203:0/2578717321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a8148170 0x7f26a80ba520 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26aee76700 1 -- 172.21.15.203:0/2578717321 >> 172.21.15.203:0/2578717321 conn(0x7f26a80c08c0 msgr2=0x7f26a80c0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:47.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26aee76700 1 -- 172.21.15.203:0/2578717321 shutdown_connections 2021-01-06T22:03:47.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26aee76700 1 -- 172.21.15.203:0/2578717321 wait complete. 2021-01-06T22:03:47.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26aee76700 1 Processor -- start 2021-01-06T22:03:47.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.727+0000 7f26aee76700 1 -- start start 2021-01-06T22:03:47.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26aee76700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26a8147d90 0x7f26a8144270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.742 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26aee76700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26a8148170 0x7f26a815efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.742 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26aee76700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a814a990 0x7f26a815f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.743 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26aee76700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a815f9b0 con 0x7f26a8148170 2021-01-06T22:03:47.743 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26aee76700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a80b6ed0 con 0x7f26a814a990 2021-01-06T22:03:47.743 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26aee76700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a80b6730 con 0x7f26a8147d90 2021-01-06T22:03:47.743 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26ae675700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a814a990 0x7f26a815f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.744 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26ae675700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a814a990 0x7f26a815f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48788/0 (socket says 172.21.15.203:48788) 2021-01-06T22:03:47.744 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26ae675700 1 -- 172.21.15.203:0/1825442909 learned_addr learned my addr 172.21.15.203:0/1825442909 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:47.744 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26ad673700 1 --2- 172.21.15.203:0/1825442909 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26a8148170 0x7f26a815efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.744 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26ade74700 1 --2- 172.21.15.203:0/1825442909 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26a8147d90 0x7f26a8144270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.745 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26ae675700 1 -- 172.21.15.203:0/1825442909 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26a8147d90 msgr2=0x7f26a8144270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.745 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26ae675700 1 --2- 172.21.15.203:0/1825442909 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26a8147d90 0x7f26a8144270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.745 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26ae675700 1 -- 172.21.15.203:0/1825442909 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26a8148170 msgr2=0x7f26a815efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.745 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26ae675700 1 --2- 172.21.15.203:0/1825442909 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26a8148170 0x7f26a815efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.745 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26ae675700 1 -- 172.21.15.203:0/1825442909 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a80b7430 con 0x7f26a814a990 2021-01-06T22:03:47.746 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26ae675700 1 --2- 172.21.15.203:0/1825442909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a814a990 0x7f26a815f4c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f26a0009710 tx=0x7f26a0006ce0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:47.746 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26a5ffb700 1 -- 172.21.15.203:0/1825442909 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a000d560 con 0x7f26a814a990 2021-01-06T22:03:47.746 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26a5ffb700 1 -- 172.21.15.203:0/1825442909 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a0000e90 con 0x7f26a814a990 2021-01-06T22:03:47.746 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26aee76700 1 -- 172.21.15.203:0/1825442909 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a80b6990 con 0x7f26a814a990 2021-01-06T22:03:47.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26a5ffb700 1 -- 172.21.15.203:0/1825442909 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a000d910 con 0x7f26a814a990 2021-01-06T22:03:47.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26aee76700 1 -- 172.21.15.203:0/1825442909 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a80b76d0 con 0x7f26a814a990 2021-01-06T22:03:47.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26aee76700 1 -- 172.21.15.203:0/1825442909 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f26a80b9700 con 0x7f26a814a990 2021-01-06T22:03:47.748 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26a5ffb700 1 -- 172.21.15.203:0/1825442909 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f26a000f030 con 0x7f26a814a990 2021-01-06T22:03:47.748 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26a5ffb700 1 --2- 172.21.15.203:0/1825442909 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f26880563d0 0x7f2688058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.748 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.731+0000 7f26a5ffb700 1 -- 172.21.15.203:0/1825442909 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f26a006f190 con 0x7f26a814a990 2021-01-06T22:03:47.749 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.735+0000 7f26ade74700 1 --2- 172.21.15.203:0/1825442909 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f26880563d0 0x7f2688058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.735+0000 7f26a5ffb700 1 --2- 172.21.15.203:0/1825442909 >> [v2:172.21.15.203:6802/18337,v1:172.21.15.203:6803/18337] conn(0x7f268805d890 0x7f268805fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.735+0000 7f26a5ffb700 1 -- 172.21.15.203:0/1825442909 --> [v2:172.21.15.203:6802/18337,v1:172.21.15.203:6803/18337] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2688060370 con 0x7f268805d890 2021-01-06T22:03:47.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.735+0000 7f26a5ffb700 1 -- 172.21.15.203:0/1825442909 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26a0071800 con 0x7f26a814a990 2021-01-06T22:03:47.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.735+0000 7f26ad673700 1 --2- 172.21.15.203:0/1825442909 >> [v2:172.21.15.203:6802/18337,v1:172.21.15.203:6803/18337] conn(0x7f268805d890 0x7f268805fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.735+0000 7f26ade74700 1 --2- 172.21.15.203:0/1825442909 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f26880563d0 0x7f2688058890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f2698007fd0 tx=0x7f2698007e50).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:47.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.735+0000 7f26ad673700 1 -- 172.21.15.203:0/1825442909 >> [v2:172.21.15.203:6802/18337,v1:172.21.15.203:6803/18337] conn(0x7f268805d890 msgr2=0x7f268805fcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2021-01-06T22:03:47.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.735+0000 7f26ad673700 1 --2- 172.21.15.203:0/1825442909 >> [v2:172.21.15.203:6802/18337,v1:172.21.15.203:6803/18337] conn(0x7f268805d890 0x7f268805fcf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:47.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.735+0000 7f26a5ffb700 1 -- 172.21.15.203:0/1825442909 <== osd.1 v2:172.21.15.203:6802/18337 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22884 (crc 0 0 0) 0x7f2688060370 con 0x7f268805d890 2021-01-06T22:03:47.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.755+0000 7f26aee76700 1 -- 172.21.15.203:0/1825442909 --> [v2:172.21.15.203:6802/18337,v1:172.21.15.203:6803/18337] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f26a80450a0 con 0x7f268805d890 2021-01-06T22:03:47.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.755+0000 7f26a5ffb700 1 -- 172.21.15.203:0/1825442909 <== osd.1 v2:172.21.15.203:6802/18337 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f26a80450a0 con 0x7f268805d890 2021-01-06T22:03:47.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.755+0000 7f26aee76700 1 -- 172.21.15.203:0/1825442909 >> [v2:172.21.15.203:6802/18337,v1:172.21.15.203:6803/18337] conn(0x7f268805d890 msgr2=0x7f268805fcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.755+0000 7f26aee76700 1 --2- 172.21.15.203:0/1825442909 >> [v2:172.21.15.203:6802/18337,v1:172.21.15.203:6803/18337] conn(0x7f268805d890 0x7f268805fcf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.755+0000 7f26aee76700 1 -- 172.21.15.203:0/1825442909 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f26880563d0 msgr2=0x7f2688058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.755+0000 7f26aee76700 1 --2- 172.21.15.203:0/1825442909 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f26880563d0 0x7f2688058890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f2698007fd0 tx=0x7f2698007e50).stop 2021-01-06T22:03:47.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.755+0000 7f26aee76700 1 -- 172.21.15.203:0/1825442909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a814a990 msgr2=0x7f26a815f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.755+0000 7f26aee76700 1 --2- 172.21.15.203:0/1825442909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a814a990 0x7f26a815f4c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f26a0009710 tx=0x7f26a0006ce0).stop 2021-01-06T22:03:47.760 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.755+0000 7f26ae675700 1 -- 172.21.15.203:0/1825442909 reap_dead start 2021-01-06T22:03:47.760 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.755+0000 7f26aee76700 1 -- 172.21.15.203:0/1825442909 shutdown_connections 2021-01-06T22:03:47.761 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.755+0000 7f26aee76700 1 -- 172.21.15.203:0/1825442909 >> 172.21.15.203:0/1825442909 conn(0x7f26a80c08c0 msgr2=0x7f26a80bbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:47.761 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.755+0000 7f26aee76700 1 -- 172.21.15.203:0/1825442909 shutdown_connections 2021-01-06T22:03:47.761 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.755+0000 7f26aee76700 1 -- 172.21.15.203:0/1825442909 wait complete. 2021-01-06T22:03:47.772 INFO:teuthology.orchestra.run.smithi203.stdout:60129542149 2021-01-06T22:03:47.772 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2021-01-06T22:03:47.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae49dd8700 1 Processor -- start 2021-01-06T22:03:47.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae49dd8700 1 -- start start 2021-01-06T22:03:47.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae49dd8700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae44063710 0x7fae440b9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae49dd8700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fae44062d80 0x7fae4414e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae49dd8700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae4414ee60 con 0x7fae440629a0 2021-01-06T22:03:47.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae49dd8700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae440b9780 con 0x7fae44062d80 2021-01-06T22:03:47.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae49dd8700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae44064e30 con 0x7fae44063710 2021-01-06T22:03:47.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae495d7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fae44062d80 0x7fae4414e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae495d7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fae44062d80 0x7fae4414e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48798/0 (socket says 172.21.15.203:48798) 2021-01-06T22:03:47.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae495d7700 1 -- 172.21.15.203:0/1612891029 learned_addr learned my addr 172.21.15.203:0/1612891029 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:47.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae43fff700 1 --2- 172.21.15.203:0/1612891029 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae44063710 0x7fae440b9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae495d7700 1 -- 172.21.15.203:0/1612891029 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae44063710 msgr2=0x7fae440b9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:47.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae495d7700 1 --2- 172.21.15.203:0/1612891029 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae44063710 0x7fae440b9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae495d7700 1 -- 172.21.15.203:0/1612891029 >> v1:172.21.15.91:6789/0 conn(0x7fae440629a0 legacy=0x7fae440bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.909 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae495d7700 1 -- 172.21.15.203:0/1612891029 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae440650d0 con 0x7fae44062d80 2021-01-06T22:03:47.909 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae495d7700 1 --2- 172.21.15.203:0/1612891029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae44062d80 0x7fae4414e970 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fae3c009d50 tx=0x7fae3c009290).ready entity=mon.1 client_cookie=6305b5a10b99915b server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:47.911 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae437fe700 1 -- 172.21.15.203:0/1612891029 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c00bbc0 con 0x7fae44062d80 2021-01-06T22:03:47.912 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae437fe700 1 -- 172.21.15.203:0/1612891029 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c00bd20 con 0x7fae44062d80 2021-01-06T22:03:47.912 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.903+0000 7fae437fe700 1 -- 172.21.15.203:0/1612891029 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c011610 con 0x7fae44062d80 2021-01-06T22:03:47.912 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae49dd8700 1 -- 172.21.15.203:0/1612891029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae44062d80 msgr2=0x7fae4414e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.912 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae49dd8700 1 --2- 172.21.15.203:0/1612891029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae44062d80 0x7fae4414e970 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fae3c009d50 tx=0x7fae3c009290).stop 2021-01-06T22:03:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae49dd8700 1 -- 172.21.15.203:0/1612891029 shutdown_connections 2021-01-06T22:03:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae49dd8700 1 --2- 172.21.15.203:0/1612891029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae44062d80 0x7fae4414e970 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae49dd8700 1 --2- 172.21.15.203:0/1612891029 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae44063710 0x7fae440b9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae49dd8700 1 -- 172.21.15.203:0/1612891029 >> 172.21.15.203:0/1612891029 conn(0x7fae44056cd0 msgr2=0x7fae440570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae49dd8700 1 -- 172.21.15.203:0/1612891029 shutdown_connections 2021-01-06T22:03:47.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae49dd8700 1 -- 172.21.15.203:0/1612891029 wait complete. 2021-01-06T22:03:47.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae49dd8700 1 Processor -- start 2021-01-06T22:03:47.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae49dd8700 1 -- start start 2021-01-06T22:03:47.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae49dd8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae440629a0 0x7fae441612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae49dd8700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae44062d80 0x7fae4415d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae49dd8700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae44063710 0x7fae4415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae49dd8700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4415e120 con 0x7fae44063710 2021-01-06T22:03:47.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae49dd8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae440c0560 con 0x7fae440629a0 2021-01-06T22:03:47.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae49dd8700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae440bfdc0 con 0x7fae44062d80 2021-01-06T22:03:47.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae48dd6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae440629a0 0x7fae441612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae43fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae44062d80 0x7fae4415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae48dd6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae440629a0 0x7fae441612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48804/0 (socket says 172.21.15.203:48804) 2021-01-06T22:03:47.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae48dd6700 1 -- 172.21.15.203:0/2409075153 learned_addr learned my addr 172.21.15.203:0/2409075153 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:47.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.907+0000 7fae495d7700 1 --2- 172.21.15.203:0/2409075153 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae44063710 0x7fae4415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae48dd6700 1 -- 172.21.15.203:0/2409075153 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae44062d80 msgr2=0x7fae4415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae48dd6700 1 --2- 172.21.15.203:0/2409075153 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae44062d80 0x7fae4415d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae48dd6700 1 -- 172.21.15.203:0/2409075153 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae44063710 msgr2=0x7fae4415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.918 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae48dd6700 1 --2- 172.21.15.203:0/2409075153 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae44063710 0x7fae4415dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.918 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae48dd6700 1 -- 172.21.15.203:0/2409075153 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4414d530 con 0x7fae440629a0 2021-01-06T22:03:47.918 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae48dd6700 1 --2- 172.21.15.203:0/2409075153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae440629a0 0x7fae441612c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fae3800c740 tx=0x7fae38007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:47.918 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae40ff9700 1 -- 172.21.15.203:0/2409075153 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38007020 con 0x7fae440629a0 2021-01-06T22:03:47.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae40ff9700 1 -- 172.21.15.203:0/2409075153 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae38007180 con 0x7fae440629a0 2021-01-06T22:03:47.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae49dd8700 1 -- 172.21.15.203:0/2409075153 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4414d7d0 con 0x7fae440629a0 2021-01-06T22:03:47.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae40ff9700 1 -- 172.21.15.203:0/2409075153 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3800e560 con 0x7fae440629a0 2021-01-06T22:03:47.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae49dd8700 1 -- 172.21.15.203:0/2409075153 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae440c02c0 con 0x7fae440629a0 2021-01-06T22:03:47.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae49dd8700 1 -- 172.21.15.203:0/2409075153 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fae440450a0 con 0x7fae440629a0 2021-01-06T22:03:47.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae40ff9700 1 -- 172.21.15.203:0/2409075153 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fae3801a060 con 0x7fae440629a0 2021-01-06T22:03:47.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae40ff9700 1 --2- 172.21.15.203:0/2409075153 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fae240563d0 0x7fae24058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae40ff9700 1 -- 172.21.15.203:0/2409075153 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fae3806ee00 con 0x7fae440629a0 2021-01-06T22:03:47.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae40ff9700 1 --2- 172.21.15.203:0/2409075153 >> [v2:172.21.15.203:6810/18354,v1:172.21.15.203:6811/18354] conn(0x7fae2405d890 0x7fae2405fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:47.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae40ff9700 1 -- 172.21.15.203:0/2409075153 --> [v2:172.21.15.203:6810/18354,v1:172.21.15.203:6811/18354] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae24060370 con 0x7fae2405d890 2021-01-06T22:03:47.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.911+0000 7fae40ff9700 1 -- 172.21.15.203:0/2409075153 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae38071470 con 0x7fae440629a0 2021-01-06T22:03:47.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.915+0000 7fae43fff700 1 --2- 172.21.15.203:0/2409075153 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fae240563d0 0x7fae24058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.915+0000 7fae495d7700 1 --2- 172.21.15.203:0/2409075153 >> [v2:172.21.15.203:6810/18354,v1:172.21.15.203:6811/18354] conn(0x7fae2405d890 0x7fae2405fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:47.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.915+0000 7fae43fff700 1 --2- 172.21.15.203:0/2409075153 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fae240563d0 0x7fae24058890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fae44149d50 tx=0x7fae30006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:47.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.915+0000 7fae495d7700 1 -- 172.21.15.203:0/2409075153 >> [v2:172.21.15.203:6810/18354,v1:172.21.15.203:6811/18354] conn(0x7fae2405d890 msgr2=0x7fae2405fcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2021-01-06T22:03:47.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.915+0000 7fae495d7700 1 --2- 172.21.15.203:0/2409075153 >> [v2:172.21.15.203:6810/18354,v1:172.21.15.203:6811/18354] conn(0x7fae2405d890 0x7fae2405fcf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:47.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.915+0000 7fae40ff9700 1 -- 172.21.15.203:0/2409075153 <== osd.2 v2:172.21.15.203:6810/18354 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22884 (crc 0 0 0) 0x7fae24060370 con 0x7fae2405d890 2021-01-06T22:03:47.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.935+0000 7fae49dd8700 1 -- 172.21.15.203:0/2409075153 --> [v2:172.21.15.203:6810/18354,v1:172.21.15.203:6811/18354] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fae44061ee0 con 0x7fae2405d890 2021-01-06T22:03:47.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.935+0000 7fae40ff9700 1 -- 172.21.15.203:0/2409075153 <== osd.2 v2:172.21.15.203:6810/18354 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fae44061ee0 con 0x7fae2405d890 2021-01-06T22:03:47.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.935+0000 7fae49dd8700 1 -- 172.21.15.203:0/2409075153 >> [v2:172.21.15.203:6810/18354,v1:172.21.15.203:6811/18354] conn(0x7fae2405d890 msgr2=0x7fae2405fcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.935+0000 7fae49dd8700 1 --2- 172.21.15.203:0/2409075153 >> [v2:172.21.15.203:6810/18354,v1:172.21.15.203:6811/18354] conn(0x7fae2405d890 0x7fae2405fcf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:47.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.935+0000 7fae49dd8700 1 -- 172.21.15.203:0/2409075153 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fae240563d0 msgr2=0x7fae24058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.935+0000 7fae49dd8700 1 --2- 172.21.15.203:0/2409075153 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fae240563d0 0x7fae24058890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fae44149d50 tx=0x7fae30006de0).stop 2021-01-06T22:03:47.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.935+0000 7fae49dd8700 1 -- 172.21.15.203:0/2409075153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae440629a0 msgr2=0x7fae441612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:47.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.935+0000 7fae49dd8700 1 --2- 172.21.15.203:0/2409075153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae440629a0 0x7fae441612c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fae3800c740 tx=0x7fae38007c40).stop 2021-01-06T22:03:47.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.935+0000 7fae495d7700 1 -- 172.21.15.203:0/2409075153 reap_dead start 2021-01-06T22:03:47.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.935+0000 7fae49dd8700 1 -- 172.21.15.203:0/2409075153 shutdown_connections 2021-01-06T22:03:47.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.935+0000 7fae49dd8700 1 -- 172.21.15.203:0/2409075153 >> 172.21.15.203:0/2409075153 conn(0x7fae44056cd0 msgr2=0x7fae440b9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:47.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.935+0000 7fae49dd8700 1 -- 172.21.15.203:0/2409075153 shutdown_connections 2021-01-06T22:03:47.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:47.935+0000 7fae49dd8700 1 -- 172.21.15.203:0/2409075153 wait complete. 2021-01-06T22:03:47.959 INFO:teuthology.orchestra.run.smithi203.stdout:60129542150 2021-01-06T22:03:47.959 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2021-01-06T22:03:48.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762759e700 1 Processor -- start 2021-01-06T22:03:48.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762759e700 1 -- start start 2021-01-06T22:03:48.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762759e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7628063c50 0x7f762814f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762759e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7628068370 0x7f762814f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762759e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76280ba9e0 con 0x7f7628064030 2021-01-06T22:03:48.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762759e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f762814fda0 con 0x7f7628063c50 2021-01-06T22:03:48.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762759e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76280bac80 con 0x7f7628068370 2021-01-06T22:03:48.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762659c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7628063c50 0x7f762814f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762659c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7628063c50 0x7f762814f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48810/0 (socket says 172.21.15.203:48810) 2021-01-06T22:03:48.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762659c700 1 -- 172.21.15.203:0/2933987548 learned_addr learned my addr 172.21.15.203:0/2933987548 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:48.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f7625d9b700 1 --2- 172.21.15.203:0/2933987548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7628068370 0x7f762814f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762659c700 1 -- 172.21.15.203:0/2933987548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7628068370 msgr2=0x7f762814f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:48.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762659c700 1 --2- 172.21.15.203:0/2933987548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7628068370 0x7f762814f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762659c700 1 -- 172.21.15.203:0/2933987548 >> v1:172.21.15.91:6789/0 conn(0x7f7628064030 legacy=0x7f7628155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762659c700 1 -- 172.21.15.203:0/2933987548 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76280baf20 con 0x7f7628063c50 2021-01-06T22:03:48.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762659c700 1 --2- 172.21.15.203:0/2933987548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7628063c50 0x7f762814f3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f761800b9e0 tx=0x7f7618006ea0).ready entity=mon.1 client_cookie=d9e31f6ab4be45eb server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762559a700 1 -- 172.21.15.203:0/2933987548 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618009810 con 0x7f7628063c50 2021-01-06T22:03:48.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762559a700 1 -- 172.21.15.203:0/2933987548 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7618009970 con 0x7f7628063c50 2021-01-06T22:03:48.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762559a700 1 -- 172.21.15.203:0/2933987548 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618009cf0 con 0x7f7628063c50 2021-01-06T22:03:48.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762759e700 1 -- 172.21.15.203:0/2933987548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7628063c50 msgr2=0x7f762814f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762759e700 1 --2- 172.21.15.203:0/2933987548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7628063c50 0x7f762814f3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f761800b9e0 tx=0x7f7618006ea0).stop 2021-01-06T22:03:48.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762759e700 1 -- 172.21.15.203:0/2933987548 shutdown_connections 2021-01-06T22:03:48.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762759e700 1 --2- 172.21.15.203:0/2933987548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7628063c50 0x7f762814f3c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762759e700 1 --2- 172.21.15.203:0/2933987548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7628068370 0x7f762814f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762759e700 1 -- 172.21.15.203:0/2933987548 >> 172.21.15.203:0/2933987548 conn(0x7f76280631f0 msgr2=0x7f76280c0040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:48.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762759e700 1 -- 172.21.15.203:0/2933987548 shutdown_connections 2021-01-06T22:03:48.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.087+0000 7f762759e700 1 -- 172.21.15.203:0/2933987548 wait complete. 2021-01-06T22:03:48.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f762759e700 1 Processor -- start 2021-01-06T22:03:48.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f762759e700 1 -- start start 2021-01-06T22:03:48.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f762759e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7628063c50 0x7f76281547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f762759e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7628064030 0x7f76280be0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f762759e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7628068370 0x7f76280be5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f762759e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7628161a20 con 0x7f7628063c50 2021-01-06T22:03:48.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f762759e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76280bead0 con 0x7f7628068370 2021-01-06T22:03:48.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f762759e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7628161c10 con 0x7f7628064030 2021-01-06T22:03:48.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f7626d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7628068370 0x7f76280be5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f7626d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7628068370 0x7f76280be5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48820/0 (socket says 172.21.15.203:48820) 2021-01-06T22:03:48.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f7625d9b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7628064030 0x7f76280be0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f762659c700 1 --2- 172.21.15.203:0/24792429 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7628063c50 0x7f76281547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f7626d9d700 1 -- 172.21.15.203:0/24792429 learned_addr learned my addr 172.21.15.203:0/24792429 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:48.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f7626d9d700 1 -- 172.21.15.203:0/24792429 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7628064030 msgr2=0x7f76280be0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f7626d9d700 1 --2- 172.21.15.203:0/24792429 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7628064030 0x7f76280be0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f7626d9d700 1 -- 172.21.15.203:0/24792429 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7628063c50 msgr2=0x7f76281547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f7626d9d700 1 --2- 172.21.15.203:0/24792429 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7628063c50 0x7f76281547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f7626d9d700 1 -- 172.21.15.203:0/24792429 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7628161eb0 con 0x7f7628068370 2021-01-06T22:03:48.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f7626d9d700 1 --2- 172.21.15.203:0/24792429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7628068370 0x7f76280be5e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f762000ca60 tx=0x7f7620007e90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f76167fc700 1 -- 172.21.15.203:0/24792429 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762000ce30 con 0x7f7628068370 2021-01-06T22:03:48.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f76167fc700 1 -- 172.21.15.203:0/24792429 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762000a800 con 0x7f7628068370 2021-01-06T22:03:48.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f762759e700 1 -- 172.21.15.203:0/24792429 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7628162150 con 0x7f7628068370 2021-01-06T22:03:48.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f76167fc700 1 -- 172.21.15.203:0/24792429 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762000aba0 con 0x7f7628068370 2021-01-06T22:03:48.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.091+0000 7f762759e700 1 -- 172.21.15.203:0/24792429 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7628162690 con 0x7f7628068370 2021-01-06T22:03:48.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.095+0000 7f762759e700 1 -- 172.21.15.203:0/24792429 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7628162990 con 0x7f7628068370 2021-01-06T22:03:48.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.095+0000 7f76167fc700 1 -- 172.21.15.203:0/24792429 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f7620019030 con 0x7f7628068370 2021-01-06T22:03:48.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.095+0000 7f76167fc700 1 --2- 172.21.15.203:0/24792429 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f76040563d0 0x7f7604058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.095+0000 7f76167fc700 1 -- 172.21.15.203:0/24792429 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f762006ec30 con 0x7f7628068370 2021-01-06T22:03:48.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.095+0000 7f76167fc700 1 --2- 172.21.15.203:0/24792429 >> [v2:172.21.15.203:6813/18346,v1:172.21.15.203:6817/18346] conn(0x7f760405d890 0x7f760405fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.095+0000 7f76167fc700 1 -- 172.21.15.203:0/24792429 --> [v2:172.21.15.203:6813/18346,v1:172.21.15.203:6817/18346] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7604060370 con 0x7f760405d890 2021-01-06T22:03:48.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.095+0000 7f76167fc700 1 -- 172.21.15.203:0/24792429 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76200712a0 con 0x7f7628068370 2021-01-06T22:03:48.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.095+0000 7f762659c700 1 --2- 172.21.15.203:0/24792429 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f76040563d0 0x7f7604058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.095+0000 7f7625d9b700 1 --2- 172.21.15.203:0/24792429 >> [v2:172.21.15.203:6813/18346,v1:172.21.15.203:6817/18346] conn(0x7f760405d890 0x7f760405fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.095+0000 7f7625d9b700 1 -- 172.21.15.203:0/24792429 >> [v2:172.21.15.203:6813/18346,v1:172.21.15.203:6817/18346] conn(0x7f760405d890 msgr2=0x7f760405fcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2021-01-06T22:03:48.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.095+0000 7f7625d9b700 1 --2- 172.21.15.203:0/24792429 >> [v2:172.21.15.203:6813/18346,v1:172.21.15.203:6817/18346] conn(0x7f760405d890 0x7f760405fcf0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.095+0000 7f762659c700 1 --2- 172.21.15.203:0/24792429 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f76040563d0 0x7f7604058890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f7618006ee0 tx=0x7f7618016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.099+0000 7f76167fc700 1 -- 172.21.15.203:0/24792429 <== osd.3 v2:172.21.15.203:6813/18346 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22884 (crc 0 0 0) 0x7f7604060370 con 0x7f760405d890 2021-01-06T22:03:48.118 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.115+0000 7f762759e700 1 -- 172.21.15.203:0/24792429 --> [v2:172.21.15.203:6813/18346,v1:172.21.15.203:6817/18346] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f76280450a0 con 0x7f760405d890 2021-01-06T22:03:48.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.119+0000 7f76167fc700 1 -- 172.21.15.203:0/24792429 <== osd.3 v2:172.21.15.203:6813/18346 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f76280450a0 con 0x7f760405d890 2021-01-06T22:03:48.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.119+0000 7f762759e700 1 -- 172.21.15.203:0/24792429 >> [v2:172.21.15.203:6813/18346,v1:172.21.15.203:6817/18346] conn(0x7f760405d890 msgr2=0x7f760405fcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.119+0000 7f762759e700 1 --2- 172.21.15.203:0/24792429 >> [v2:172.21.15.203:6813/18346,v1:172.21.15.203:6817/18346] conn(0x7f760405d890 0x7f760405fcf0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.119+0000 7f762759e700 1 -- 172.21.15.203:0/24792429 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f76040563d0 msgr2=0x7f7604058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.119+0000 7f762759e700 1 --2- 172.21.15.203:0/24792429 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f76040563d0 0x7f7604058890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f7618006ee0 tx=0x7f7618016000).stop 2021-01-06T22:03:48.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.119+0000 7f762759e700 1 -- 172.21.15.203:0/24792429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7628068370 msgr2=0x7f76280be5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.119+0000 7f762759e700 1 --2- 172.21.15.203:0/24792429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7628068370 0x7f76280be5e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f762000ca60 tx=0x7f7620007e90).stop 2021-01-06T22:03:48.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.119+0000 7f7626d9d700 1 -- 172.21.15.203:0/24792429 reap_dead start 2021-01-06T22:03:48.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.119+0000 7f762759e700 1 -- 172.21.15.203:0/24792429 shutdown_connections 2021-01-06T22:03:48.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.119+0000 7f762759e700 1 -- 172.21.15.203:0/24792429 >> 172.21.15.203:0/24792429 conn(0x7f76280631f0 msgr2=0x7f7628060560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:48.122 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.119+0000 7f762759e700 1 -- 172.21.15.203:0/24792429 shutdown_connections 2021-01-06T22:03:48.122 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.119+0000 7f762759e700 1 -- 172.21.15.203:0/24792429 wait complete. 2021-01-06T22:03:48.132 INFO:teuthology.orchestra.run.smithi203.stdout:60129542149 2021-01-06T22:03:48.132 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2021-01-06T22:03:48.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.255+0000 7feeaac4f700 1 Processor -- start 2021-01-06T22:03:48.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.255+0000 7feeaac4f700 1 -- start start 2021-01-06T22:03:48.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feeaac4f700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7feea4064bc0 0x7feea40bf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feeaac4f700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea4151560 0x7feea4147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feeaac4f700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea4152460 con 0x7feea40bf100 2021-01-06T22:03:48.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feeaac4f700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea414e740 con 0x7feea4064bc0 2021-01-06T22:03:48.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feeaac4f700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea414e9e0 con 0x7feea4151560 2021-01-06T22:03:48.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feea91ec700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea4151560 0x7feea4147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feea3fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7feea4064bc0 0x7feea40bf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feea3fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7feea4064bc0 0x7feea40bf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:56676/0 (socket says 172.21.15.203:56676) 2021-01-06T22:03:48.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feea91ec700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea4151560 0x7feea4147bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:53912/0 (socket says 172.21.15.203:53912) 2021-01-06T22:03:48.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feea3fff700 1 -- 172.21.15.203:0/3265541652 learned_addr learned my addr 172.21.15.203:0/3265541652 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:48.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feea89eb700 1 --1- 172.21.15.203:0/3265541652 >> v1:172.21.15.91:6789/0 conn(0x7feea40bf100 0x7feea414df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:42074/0 (socket says 172.21.15.203:42074) 2021-01-06T22:03:48.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feea91ec700 1 -- 172.21.15.203:0/3265541652 >> v2:172.21.15.91:3300/0 conn(0x7feea4064bc0 msgr2=0x7feea40bf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:48.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feea91ec700 1 --2- 172.21.15.203:0/3265541652 >> v2:172.21.15.91:3300/0 conn(0x7feea4064bc0 0x7feea40bf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feea91ec700 1 -- 172.21.15.203:0/3265541652 >> v1:172.21.15.91:6789/0 conn(0x7feea40bf100 legacy=0x7feea414df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feea91ec700 1 -- 172.21.15.203:0/3265541652 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea414ec80 con 0x7feea4151560 2021-01-06T22:03:48.267 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feea3fff700 1 --2- 172.21.15.203:0/3265541652 >> v2:172.21.15.91:3300/0 conn(0x7feea4064bc0 0x7feea40bf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:48.267 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feea91ec700 1 --2- 172.21.15.203:0/3265541652 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea4151560 0x7feea4147bf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fee9c009c90 tx=0x7fee9c0091d0).ready entity=mon.2 client_cookie=83a2c58ad8254c1a server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.267 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feea37fe700 1 -- 172.21.15.203:0/3265541652 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00ba70 con 0x7feea4151560 2021-01-06T22:03:48.268 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feea37fe700 1 -- 172.21.15.203:0/3265541652 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c00bbd0 con 0x7feea4151560 2021-01-06T22:03:48.268 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.259+0000 7feea37fe700 1 -- 172.21.15.203:0/3265541652 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c011400 con 0x7feea4151560 2021-01-06T22:03:48.269 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.263+0000 7feeaac4f700 1 -- 172.21.15.203:0/3265541652 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea4151560 msgr2=0x7feea4147bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.269 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.263+0000 7feeaac4f700 1 --2- 172.21.15.203:0/3265541652 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea4151560 0x7feea4147bf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fee9c009c90 tx=0x7fee9c0091d0).stop 2021-01-06T22:03:48.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.263+0000 7feeaac4f700 1 -- 172.21.15.203:0/3265541652 shutdown_connections 2021-01-06T22:03:48.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.263+0000 7feeaac4f700 1 --2- 172.21.15.203:0/3265541652 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea4151560 0x7feea4147bf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.263+0000 7feeaac4f700 1 --2- 172.21.15.203:0/3265541652 >> v2:172.21.15.91:3300/0 conn(0x7feea4064bc0 0x7feea40bf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.263+0000 7feeaac4f700 1 -- 172.21.15.203:0/3265541652 >> 172.21.15.203:0/3265541652 conn(0x7feea40602b0 msgr2=0x7feea40606c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:48.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.263+0000 7feeaac4f700 1 -- 172.21.15.203:0/3265541652 shutdown_connections 2021-01-06T22:03:48.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.263+0000 7feeaac4f700 1 -- 172.21.15.203:0/3265541652 wait complete. 2021-01-06T22:03:48.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.263+0000 7feeaac4f700 1 Processor -- start 2021-01-06T22:03:48.275 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.263+0000 7feeaac4f700 1 -- start start 2021-01-06T22:03:48.276 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.263+0000 7feeaac4f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feea4064bc0 0x7feea40c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.276 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.263+0000 7feeaac4f700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea40bf100 0x7feea414a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.276 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.263+0000 7feeaac4f700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea4151560 0x7feea414ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.277 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.263+0000 7feeaac4f700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea40babe0 con 0x7feea4151560 2021-01-06T22:03:48.277 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.263+0000 7feeaac4f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea4150af0 con 0x7feea4064bc0 2021-01-06T22:03:48.277 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.263+0000 7feeaac4f700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea414b240 con 0x7feea40bf100 2021-01-06T22:03:48.278 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feea89eb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feea4064bc0 0x7feea40c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.278 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feea89eb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feea4064bc0 0x7feea40c38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48832/0 (socket says 172.21.15.203:48832) 2021-01-06T22:03:48.278 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feea89eb700 1 -- 172.21.15.203:0/3938367125 learned_addr learned my addr 172.21.15.203:0/3938367125 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:48.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feea3fff700 1 --2- 172.21.15.203:0/3938367125 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea40bf100 0x7feea414a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feea91ec700 1 --2- 172.21.15.203:0/3938367125 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea4151560 0x7feea414ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.282 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feea89eb700 1 -- 172.21.15.203:0/3938367125 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea40bf100 msgr2=0x7feea414a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.282 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feea89eb700 1 --2- 172.21.15.203:0/3938367125 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea40bf100 0x7feea414a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.282 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feea89eb700 1 -- 172.21.15.203:0/3938367125 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea4151560 msgr2=0x7feea414ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.283 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feea89eb700 1 --2- 172.21.15.203:0/3938367125 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea4151560 0x7feea414ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.283 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feea89eb700 1 -- 172.21.15.203:0/3938367125 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea40b1330 con 0x7feea4064bc0 2021-01-06T22:03:48.283 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feea89eb700 1 --2- 172.21.15.203:0/3938367125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feea4064bc0 0x7feea40c38a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fee9000c6e0 tx=0x7fee90007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.284 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feea0ff9700 1 -- 172.21.15.203:0/3938367125 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee900072e0 con 0x7feea4064bc0 2021-01-06T22:03:48.284 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feea0ff9700 1 -- 172.21.15.203:0/3938367125 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee90005520 con 0x7feea4064bc0 2021-01-06T22:03:48.284 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feea0ff9700 1 -- 172.21.15.203:0/3938367125 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee900058a0 con 0x7feea4064bc0 2021-01-06T22:03:48.284 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feeaac4f700 1 -- 172.21.15.203:0/3938367125 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea415e400 con 0x7feea4064bc0 2021-01-06T22:03:48.285 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feeaac4f700 1 -- 172.21.15.203:0/3938367125 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea40b1800 con 0x7feea4064bc0 2021-01-06T22:03:48.285 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.267+0000 7feeaac4f700 1 -- 172.21.15.203:0/3938367125 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feea405d600 con 0x7feea4064bc0 2021-01-06T22:03:48.285 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.271+0000 7feea0ff9700 1 -- 172.21.15.203:0/3938367125 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fee90010070 con 0x7feea4064bc0 2021-01-06T22:03:48.285 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.271+0000 7feea0ff9700 1 --2- 172.21.15.203:0/3938367125 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fee80056400 0x7fee800588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.286 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.271+0000 7feea0ff9700 1 -- 172.21.15.203:0/3938367125 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fee9006eae0 con 0x7feea4064bc0 2021-01-06T22:03:48.286 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.271+0000 7feea3fff700 1 --2- 172.21.15.203:0/3938367125 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fee80056400 0x7fee800588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.286 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.271+0000 7feea0ff9700 1 --2- 172.21.15.203:0/3938367125 >> [v2:172.21.15.91:6808/18370,v1:172.21.15.91:6809/18370] conn(0x7fee8005d8c0 0x7fee8005fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.286 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.271+0000 7feea0ff9700 1 -- 172.21.15.203:0/3938367125 --> [v2:172.21.15.91:6808/18370,v1:172.21.15.91:6809/18370] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee800603a0 con 0x7fee8005d8c0 2021-01-06T22:03:48.287 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.271+0000 7feea0ff9700 1 -- 172.21.15.203:0/3938367125 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee90073220 con 0x7feea4064bc0 2021-01-06T22:03:48.287 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.271+0000 7feea3fff700 1 --2- 172.21.15.203:0/3938367125 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fee80056400 0x7fee800588c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7feea4147bc0 tx=0x7fee94006ea0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.287 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.271+0000 7feea91ec700 1 --2- 172.21.15.203:0/3938367125 >> [v2:172.21.15.91:6808/18370,v1:172.21.15.91:6809/18370] conn(0x7fee8005d8c0 0x7fee8005fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.287 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.271+0000 7feea91ec700 1 -- 172.21.15.203:0/3938367125 >> [v2:172.21.15.91:6808/18370,v1:172.21.15.91:6809/18370] conn(0x7fee8005d8c0 msgr2=0x7fee8005fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2021-01-06T22:03:48.288 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.271+0000 7feea91ec700 1 --2- 172.21.15.203:0/3938367125 >> [v2:172.21.15.91:6808/18370,v1:172.21.15.91:6809/18370] conn(0x7fee8005d8c0 0x7fee8005fd20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.288 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.275+0000 7feea0ff9700 1 -- 172.21.15.203:0/3938367125 <== osd.4 v2:172.21.15.91:6808/18370 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22884 (crc 0 0 0) 0x7fee800603a0 con 0x7fee8005d8c0 2021-01-06T22:03:48.292 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.291+0000 7feeaac4f700 1 -- 172.21.15.203:0/3938367125 --> [v2:172.21.15.91:6808/18370,v1:172.21.15.91:6809/18370] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7feea40c3d90 con 0x7fee8005d8c0 2021-01-06T22:03:48.293 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.291+0000 7feea0ff9700 1 -- 172.21.15.203:0/3938367125 <== osd.4 v2:172.21.15.91:6808/18370 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7feea40c3d90 con 0x7fee8005d8c0 2021-01-06T22:03:48.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.291+0000 7feeaac4f700 1 -- 172.21.15.203:0/3938367125 >> [v2:172.21.15.91:6808/18370,v1:172.21.15.91:6809/18370] conn(0x7fee8005d8c0 msgr2=0x7fee8005fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.291+0000 7feeaac4f700 1 --2- 172.21.15.203:0/3938367125 >> [v2:172.21.15.91:6808/18370,v1:172.21.15.91:6809/18370] conn(0x7fee8005d8c0 0x7fee8005fd20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.291+0000 7feeaac4f700 1 -- 172.21.15.203:0/3938367125 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fee80056400 msgr2=0x7fee800588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.291+0000 7feeaac4f700 1 --2- 172.21.15.203:0/3938367125 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fee80056400 0x7fee800588c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7feea4147bc0 tx=0x7fee94006ea0).stop 2021-01-06T22:03:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.291+0000 7feeaac4f700 1 -- 172.21.15.203:0/3938367125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feea4064bc0 msgr2=0x7feea40c38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.291+0000 7feeaac4f700 1 --2- 172.21.15.203:0/3938367125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feea4064bc0 0x7feea40c38a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fee9000c6e0 tx=0x7fee90007c30).stop 2021-01-06T22:03:48.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.291+0000 7feea91ec700 1 -- 172.21.15.203:0/3938367125 reap_dead start 2021-01-06T22:03:48.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.291+0000 7feeaac4f700 1 -- 172.21.15.203:0/3938367125 shutdown_connections 2021-01-06T22:03:48.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.291+0000 7feeaac4f700 1 -- 172.21.15.203:0/3938367125 >> 172.21.15.203:0/3938367125 conn(0x7feea40602b0 msgr2=0x7feea4066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:48.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.291+0000 7feeaac4f700 1 -- 172.21.15.203:0/3938367125 shutdown_connections 2021-01-06T22:03:48.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.291+0000 7feeaac4f700 1 -- 172.21.15.203:0/3938367125 wait complete. 2021-01-06T22:03:48.308 INFO:teuthology.orchestra.run.smithi203.stdout:60129542150 2021-01-06T22:03:48.308 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2021-01-06T22:03:48.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.467+0000 7f6a90ae5700 1 Processor -- start 2021-01-06T22:03:48.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.467+0000 7f6a90ae5700 1 -- start start 2021-01-06T22:03:48.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a90ae5700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6a8c0c04c0 0x7f6a8c066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.478 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a90ae5700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8c0bb860 con 0x7f6a8c14ab60 2021-01-06T22:03:48.478 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a90ae5700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8c14b6e0 con 0x7f6a8c0c08a0 2021-01-06T22:03:48.480 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a90ae5700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a8c14b940 con 0x7f6a8c0c04c0 2021-01-06T22:03:48.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a89d9b700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f6a8c0c08a0 0x7f6a8c147cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47436/0 (socket says 172.21.15.203:47436) 2021-01-06T22:03:48.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a89d9b700 1 -- 172.21.15.203:0/1928639757 learned_addr learned my addr 172.21.15.203:0/1928639757 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:48.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a8a59c700 1 --2- 172.21.15.203:0/1928639757 >> v2:172.21.15.91:3300/0 conn(0x7f6a8c0c04c0 0x7f6a8c066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a8959a700 1 -- 172.21.15.203:0/1928639757 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509155503 0 0) 0x7f6a8c14b6e0 con 0x7f6a8c0c08a0 2021-01-06T22:03:48.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a8959a700 1 -- 172.21.15.203:0/1928639757 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a70003230 con 0x7f6a8c0c08a0 2021-01-06T22:03:48.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a8a59c700 1 -- 172.21.15.203:0/1928639757 >> v1:172.21.15.203:6789/0 conn(0x7f6a8c0c08a0 legacy=0x7f6a8c147cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a8a59c700 1 -- 172.21.15.203:0/1928639757 >> v1:172.21.15.91:6789/0 conn(0x7f6a8c14ab60 legacy=0x7f6a8c14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a8959a700 1 -- 172.21.15.203:0/1928639757 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058051461 0 0) 0x7f6a8c0bb860 con 0x7f6a8c14ab60 2021-01-06T22:03:48.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a8a59c700 1 -- 172.21.15.203:0/1928639757 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c0bba90 con 0x7f6a8c0c04c0 2021-01-06T22:03:48.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a8a59c700 1 --2- 172.21.15.203:0/1928639757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a8c0c04c0 0x7f6a8c066cd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f6a8000b980 tx=0x7f6a80006de0).ready entity=mon.0 client_cookie=3c76d43dc8f5d53e server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a8959a700 1 -- 172.21.15.203:0/1928639757 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800097b0 con 0x7f6a8c0c04c0 2021-01-06T22:03:48.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a8959a700 1 -- 172.21.15.203:0/1928639757 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a80009910 con 0x7f6a8c0c04c0 2021-01-06T22:03:48.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a8959a700 1 -- 172.21.15.203:0/1928639757 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a80009c90 con 0x7f6a8c0c04c0 2021-01-06T22:03:48.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a90ae5700 1 -- 172.21.15.203:0/1928639757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a8c0c04c0 msgr2=0x7f6a8c066cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a90ae5700 1 --2- 172.21.15.203:0/1928639757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a8c0c04c0 0x7f6a8c066cd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f6a8000b980 tx=0x7f6a80006de0).stop 2021-01-06T22:03:48.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a90ae5700 1 -- 172.21.15.203:0/1928639757 shutdown_connections 2021-01-06T22:03:48.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a90ae5700 1 --2- 172.21.15.203:0/1928639757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a8c0c04c0 0x7f6a8c066cd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a90ae5700 1 -- 172.21.15.203:0/1928639757 >> 172.21.15.203:0/1928639757 conn(0x7f6a8c056cd0 msgr2=0x7f6a8c0626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:48.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a90ae5700 1 -- 172.21.15.203:0/1928639757 shutdown_connections 2021-01-06T22:03:48.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.471+0000 7f6a90ae5700 1 -- 172.21.15.203:0/1928639757 wait complete. 2021-01-06T22:03:48.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a90ae5700 1 Processor -- start 2021-01-06T22:03:48.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a90ae5700 1 -- start start 2021-01-06T22:03:48.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a90ae5700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a8c0c08a0 0x7f6a8c14c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a90ae5700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a8c14ab60 0x7f6a8c0be650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a90ae5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a8c0beb40 0x7f6a8c12ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a90ae5700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c1476f0 con 0x7f6a8c0c08a0 2021-01-06T22:03:48.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a90ae5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c0be220 con 0x7f6a8c0beb40 2021-01-06T22:03:48.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a90ae5700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c14d5f0 con 0x7f6a8c14ab60 2021-01-06T22:03:48.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a8ad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a8c0beb40 0x7f6a8c12ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a8ad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a8c0beb40 0x7f6a8c12ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48852/0 (socket says 172.21.15.203:48852) 2021-01-06T22:03:48.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a8ad9d700 1 -- 172.21.15.203:0/3547762737 learned_addr learned my addr 172.21.15.203:0/3547762737 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:48.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a8a59c700 1 --2- 172.21.15.203:0/3547762737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a8c0c08a0 0x7f6a8c14c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a89d9b700 1 --2- 172.21.15.203:0/3547762737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a8c14ab60 0x7f6a8c0be650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a8ad9d700 1 -- 172.21.15.203:0/3547762737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a8c14ab60 msgr2=0x7f6a8c0be650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a8ad9d700 1 --2- 172.21.15.203:0/3547762737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a8c14ab60 0x7f6a8c0be650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a8ad9d700 1 -- 172.21.15.203:0/3547762737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a8c0c08a0 msgr2=0x7f6a8c14c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a8ad9d700 1 --2- 172.21.15.203:0/3547762737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a8c0c08a0 0x7f6a8c14c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a8ad9d700 1 -- 172.21.15.203:0/3547762737 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c14c8c0 con 0x7f6a8c0beb40 2021-01-06T22:03:48.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a8a59c700 1 --2- 172.21.15.203:0/3547762737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a8c0c08a0 0x7f6a8c14c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:48.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a8ad9d700 1 --2- 172.21.15.203:0/3547762737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a8c0beb40 0x7f6a8c12ba80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f6a8400c830 tx=0x7f6a8400cbf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a7a7fc700 1 -- 172.21.15.203:0/3547762737 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400a710 con 0x7f6a8c0beb40 2021-01-06T22:03:48.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a7a7fc700 1 -- 172.21.15.203:0/3547762737 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8400a870 con 0x7f6a8c0beb40 2021-01-06T22:03:48.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a90ae5700 1 -- 172.21.15.203:0/3547762737 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8c14d860 con 0x7f6a8c0beb40 2021-01-06T22:03:48.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a7a7fc700 1 -- 172.21.15.203:0/3547762737 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400ac30 con 0x7f6a8c0beb40 2021-01-06T22:03:48.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.475+0000 7f6a90ae5700 1 -- 172.21.15.203:0/3547762737 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8c14dda0 con 0x7f6a8c0beb40 2021-01-06T22:03:48.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.479+0000 7f6a90ae5700 1 -- 172.21.15.203:0/3547762737 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a8c0befa0 con 0x7f6a8c0beb40 2021-01-06T22:03:48.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.479+0000 7f6a7a7fc700 1 -- 172.21.15.203:0/3547762737 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f6a84010070 con 0x7f6a8c0beb40 2021-01-06T22:03:48.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.479+0000 7f6a7a7fc700 1 --2- 172.21.15.203:0/3547762737 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a680563d0 0x7f6a68058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.479+0000 7f6a7a7fc700 1 -- 172.21.15.203:0/3547762737 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6a8406f3e0 con 0x7f6a8c0beb40 2021-01-06T22:03:48.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.479+0000 7f6a7a7fc700 1 --2- 172.21.15.203:0/3547762737 >> [v2:172.21.15.91:6800/18372,v1:172.21.15.91:6801/18372] conn(0x7f6a6805d800 0x7f6a6805fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.479+0000 7f6a8a59c700 1 --2- 172.21.15.203:0/3547762737 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a680563d0 0x7f6a68058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.479+0000 7f6a7a7fc700 1 -- 172.21.15.203:0/3547762737 --> [v2:172.21.15.91:6800/18372,v1:172.21.15.91:6801/18372] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6a680602e0 con 0x7f6a6805d800 2021-01-06T22:03:48.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.479+0000 7f6a7a7fc700 1 -- 172.21.15.203:0/3547762737 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a84071a50 con 0x7f6a8c0beb40 2021-01-06T22:03:48.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.479+0000 7f6a89d9b700 1 --2- 172.21.15.203:0/3547762737 >> [v2:172.21.15.91:6800/18372,v1:172.21.15.91:6801/18372] conn(0x7f6a6805d800 0x7f6a6805fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.479+0000 7f6a8a59c700 1 --2- 172.21.15.203:0/3547762737 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a680563d0 0x7f6a68058890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f6a8c0bdf20 tx=0x7f6a80016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.479+0000 7f6a89d9b700 1 -- 172.21.15.203:0/3547762737 >> [v2:172.21.15.91:6800/18372,v1:172.21.15.91:6801/18372] conn(0x7f6a6805d800 msgr2=0x7f6a6805fc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2021-01-06T22:03:48.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.479+0000 7f6a89d9b700 1 --2- 172.21.15.203:0/3547762737 >> [v2:172.21.15.91:6800/18372,v1:172.21.15.91:6801/18372] conn(0x7f6a6805d800 0x7f6a6805fc60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.483+0000 7f6a7a7fc700 1 -- 172.21.15.203:0/3547762737 <== osd.5 v2:172.21.15.91:6800/18372 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22884 (crc 0 0 0) 0x7f6a680602e0 con 0x7f6a6805d800 2021-01-06T22:03:48.510 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.503+0000 7f6a90ae5700 1 -- 172.21.15.203:0/3547762737 --> [v2:172.21.15.91:6800/18372,v1:172.21.15.91:6801/18372] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6a8c14df30 con 0x7f6a6805d800 2021-01-06T22:03:48.510 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.503+0000 7f6a7a7fc700 1 -- 172.21.15.203:0/3547762737 <== osd.5 v2:172.21.15.91:6800/18372 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6a7c007090 con 0x7f6a6805d800 2021-01-06T22:03:48.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.503+0000 7f6a90ae5700 1 -- 172.21.15.203:0/3547762737 >> [v2:172.21.15.91:6800/18372,v1:172.21.15.91:6801/18372] conn(0x7f6a6805d800 msgr2=0x7f6a6805fc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.503+0000 7f6a90ae5700 1 --2- 172.21.15.203:0/3547762737 >> [v2:172.21.15.91:6800/18372,v1:172.21.15.91:6801/18372] conn(0x7f6a6805d800 0x7f6a6805fc60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.503+0000 7f6a90ae5700 1 -- 172.21.15.203:0/3547762737 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a680563d0 msgr2=0x7f6a68058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.503+0000 7f6a90ae5700 1 --2- 172.21.15.203:0/3547762737 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a680563d0 0x7f6a68058890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f6a8c0bdf20 tx=0x7f6a80016000).stop 2021-01-06T22:03:48.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.503+0000 7f6a90ae5700 1 -- 172.21.15.203:0/3547762737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a8c0beb40 msgr2=0x7f6a8c12ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.503+0000 7f6a90ae5700 1 --2- 172.21.15.203:0/3547762737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a8c0beb40 0x7f6a8c12ba80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f6a8400c830 tx=0x7f6a8400cbf0).stop 2021-01-06T22:03:48.513 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.503+0000 7f6a8ad9d700 1 -- 172.21.15.203:0/3547762737 reap_dead start 2021-01-06T22:03:48.513 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.503+0000 7f6a90ae5700 1 -- 172.21.15.203:0/3547762737 shutdown_connections 2021-01-06T22:03:48.514 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.503+0000 7f6a90ae5700 1 -- 172.21.15.203:0/3547762737 >> 172.21.15.203:0/3547762737 conn(0x7f6a8c056cd0 msgr2=0x7f6a8c0602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:48.514 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.503+0000 7f6a90ae5700 1 -- 172.21.15.203:0/3547762737 shutdown_connections 2021-01-06T22:03:48.514 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.503+0000 7f6a90ae5700 1 -- 172.21.15.203:0/3547762737 wait complete. 2021-01-06T22:03:48.520 INFO:teuthology.orchestra.run.smithi203.stdout:55834574854 2021-01-06T22:03:48.521 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2021-01-06T22:03:48.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc92c9700 1 Processor -- start 2021-01-06T22:03:48.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc92c9700 1 -- start start 2021-01-06T22:03:48.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc92c9700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbdc40c4890 0x7fbdc40c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc92c9700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbdc40c4c70 0x7fbdc414dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc92c9700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdc414e3b0 0x7fbdc414ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc92c9700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc414e830 con 0x7fbdc40c4c70 2021-01-06T22:03:48.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc92c9700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc414ead0 con 0x7fbdc40c4890 2021-01-06T22:03:48.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc92c9700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc414ed70 con 0x7fbdc414e3b0 2021-01-06T22:03:48.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc2ffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbdc40c4890 0x7fbdc40c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc2ffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbdc40c4890 0x7fbdc40c1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48858/0 (socket says 172.21.15.203:48858) 2021-01-06T22:03:48.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc2ffd700 1 -- 172.21.15.203:0/2550044325 learned_addr learned my addr 172.21.15.203:0/2550044325 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:48.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc37fe700 1 --2- 172.21.15.203:0/2550044325 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdc414e3b0 0x7fbdc414ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc27fc700 1 --2- 172.21.15.203:0/2550044325 >> v2:172.21.15.91:3300/0 conn(0x7fbdc40c4c70 0x7fbdc414dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc2ffd700 1 -- 172.21.15.203:0/2550044325 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdc414e3b0 msgr2=0x7fbdc414ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:48.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc2ffd700 1 --2- 172.21.15.203:0/2550044325 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdc414e3b0 0x7fbdc414ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc2ffd700 1 -- 172.21.15.203:0/2550044325 >> v2:172.21.15.91:3300/0 conn(0x7fbdc40c4c70 msgr2=0x7fbdc414dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:48.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc2ffd700 1 --2- 172.21.15.203:0/2550044325 >> v2:172.21.15.91:3300/0 conn(0x7fbdc40c4c70 0x7fbdc414dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc2ffd700 1 -- 172.21.15.203:0/2550044325 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc40c1040 con 0x7fbdc40c4890 2021-01-06T22:03:48.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc37fe700 1 --2- 172.21.15.203:0/2550044325 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdc414e3b0 0x7fbdc414ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:48.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc2ffd700 1 --2- 172.21.15.203:0/2550044325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc40c4890 0x7fbdc40c1a20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fbdc4065f50 tx=0x7fbdb4006e40).ready entity=mon.1 client_cookie=b7b2e7e4a66400d1 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc1ffb700 1 -- 172.21.15.203:0/2550044325 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4006030 con 0x7fbdc40c4890 2021-01-06T22:03:48.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc1ffb700 1 -- 172.21.15.203:0/2550044325 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb4006190 con 0x7fbdc40c4890 2021-01-06T22:03:48.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc1ffb700 1 -- 172.21.15.203:0/2550044325 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4006510 con 0x7fbdc40c4890 2021-01-06T22:03:48.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc92c9700 1 -- 172.21.15.203:0/2550044325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc40c4890 msgr2=0x7fbdc40c1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc92c9700 1 --2- 172.21.15.203:0/2550044325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc40c4890 0x7fbdc40c1a20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fbdc4065f50 tx=0x7fbdb4006e40).stop 2021-01-06T22:03:48.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc92c9700 1 -- 172.21.15.203:0/2550044325 shutdown_connections 2021-01-06T22:03:48.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc92c9700 1 --2- 172.21.15.203:0/2550044325 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdc414e3b0 0x7fbdc414ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc92c9700 1 --2- 172.21.15.203:0/2550044325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc40c4890 0x7fbdc40c1a20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc92c9700 1 --2- 172.21.15.203:0/2550044325 >> v2:172.21.15.91:3300/0 conn(0x7fbdc40c4c70 0x7fbdc414dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.687+0000 7fbdc92c9700 1 -- 172.21.15.203:0/2550044325 >> 172.21.15.203:0/2550044325 conn(0x7fbdc4061a30 msgr2=0x7fbdc4061e10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:48.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.691+0000 7fbdc92c9700 1 -- 172.21.15.203:0/2550044325 shutdown_connections 2021-01-06T22:03:48.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.691+0000 7fbdc92c9700 1 -- 172.21.15.203:0/2550044325 wait complete. 2021-01-06T22:03:48.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.691+0000 7fbdc92c9700 1 Processor -- start 2021-01-06T22:03:48.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.691+0000 7fbdc92c9700 1 -- start start 2021-01-06T22:03:48.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.691+0000 7fbdc92c9700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdc40c4890 0x7fbdc40bb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.691+0000 7fbdc92c9700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdc40c4c70 0x7fbdc40bccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.691+0000 7fbdc92c9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc414e3b0 0x7fbdc41515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.691+0000 7fbdc92c9700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc41511c0 con 0x7fbdc40c4c70 2021-01-06T22:03:48.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.691+0000 7fbdc92c9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc415e470 con 0x7fbdc414e3b0 2021-01-06T22:03:48.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.691+0000 7fbdc92c9700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc415e710 con 0x7fbdc40c4890 2021-01-06T22:03:48.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.691+0000 7fbdc37fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc414e3b0 0x7fbdc41515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.691+0000 7fbdc37fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc414e3b0 0x7fbdc41515f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48868/0 (socket says 172.21.15.203:48868) 2021-01-06T22:03:48.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.691+0000 7fbdc37fe700 1 -- 172.21.15.203:0/4113768980 learned_addr learned my addr 172.21.15.203:0/4113768980 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:48.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.691+0000 7fbdc2ffd700 1 --2- 172.21.15.203:0/4113768980 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdc40c4890 0x7fbdc40bb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.691+0000 7fbdc27fc700 1 --2- 172.21.15.203:0/4113768980 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdc40c4c70 0x7fbdc40bccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.695+0000 7fbdc37fe700 1 -- 172.21.15.203:0/4113768980 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdc40c4890 msgr2=0x7fbdc40bb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.695+0000 7fbdc37fe700 1 --2- 172.21.15.203:0/4113768980 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdc40c4890 0x7fbdc40bb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.695+0000 7fbdc37fe700 1 -- 172.21.15.203:0/4113768980 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdc40c4c70 msgr2=0x7fbdc40bccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.695+0000 7fbdc37fe700 1 --2- 172.21.15.203:0/4113768980 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdc40c4c70 0x7fbdc40bccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.713 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.695+0000 7fbdc37fe700 1 -- 172.21.15.203:0/4113768980 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc4151de0 con 0x7fbdc414e3b0 2021-01-06T22:03:48.713 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.695+0000 7fbdc37fe700 1 --2- 172.21.15.203:0/4113768980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc414e3b0 0x7fbdc41515f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fbdbc00db30 tx=0x7fbdbc008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.695+0000 7fbdaaffd700 1 -- 172.21.15.203:0/4113768980 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc00b870 con 0x7fbdc414e3b0 2021-01-06T22:03:48.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.695+0000 7fbdaaffd700 1 -- 172.21.15.203:0/4113768980 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdbc00b9d0 con 0x7fbdc414e3b0 2021-01-06T22:03:48.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.695+0000 7fbdc92c9700 1 -- 172.21.15.203:0/4113768980 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc4152080 con 0x7fbdc414e3b0 2021-01-06T22:03:48.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.695+0000 7fbdaaffd700 1 -- 172.21.15.203:0/4113768980 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc00bd70 con 0x7fbdc414e3b0 2021-01-06T22:03:48.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.695+0000 7fbdc92c9700 1 -- 172.21.15.203:0/4113768980 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc40bf7c0 con 0x7fbdc414e3b0 2021-01-06T22:03:48.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.695+0000 7fbdc92c9700 1 -- 172.21.15.203:0/4113768980 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbdc40be460 con 0x7fbdc414e3b0 2021-01-06T22:03:48.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.695+0000 7fbdaaffd700 1 -- 172.21.15.203:0/4113768980 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fbdbc012630 con 0x7fbdc414e3b0 2021-01-06T22:03:48.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.695+0000 7fbdaaffd700 1 --2- 172.21.15.203:0/4113768980 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbda4056400 0x7fbda40588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.695+0000 7fbdaaffd700 1 -- 172.21.15.203:0/4113768980 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbdbc070500 con 0x7fbdc414e3b0 2021-01-06T22:03:48.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.699+0000 7fbdc2ffd700 1 --2- 172.21.15.203:0/4113768980 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbda4056400 0x7fbda40588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.699+0000 7fbdaaffd700 1 --2- 172.21.15.203:0/4113768980 >> [v2:172.21.15.91:6815/18371,v1:172.21.15.91:6817/18371] conn(0x7fbda405d8c0 0x7fbda405fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.699+0000 7fbdaaffd700 1 -- 172.21.15.203:0/4113768980 --> [v2:172.21.15.91:6815/18371,v1:172.21.15.91:6817/18371] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbda40603a0 con 0x7fbda405d8c0 2021-01-06T22:03:48.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.699+0000 7fbdaaffd700 1 -- 172.21.15.203:0/4113768980 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdbc070900 con 0x7fbdc414e3b0 2021-01-06T22:03:48.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.699+0000 7fbdc2ffd700 1 --2- 172.21.15.203:0/4113768980 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbda4056400 0x7fbda40588c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fbdb4006760 tx=0x7fbdb4015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.699+0000 7fbdc27fc700 1 --2- 172.21.15.203:0/4113768980 >> [v2:172.21.15.91:6815/18371,v1:172.21.15.91:6817/18371] conn(0x7fbda405d8c0 0x7fbda405fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.699+0000 7fbdc27fc700 1 -- 172.21.15.203:0/4113768980 >> [v2:172.21.15.91:6815/18371,v1:172.21.15.91:6817/18371] conn(0x7fbda405d8c0 msgr2=0x7fbda405fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2021-01-06T22:03:48.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.699+0000 7fbdc27fc700 1 --2- 172.21.15.203:0/4113768980 >> [v2:172.21.15.91:6815/18371,v1:172.21.15.91:6817/18371] conn(0x7fbda405d8c0 0x7fbda405fd20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.699+0000 7fbdaaffd700 1 -- 172.21.15.203:0/4113768980 <== osd.6 v2:172.21.15.91:6815/18371 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22884 (crc 0 0 0) 0x7fbda40603a0 con 0x7fbda405d8c0 2021-01-06T22:03:48.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.719+0000 7fbdc92c9700 1 -- 172.21.15.203:0/4113768980 --> [v2:172.21.15.91:6815/18371,v1:172.21.15.91:6817/18371] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbdc405cfe0 con 0x7fbda405d8c0 2021-01-06T22:03:48.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.723+0000 7fbdaaffd700 1 -- 172.21.15.203:0/4113768980 <== osd.6 v2:172.21.15.91:6815/18371 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbdc405cfe0 con 0x7fbda405d8c0 2021-01-06T22:03:48.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.723+0000 7fbdc92c9700 1 -- 172.21.15.203:0/4113768980 >> [v2:172.21.15.91:6815/18371,v1:172.21.15.91:6817/18371] conn(0x7fbda405d8c0 msgr2=0x7fbda405fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.723+0000 7fbdc92c9700 1 --2- 172.21.15.203:0/4113768980 >> [v2:172.21.15.91:6815/18371,v1:172.21.15.91:6817/18371] conn(0x7fbda405d8c0 0x7fbda405fd20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.723+0000 7fbdc92c9700 1 -- 172.21.15.203:0/4113768980 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbda4056400 msgr2=0x7fbda40588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.723+0000 7fbdc92c9700 1 --2- 172.21.15.203:0/4113768980 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbda4056400 0x7fbda40588c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fbdb4006760 tx=0x7fbdb4015000).stop 2021-01-06T22:03:48.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.723+0000 7fbdc92c9700 1 -- 172.21.15.203:0/4113768980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc414e3b0 msgr2=0x7fbdc41515f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.723+0000 7fbdc92c9700 1 --2- 172.21.15.203:0/4113768980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc414e3b0 0x7fbdc41515f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fbdbc00db30 tx=0x7fbdbc008040).stop 2021-01-06T22:03:48.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.723+0000 7fbdc37fe700 1 -- 172.21.15.203:0/4113768980 reap_dead start 2021-01-06T22:03:48.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.723+0000 7fbdc92c9700 1 -- 172.21.15.203:0/4113768980 shutdown_connections 2021-01-06T22:03:48.727 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.723+0000 7fbdc92c9700 1 -- 172.21.15.203:0/4113768980 >> 172.21.15.203:0/4113768980 conn(0x7fbdc4061a30 msgr2=0x7fbdc4061e10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:48.727 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.723+0000 7fbdc92c9700 1 -- 172.21.15.203:0/4113768980 shutdown_connections 2021-01-06T22:03:48.727 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.723+0000 7fbdc92c9700 1 -- 172.21.15.203:0/4113768980 wait complete. 2021-01-06T22:03:48.738 INFO:teuthology.orchestra.run.smithi203.stdout:60129542150 2021-01-06T22:03:48.739 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2021-01-06T22:03:48.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.895+0000 7f0c1db63700 1 Processor -- start 2021-01-06T22:03:48.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.895+0000 7f0c1db63700 1 -- start start 2021-01-06T22:03:48.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.895+0000 7f0c1db63700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0c18063c50 0x7f0c1814f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.895+0000 7f0c1db63700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0c18156310 0x7f0c1814f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.895+0000 7f0c1db63700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c180ba9e0 con 0x7f0c18068370 2021-01-06T22:03:48.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.895+0000 7f0c1db63700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1814fda0 con 0x7f0c18063c50 2021-01-06T22:03:48.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.895+0000 7f0c1db63700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c180bac80 con 0x7f0c18156310 2021-01-06T22:03:48.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.895+0000 7f0c1d362700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0c18156310 0x7f0c1814f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.895+0000 7f0c1d362700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0c18156310 0x7f0c1814f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48878/0 (socket says 172.21.15.203:48878) 2021-01-06T22:03:48.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.895+0000 7f0c1d362700 1 -- 172.21.15.203:0/553097156 learned_addr learned my addr 172.21.15.203:0/553097156 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:48.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.895+0000 7f0c1cb61700 1 --2- 172.21.15.203:0/553097156 >> v2:172.21.15.91:3300/0 conn(0x7f0c18063c50 0x7f0c1814f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c1d362700 1 -- 172.21.15.203:0/553097156 >> v2:172.21.15.91:3300/0 conn(0x7f0c18063c50 msgr2=0x7f0c1814f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:48.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c1d362700 1 --2- 172.21.15.203:0/553097156 >> v2:172.21.15.91:3300/0 conn(0x7f0c18063c50 0x7f0c1814f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c1d362700 1 -- 172.21.15.203:0/553097156 >> v1:172.21.15.91:6789/0 conn(0x7f0c18068370 legacy=0x7f0c18155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:48.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c1d362700 1 -- 172.21.15.203:0/553097156 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c180baf20 con 0x7f0c18156310 2021-01-06T22:03:48.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c1d362700 1 --2- 172.21.15.203:0/553097156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c18156310 0x7f0c1814f8b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f0c18068290 tx=0x7f0c10009290).ready entity=mon.1 client_cookie=8866ba007ee25d1a server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c177fe700 1 -- 172.21.15.203:0/553097156 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1000bb80 con 0x7f0c18156310 2021-01-06T22:03:48.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c177fe700 1 -- 172.21.15.203:0/553097156 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1000bce0 con 0x7f0c18156310 2021-01-06T22:03:48.905 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c177fe700 1 -- 172.21.15.203:0/553097156 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10011610 con 0x7f0c18156310 2021-01-06T22:03:48.910 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c1db63700 1 -- 172.21.15.203:0/553097156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c18156310 msgr2=0x7f0c1814f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.910 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c1db63700 1 --2- 172.21.15.203:0/553097156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c18156310 0x7f0c1814f8b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f0c18068290 tx=0x7f0c10009290).stop 2021-01-06T22:03:48.911 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c1db63700 1 -- 172.21.15.203:0/553097156 shutdown_connections 2021-01-06T22:03:48.911 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c1db63700 1 --2- 172.21.15.203:0/553097156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c18156310 0x7f0c1814f8b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.912 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c1db63700 1 --2- 172.21.15.203:0/553097156 >> v2:172.21.15.91:3300/0 conn(0x7f0c18063c50 0x7f0c1814f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.912 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c1db63700 1 -- 172.21.15.203:0/553097156 >> 172.21.15.203:0/553097156 conn(0x7f0c180631f0 msgr2=0x7f0c180c0040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:48.912 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c1db63700 1 -- 172.21.15.203:0/553097156 shutdown_connections 2021-01-06T22:03:48.912 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c1db63700 1 -- 172.21.15.203:0/553097156 wait complete. 2021-01-06T22:03:48.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c1db63700 1 Processor -- start 2021-01-06T22:03:48.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.899+0000 7f0c1db63700 1 -- start start 2021-01-06T22:03:48.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1db63700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0c18063c50 0x7f0c180bde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1db63700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c18068370 0x7f0c181658a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1db63700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c18156310 0x7f0c181547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1db63700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180a04e0 con 0x7f0c18068370 2021-01-06T22:03:48.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1db63700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180a0780 con 0x7f0c18156310 2021-01-06T22:03:48.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1db63700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180a0a20 con 0x7f0c18063c50 2021-01-06T22:03:48.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1d362700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c18156310 0x7f0c181547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1d362700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c18156310 0x7f0c181547e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48884/0 (socket says 172.21.15.203:48884) 2021-01-06T22:03:48.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1d362700 1 -- 172.21.15.203:0/399565203 learned_addr learned my addr 172.21.15.203:0/399565203 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:48.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1cb61700 1 --2- 172.21.15.203:0/399565203 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0c18063c50 0x7f0c180bde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c17fff700 1 --2- 172.21.15.203:0/399565203 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c18068370 0x7f0c181658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1d362700 1 -- 172.21.15.203:0/399565203 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0c18063c50 msgr2=0x7f0c180bde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1d362700 1 --2- 172.21.15.203:0/399565203 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0c18063c50 0x7f0c180bde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1d362700 1 -- 172.21.15.203:0/399565203 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c18068370 msgr2=0x7f0c181658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.918 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1d362700 1 --2- 172.21.15.203:0/399565203 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c18068370 0x7f0c181658a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.918 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1d362700 1 -- 172.21.15.203:0/399565203 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c18165160 con 0x7f0c18156310 2021-01-06T22:03:48.918 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c17fff700 1 --2- 172.21.15.203:0/399565203 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c18068370 0x7f0c181658a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:03:48.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1d362700 1 --2- 172.21.15.203:0/399565203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c18156310 0x7f0c181547e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f0c1814fee0 tx=0x7f0c10017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c14ff9700 1 -- 172.21.15.203:0/399565203 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10011e70 con 0x7f0c18156310 2021-01-06T22:03:48.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c14ff9700 1 -- 172.21.15.203:0/399565203 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1000be50 con 0x7f0c18156310 2021-01-06T22:03:48.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1db63700 1 -- 172.21.15.203:0/399565203 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c18165400 con 0x7f0c18156310 2021-01-06T22:03:48.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c14ff9700 1 -- 172.21.15.203:0/399565203 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10010df0 con 0x7f0c18156310 2021-01-06T22:03:48.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1db63700 1 -- 172.21.15.203:0/399565203 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c18066bb0 con 0x7f0c18156310 2021-01-06T22:03:48.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.903+0000 7f0c1db63700 1 -- 172.21.15.203:0/399565203 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c180a0be0 con 0x7f0c18156310 2021-01-06T22:03:48.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.907+0000 7f0c14ff9700 1 -- 172.21.15.203:0/399565203 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f0c1002a070 con 0x7f0c18156310 2021-01-06T22:03:48.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.907+0000 7f0c14ff9700 1 --2- 172.21.15.203:0/399565203 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0bf8056400 0x7f0bf80588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.907+0000 7f0c14ff9700 1 -- 172.21.15.203:0/399565203 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0c10071390 con 0x7f0c18156310 2021-01-06T22:03:48.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.907+0000 7f0c1cb61700 1 --2- 172.21.15.203:0/399565203 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0bf8056400 0x7f0bf80588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.907+0000 7f0c14ff9700 1 --2- 172.21.15.203:0/399565203 >> [v2:172.21.15.91:6824/18373,v1:172.21.15.91:6825/18373] conn(0x7f0bf805d8c0 0x7f0bf805fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:48.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.907+0000 7f0c14ff9700 1 -- 172.21.15.203:0/399565203 --> [v2:172.21.15.91:6824/18373,v1:172.21.15.91:6825/18373] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0bf80603a0 con 0x7f0bf805d8c0 2021-01-06T22:03:48.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.907+0000 7f0c14ff9700 1 -- 172.21.15.203:0/399565203 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c100738f0 con 0x7f0c18156310 2021-01-06T22:03:48.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.907+0000 7f0c17fff700 1 --2- 172.21.15.203:0/399565203 >> [v2:172.21.15.91:6824/18373,v1:172.21.15.91:6825/18373] conn(0x7f0bf805d8c0 0x7f0bf805fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:48.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.907+0000 7f0c1cb61700 1 --2- 172.21.15.203:0/399565203 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0bf8056400 0x7f0bf80588c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f0c08006fd0 tx=0x7f0c08008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.907+0000 7f0c17fff700 1 -- 172.21.15.203:0/399565203 >> [v2:172.21.15.91:6824/18373,v1:172.21.15.91:6825/18373] conn(0x7f0bf805d8c0 msgr2=0x7f0bf805fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2021-01-06T22:03:48.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.907+0000 7f0c17fff700 1 --2- 172.21.15.203:0/399565203 >> [v2:172.21.15.91:6824/18373,v1:172.21.15.91:6825/18373] conn(0x7f0bf805d8c0 0x7f0bf805fd20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:48.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.911+0000 7f0c14ff9700 1 -- 172.21.15.203:0/399565203 <== osd.7 v2:172.21.15.91:6824/18373 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22884 (crc 0 0 0) 0x7f0bf80603a0 con 0x7f0bf805d8c0 2021-01-06T22:03:48.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.931+0000 7f0c1db63700 1 -- 172.21.15.203:0/399565203 --> [v2:172.21.15.91:6824/18373,v1:172.21.15.91:6825/18373] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0c180be940 con 0x7f0bf805d8c0 2021-01-06T22:03:48.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.931+0000 7f0c14ff9700 1 -- 172.21.15.203:0/399565203 <== osd.7 v2:172.21.15.91:6824/18373 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0c180be940 con 0x7f0bf805d8c0 2021-01-06T22:03:48.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.931+0000 7f0c1db63700 1 -- 172.21.15.203:0/399565203 >> [v2:172.21.15.91:6824/18373,v1:172.21.15.91:6825/18373] conn(0x7f0bf805d8c0 msgr2=0x7f0bf805fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.931+0000 7f0c1db63700 1 --2- 172.21.15.203:0/399565203 >> [v2:172.21.15.91:6824/18373,v1:172.21.15.91:6825/18373] conn(0x7f0bf805d8c0 0x7f0bf805fd20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:48.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.931+0000 7f0c1db63700 1 -- 172.21.15.203:0/399565203 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0bf8056400 msgr2=0x7f0bf80588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.931+0000 7f0c1db63700 1 --2- 172.21.15.203:0/399565203 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0bf8056400 0x7f0bf80588c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f0c08006fd0 tx=0x7f0c08008040).stop 2021-01-06T22:03:48.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.931+0000 7f0c1db63700 1 -- 172.21.15.203:0/399565203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c18156310 msgr2=0x7f0c181547e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:48.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.931+0000 7f0c1db63700 1 --2- 172.21.15.203:0/399565203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c18156310 0x7f0c181547e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f0c1814fee0 tx=0x7f0c10017000).stop 2021-01-06T22:03:48.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.931+0000 7f0c1d362700 1 -- 172.21.15.203:0/399565203 reap_dead start 2021-01-06T22:03:48.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.931+0000 7f0c1db63700 1 -- 172.21.15.203:0/399565203 shutdown_connections 2021-01-06T22:03:48.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.931+0000 7f0c1db63700 1 -- 172.21.15.203:0/399565203 >> 172.21.15.203:0/399565203 conn(0x7f0c180631f0 msgr2=0x7f0c18155b20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:48.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.931+0000 7f0c1db63700 1 -- 172.21.15.203:0/399565203 shutdown_connections 2021-01-06T22:03:48.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:48.931+0000 7f0c1db63700 1 -- 172.21.15.203:0/399565203 wait complete. 2021-01-06T22:03:48.950 INFO:teuthology.orchestra.run.smithi203.stdout:60129542150 2021-01-06T22:03:48.950 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2021-01-06T22:03:49.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd9a1217700 1 Processor -- start 2021-01-06T22:03:49.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd9a1217700 1 -- start start 2021-01-06T22:03:49.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd9a1217700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd99c0bf120 0x7fd99c0668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:49.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd9a1217700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd99c066da0 0x7fd99c064d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:49.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd9a1217700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99c14be20 con 0x7fd99c0bf500 2021-01-06T22:03:49.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd9a1217700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd99c14c0c0 con 0x7fd99c066da0 2021-01-06T22:03:49.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd9a1217700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c14c360 con 0x7fd99c0bf120 2021-01-06T22:03:49.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd99a59c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd99c066da0 0x7fd99c064d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:49.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd99ad9d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd99c0bf120 0x7fd99c0668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:49.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd99b59e700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fd99c0bf500 0x7fd99c148ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47486/0 (socket says 172.21.15.203:47486) 2021-01-06T22:03:49.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd99b59e700 1 -- 172.21.15.203:0/1282116752 learned_addr learned my addr 172.21.15.203:0/1282116752 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:49.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd999d9b700 1 -- 172.21.15.203:0/1282116752 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843953826 0 0) 0x7fd99c14be20 con 0x7fd99c0bf500 2021-01-06T22:03:49.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd999d9b700 1 -- 172.21.15.203:0/1282116752 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd988003230 con 0x7fd99c0bf500 2021-01-06T22:03:49.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd99ad9d700 1 -- 172.21.15.203:0/1282116752 >> v2:172.21.15.203:3300/0 conn(0x7fd99c066da0 msgr2=0x7fd99c064d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:49.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd99ad9d700 1 --2- 172.21.15.203:0/1282116752 >> v2:172.21.15.203:3300/0 conn(0x7fd99c066da0 0x7fd99c064d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:49.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd99ad9d700 1 -- 172.21.15.203:0/1282116752 >> v1:172.21.15.203:6789/0 conn(0x7fd99c0bf500 legacy=0x7fd99c148ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:49.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd99ad9d700 1 -- 172.21.15.203:0/1282116752 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c14c600 con 0x7fd99c0bf120 2021-01-06T22:03:49.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd99ad9d700 1 --2- 172.21.15.203:0/1282116752 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd99c0bf120 0x7fd99c0668b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fd98c00b980 tx=0x7fd98c006de0).ready entity=mon.2 client_cookie=a8de47880eddffb8 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:49.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd999d9b700 1 -- 172.21.15.203:0/1282116752 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0096d0 con 0x7fd99c0bf120 2021-01-06T22:03:49.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd999d9b700 1 -- 172.21.15.203:0/1282116752 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c009830 con 0x7fd99c0bf120 2021-01-06T22:03:49.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd999d9b700 1 -- 172.21.15.203:0/1282116752 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c009bb0 con 0x7fd99c0bf120 2021-01-06T22:03:49.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd9a1217700 1 -- 172.21.15.203:0/1282116752 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd99c0bf120 msgr2=0x7fd99c0668b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:49.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.091+0000 7fd9a1217700 1 --2- 172.21.15.203:0/1282116752 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd99c0bf120 0x7fd99c0668b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fd98c00b980 tx=0x7fd98c006de0).stop 2021-01-06T22:03:49.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd9a1217700 1 -- 172.21.15.203:0/1282116752 shutdown_connections 2021-01-06T22:03:49.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd9a1217700 1 --2- 172.21.15.203:0/1282116752 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd99c0bf120 0x7fd99c0668b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:49.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd9a1217700 1 --2- 172.21.15.203:0/1282116752 >> v2:172.21.15.203:3300/0 conn(0x7fd99c066da0 0x7fd99c064d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:49.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd9a1217700 1 -- 172.21.15.203:0/1282116752 >> 172.21.15.203:0/1282116752 conn(0x7fd99c060d80 msgr2=0x7fd99c0661a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:49.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd9a1217700 1 -- 172.21.15.203:0/1282116752 shutdown_connections 2021-01-06T22:03:49.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd9a1217700 1 -- 172.21.15.203:0/1282116752 wait complete. 2021-01-06T22:03:49.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd9a1217700 1 Processor -- start 2021-01-06T22:03:49.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd9a1217700 1 -- start start 2021-01-06T22:03:49.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd9a1217700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd99c066da0 0x7fd99c15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:49.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd9a1217700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd99c0bf120 0x7fd99c15bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:49.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd9a1217700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd99c0bf500 0x7fd99c0ba540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:49.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd9a1217700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c0bae40 con 0x7fd99c0bf500 2021-01-06T22:03:49.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd9a1217700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c0bb0e0 con 0x7fd99c0bf120 2021-01-06T22:03:49.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd9a1217700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c0bb380 con 0x7fd99c066da0 2021-01-06T22:03:49.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd99a59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd99c0bf120 0x7fd99c15bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:49.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd99a59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd99c0bf120 0x7fd99c15bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48898/0 (socket says 172.21.15.203:48898) 2021-01-06T22:03:49.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd99a59c700 1 -- 172.21.15.203:0/4097688726 learned_addr learned my addr 172.21.15.203:0/4097688726 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:49.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd99ad9d700 1 --2- 172.21.15.203:0/4097688726 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd99c066da0 0x7fd99c15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:49.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd99b59e700 1 --2- 172.21.15.203:0/4097688726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd99c0bf500 0x7fd99c0ba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:49.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd99a59c700 1 -- 172.21.15.203:0/4097688726 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd99c066da0 msgr2=0x7fd99c15b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:49.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd99a59c700 1 --2- 172.21.15.203:0/4097688726 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd99c066da0 0x7fd99c15b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:49.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd99a59c700 1 -- 172.21.15.203:0/4097688726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd99c0bf500 msgr2=0x7fd99c0ba540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:49.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd99a59c700 1 --2- 172.21.15.203:0/4097688726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd99c0bf500 0x7fd99c0ba540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:49.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd99a59c700 1 -- 172.21.15.203:0/4097688726 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c142120 con 0x7fd99c0bf120 2021-01-06T22:03:49.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.095+0000 7fd99b59e700 1 --2- 172.21.15.203:0/4097688726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd99c0bf500 0x7fd99c0ba540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:49.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.099+0000 7fd99a59c700 1 --2- 172.21.15.203:0/4097688726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd99c0bf120 0x7fd99c15bbe0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fd99000eba0 tx=0x7fd990006de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:49.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.099+0000 7fd982ffd700 1 -- 172.21.15.203:0/4097688726 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99000c820 con 0x7fd99c0bf120 2021-01-06T22:03:49.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.099+0000 7fd982ffd700 1 -- 172.21.15.203:0/4097688726 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99000c980 con 0x7fd99c0bf120 2021-01-06T22:03:49.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.099+0000 7fd9a1217700 1 -- 172.21.15.203:0/4097688726 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99c1423c0 con 0x7fd99c0bf120 2021-01-06T22:03:49.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.099+0000 7fd982ffd700 1 -- 172.21.15.203:0/4097688726 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99000cd00 con 0x7fd99c0bf120 2021-01-06T22:03:49.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.099+0000 7fd9a1217700 1 -- 172.21.15.203:0/4097688726 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99c15c0d0 con 0x7fd99c0bf120 2021-01-06T22:03:49.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.099+0000 7fd982ffd700 1 -- 172.21.15.203:0/4097688726 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fd99001a030 con 0x7fd99c0bf120 2021-01-06T22:03:49.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.099+0000 7fd9a1217700 1 -- 172.21.15.203:0/4097688726 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd99c0bb540 con 0x7fd99c0bf120 2021-01-06T22:03:49.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.099+0000 7fd982ffd700 1 --2- 172.21.15.203:0/4097688726 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd97c0563d0 0x7fd97c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:49.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.099+0000 7fd99ad9d700 1 --2- 172.21.15.203:0/4097688726 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd97c0563d0 0x7fd97c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:49.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.099+0000 7fd982ffd700 1 -- 172.21.15.203:0/4097688726 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd99006f0f0 con 0x7fd99c0bf120 2021-01-06T22:03:49.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.103+0000 7fd99ad9d700 1 --2- 172.21.15.203:0/4097688726 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd97c0563d0 0x7fd97c058890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fd98c00b130 tx=0x7fd98c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:49.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.107+0000 7fd982ffd700 1 -- 172.21.15.203:0/4097688726 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fd990044910 con 0x7fd99c0bf120 2021-01-06T22:03:49.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.259+0000 7fd9a1217700 1 -- 172.21.15.203:0/4097688726 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd99c142550 con 0x7fd99c0bf120 2021-01-06T22:03:49.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.259+0000 7fd982ffd700 1 -- 172.21.15.203:0/4097688726 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd990047f30 con 0x7fd99c0bf120 2021-01-06T22:03:49.263 INFO:teuthology.orchestra.run.smithi203.stdout:55834574853 2021-01-06T22:03:49.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.263+0000 7fd9a1217700 1 -- 172.21.15.203:0/4097688726 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd97c0563d0 msgr2=0x7fd97c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:49.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.263+0000 7fd9a1217700 1 --2- 172.21.15.203:0/4097688726 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd97c0563d0 0x7fd97c058890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fd98c00b130 tx=0x7fd98c016000).stop 2021-01-06T22:03:49.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.263+0000 7fd9a1217700 1 -- 172.21.15.203:0/4097688726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd99c0bf120 msgr2=0x7fd99c15bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:49.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.263+0000 7fd9a1217700 1 --2- 172.21.15.203:0/4097688726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd99c0bf120 0x7fd99c15bbe0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fd99000eba0 tx=0x7fd990006de0).stop 2021-01-06T22:03:49.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.263+0000 7fd9a1217700 1 -- 172.21.15.203:0/4097688726 shutdown_connections 2021-01-06T22:03:49.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.263+0000 7fd9a1217700 1 --2- 172.21.15.203:0/4097688726 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd97c0563d0 0x7fd97c058890 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:49.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.263+0000 7fd9a1217700 1 --2- 172.21.15.203:0/4097688726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd99c0bf500 0x7fd99c0ba540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:49.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.263+0000 7fd9a1217700 1 --2- 172.21.15.203:0/4097688726 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd99c066da0 0x7fd99c15b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:49.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.263+0000 7fd9a1217700 1 --2- 172.21.15.203:0/4097688726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd99c0bf120 0x7fd99c15bbe0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:49.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.263+0000 7fd9a1217700 1 -- 172.21.15.203:0/4097688726 >> 172.21.15.203:0/4097688726 conn(0x7fd99c060d80 msgr2=0x7fd99c147350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:49.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.263+0000 7fd9a1217700 1 -- 172.21.15.203:0/4097688726 shutdown_connections 2021-01-06T22:03:49.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:49.263+0000 7fd9a1217700 1 -- 172.21.15.203:0/4097688726 wait complete. 2021-01-06T22:03:49.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2021-01-06T22:03:50.279 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2021-01-06T22:03:50.427 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0ee0e2e700 1 Processor -- start 2021-01-06T22:03:50.428 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0ee0e2e700 1 -- start start 2021-01-06T22:03:50.428 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0ee0e2e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0edc0be5c0 0x7f0edc0bedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:50.428 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0ee0e2e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0edc151d30 0x7f0edc063ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:50.428 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0ee0e2e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0edc065e40 con 0x7f0edc0be1e0 2021-01-06T22:03:50.429 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0ee0e2e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0edc0bb890 con 0x7f0edc151d30 2021-01-06T22:03:50.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0ee0e2e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc0684f0 con 0x7f0edc0be5c0 2021-01-06T22:03:50.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0edbfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0edc151d30 0x7f0edc063ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:50.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0edb7fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f0edc0be1e0 0x7f0edc0667b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47500/0 (socket says 172.21.15.203:47500) 2021-01-06T22:03:50.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0edb7fe700 1 -- 172.21.15.203:0/3094629821 learned_addr learned my addr 172.21.15.203:0/3094629821 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:50.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0ed2ffd700 1 --2- 172.21.15.203:0/3094629821 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0edc0be5c0 0x7f0edc0bedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:50.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0edaffd700 1 -- 172.21.15.203:0/3094629821 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 20672792 0 0) 0x7f0edc065e40 con 0x7f0edc0be1e0 2021-01-06T22:03:50.432 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0edaffd700 1 -- 172.21.15.203:0/3094629821 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ebc003230 con 0x7f0edc0be1e0 2021-01-06T22:03:50.432 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0edbfff700 1 -- 172.21.15.203:0/3094629821 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0edc0be5c0 msgr2=0x7f0edc0bedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:50.432 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0edbfff700 1 --2- 172.21.15.203:0/3094629821 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0edc0be5c0 0x7f0edc0bedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.432 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0edbfff700 1 -- 172.21.15.203:0/3094629821 >> v1:172.21.15.203:6789/0 conn(0x7f0edc0be1e0 legacy=0x7f0edc0667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:50.433 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0edbfff700 1 -- 172.21.15.203:0/3094629821 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc068790 con 0x7f0edc151d30 2021-01-06T22:03:50.433 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0edbfff700 1 --2- 172.21.15.203:0/3094629821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0edc151d30 0x7f0edc063ab0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f0ed4009c90 tx=0x7f0ed40091d0).ready entity=mon.1 client_cookie=ee497b327b4c3aa8 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:50.433 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0edaffd700 1 -- 172.21.15.203:0/3094629821 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed400b9c0 con 0x7f0edc151d30 2021-01-06T22:03:50.433 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0edaffd700 1 -- 172.21.15.203:0/3094629821 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed400bb20 con 0x7f0edc151d30 2021-01-06T22:03:50.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0edaffd700 1 -- 172.21.15.203:0/3094629821 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed400bea0 con 0x7f0edc151d30 2021-01-06T22:03:50.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0ee0e2e700 1 -- 172.21.15.203:0/3094629821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0edc151d30 msgr2=0x7f0edc063ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:50.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0ee0e2e700 1 --2- 172.21.15.203:0/3094629821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0edc151d30 0x7f0edc063ab0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f0ed4009c90 tx=0x7f0ed40091d0).stop 2021-01-06T22:03:50.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0ee0e2e700 1 -- 172.21.15.203:0/3094629821 shutdown_connections 2021-01-06T22:03:50.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0ee0e2e700 1 --2- 172.21.15.203:0/3094629821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0edc151d30 0x7f0edc063ab0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0ee0e2e700 1 --2- 172.21.15.203:0/3094629821 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0edc0be5c0 0x7f0edc0bedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0ee0e2e700 1 -- 172.21.15.203:0/3094629821 >> 172.21.15.203:0/3094629821 conn(0x7f0edc0631f0 msgr2=0x7f0edc0c4450 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:50.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0ee0e2e700 1 -- 172.21.15.203:0/3094629821 shutdown_connections 2021-01-06T22:03:50.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.419+0000 7f0ee0e2e700 1 -- 172.21.15.203:0/3094629821 wait complete. 2021-01-06T22:03:50.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0ee0e2e700 1 Processor -- start 2021-01-06T22:03:50.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0ee0e2e700 1 -- start start 2021-01-06T22:03:50.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0ee0e2e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0edc0be1e0 0x7f0edc0a3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:50.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0ee0e2e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0edc0be5c0 0x7f0edc0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:50.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0ee0e2e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0edc151d30 0x7f0edc0bcc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:50.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0ee0e2e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc1618a0 con 0x7f0edc151d30 2021-01-06T22:03:50.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0ee0e2e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc0a7460 con 0x7f0edc0be1e0 2021-01-06T22:03:50.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0ee0e2e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc0a7700 con 0x7f0edc0be5c0 2021-01-06T22:03:50.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0edb7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0edc0be1e0 0x7f0edc0a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:50.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0edb7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0edc0be1e0 0x7f0edc0a3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48910/0 (socket says 172.21.15.203:48910) 2021-01-06T22:03:50.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0edb7fe700 1 -- 172.21.15.203:0/987525422 learned_addr learned my addr 172.21.15.203:0/987525422 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:50.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0ed2ffd700 1 --2- 172.21.15.203:0/987525422 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0edc0be5c0 0x7f0edc0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:50.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0edbfff700 1 --2- 172.21.15.203:0/987525422 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0edc151d30 0x7f0edc0bcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:50.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0edb7fe700 1 -- 172.21.15.203:0/987525422 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0edc0be5c0 msgr2=0x7f0edc0a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:50.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0edb7fe700 1 --2- 172.21.15.203:0/987525422 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0edc0be5c0 0x7f0edc0a2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0edb7fe700 1 -- 172.21.15.203:0/987525422 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0edc151d30 msgr2=0x7f0edc0bcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:50.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0edb7fe700 1 --2- 172.21.15.203:0/987525422 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0edc151d30 0x7f0edc0bcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0edb7fe700 1 -- 172.21.15.203:0/987525422 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc0a79a0 con 0x7f0edc0be1e0 2021-01-06T22:03:50.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0edbfff700 1 --2- 172.21.15.203:0/987525422 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0edc151d30 0x7f0edc0bcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:50.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0edb7fe700 1 --2- 172.21.15.203:0/987525422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0edc0be1e0 0x7f0edc0a3080 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f0ec400c6d0 tx=0x7f0ec4007ac0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:50.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0ed3fff700 1 -- 172.21.15.203:0/987525422 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40072e0 con 0x7f0edc0be1e0 2021-01-06T22:03:50.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0ed3fff700 1 -- 172.21.15.203:0/987525422 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec4007440 con 0x7f0edc0be1e0 2021-01-06T22:03:50.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0ee0e2e700 1 -- 172.21.15.203:0/987525422 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0edc1615c0 con 0x7f0edc0be1e0 2021-01-06T22:03:50.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0ed3fff700 1 -- 172.21.15.203:0/987525422 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40057f0 con 0x7f0edc0be1e0 2021-01-06T22:03:50.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.423+0000 7f0ee0e2e700 1 -- 172.21.15.203:0/987525422 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0edc0bd120 con 0x7f0edc0be1e0 2021-01-06T22:03:50.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.427+0000 7f0ee0e2e700 1 -- 172.21.15.203:0/987525422 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0edc0450a0 con 0x7f0edc0be1e0 2021-01-06T22:03:50.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.427+0000 7f0ed3fff700 1 -- 172.21.15.203:0/987525422 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f0ec401a060 con 0x7f0edc0be1e0 2021-01-06T22:03:50.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.427+0000 7f0ed3fff700 1 --2- 172.21.15.203:0/987525422 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0ec00563d0 0x7f0ec0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:50.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.427+0000 7f0ed3fff700 1 -- 172.21.15.203:0/987525422 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0ec406ed50 con 0x7f0edc0be1e0 2021-01-06T22:03:50.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.427+0000 7f0ed2ffd700 1 --2- 172.21.15.203:0/987525422 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0ec00563d0 0x7f0ec0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:50.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.427+0000 7f0ed2ffd700 1 --2- 172.21.15.203:0/987525422 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0ec00563d0 0x7f0ec0058890 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f0edc0aa150 tx=0x7f0ecc006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:50.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.431+0000 7f0ed3fff700 1 -- 172.21.15.203:0/987525422 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f0ec40445d0 con 0x7f0edc0be1e0 2021-01-06T22:03:50.583 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.579+0000 7f0ee0e2e700 1 -- 172.21.15.203:0/987525422 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0edc0c52d0 con 0x7f0edc0be1e0 2021-01-06T22:03:50.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.583+0000 7f0ed3fff700 1 -- 172.21.15.203:0/987525422 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0ec4018030 con 0x7f0edc0be1e0 2021-01-06T22:03:50.584 INFO:teuthology.orchestra.run.smithi203.stdout:55834574854 2021-01-06T22:03:50.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.583+0000 7f0ee0e2e700 1 -- 172.21.15.203:0/987525422 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0ec00563d0 msgr2=0x7f0ec0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:50.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.583+0000 7f0ee0e2e700 1 --2- 172.21.15.203:0/987525422 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0ec00563d0 0x7f0ec0058890 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f0edc0aa150 tx=0x7f0ecc006de0).stop 2021-01-06T22:03:50.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.583+0000 7f0ee0e2e700 1 -- 172.21.15.203:0/987525422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0edc0be1e0 msgr2=0x7f0edc0a3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:50.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.583+0000 7f0ee0e2e700 1 --2- 172.21.15.203:0/987525422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0edc0be1e0 0x7f0edc0a3080 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f0ec400c6d0 tx=0x7f0ec4007ac0).stop 2021-01-06T22:03:50.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.583+0000 7f0ee0e2e700 1 -- 172.21.15.203:0/987525422 shutdown_connections 2021-01-06T22:03:50.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.583+0000 7f0ee0e2e700 1 --2- 172.21.15.203:0/987525422 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0ec00563d0 0x7f0ec0058890 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.583+0000 7f0ee0e2e700 1 --2- 172.21.15.203:0/987525422 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0edc151d30 0x7f0edc0bcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.583+0000 7f0ee0e2e700 1 --2- 172.21.15.203:0/987525422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0edc0be1e0 0x7f0edc0a3080 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.583+0000 7f0ee0e2e700 1 --2- 172.21.15.203:0/987525422 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0edc0be5c0 0x7f0edc0a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.583+0000 7f0ee0e2e700 1 -- 172.21.15.203:0/987525422 >> 172.21.15.203:0/987525422 conn(0x7f0edc0631f0 msgr2=0x7f0edc0c2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:50.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.583+0000 7f0ee0e2e700 1 -- 172.21.15.203:0/987525422 shutdown_connections 2021-01-06T22:03:50.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.583+0000 7f0ee0e2e700 1 -- 172.21.15.203:0/987525422 wait complete. 2021-01-06T22:03:50.598 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2021-01-06T22:03:50.598 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2021-01-06T22:03:50.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4cb6b2c700 1 Processor -- start 2021-01-06T22:03:50.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4cb6b2c700 1 -- start start 2021-01-06T22:03:50.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4cb6b2c700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4cb0151b80 0x7f4cb0066a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:50.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4cb6b2c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4cb01505a0 0x7f4cb0150a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:50.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4cb6b2c700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb0150f20 con 0x7f4cb01517a0 2021-01-06T22:03:50.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4cb6b2c700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb014bc00 con 0x7f4cb0151b80 2021-01-06T22:03:50.727 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4cb6b2c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb0066f30 con 0x7f4cb01505a0 2021-01-06T22:03:50.727 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4cb50c9700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4cb01505a0 0x7f4cb0150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:50.727 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4cb50c9700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4cb01505a0 0x7f4cb0150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48922/0 (socket says 172.21.15.203:48922) 2021-01-06T22:03:50.727 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4caffff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4cb0151b80 0x7f4cb0066a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:50.728 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4cb48c8700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f4cb01517a0 0x7f4cb00b8a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:42166/0 (socket says 172.21.15.203:42166) 2021-01-06T22:03:50.728 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4cb48c8700 1 -- 172.21.15.203:0/2632015379 learned_addr learned my addr 172.21.15.203:0/2632015379 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:50.729 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4caffff700 1 -- 172.21.15.203:0/2632015379 >> v2:172.21.15.203:3300/0 conn(0x7f4cb01505a0 msgr2=0x7f4cb0150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:50.729 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4caffff700 1 --2- 172.21.15.203:0/2632015379 >> v2:172.21.15.203:3300/0 conn(0x7f4cb01505a0 0x7f4cb0150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4caffff700 1 -- 172.21.15.203:0/2632015379 >> v1:172.21.15.91:6789/0 conn(0x7f4cb01517a0 legacy=0x7f4cb00b8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:50.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4caffff700 1 -- 172.21.15.203:0/2632015379 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb014f680 con 0x7f4cb0151b80 2021-01-06T22:03:50.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.723+0000 7f4cb50c9700 1 --2- 172.21.15.203:0/2632015379 >> v2:172.21.15.203:3300/0 conn(0x7f4cb01505a0 0x7f4cb0150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:50.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.727+0000 7f4caffff700 1 --2- 172.21.15.203:0/2632015379 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4cb0151b80 0x7f4cb0066a40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f4cb0151540 tx=0x7f4ca0006de0).ready entity=mon.0 client_cookie=b0877e385cc5f3b server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:50.731 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.727+0000 7f4caf7fe700 1 -- 172.21.15.203:0/2632015379 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0009750 con 0x7f4cb0151b80 2021-01-06T22:03:50.731 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.727+0000 7f4caf7fe700 1 -- 172.21.15.203:0/2632015379 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca00098b0 con 0x7f4cb0151b80 2021-01-06T22:03:50.731 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.727+0000 7f4caf7fe700 1 -- 172.21.15.203:0/2632015379 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0009c30 con 0x7f4cb0151b80 2021-01-06T22:03:50.731 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.727+0000 7f4cb6b2c700 1 -- 172.21.15.203:0/2632015379 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4cb0151b80 msgr2=0x7f4cb0066a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:50.731 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.727+0000 7f4cb6b2c700 1 --2- 172.21.15.203:0/2632015379 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4cb0151b80 0x7f4cb0066a40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f4cb0151540 tx=0x7f4ca0006de0).stop 2021-01-06T22:03:50.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.727+0000 7f4cb6b2c700 1 -- 172.21.15.203:0/2632015379 shutdown_connections 2021-01-06T22:03:50.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.727+0000 7f4cb6b2c700 1 --2- 172.21.15.203:0/2632015379 >> v2:172.21.15.203:3300/0 conn(0x7f4cb01505a0 0x7f4cb0150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.727+0000 7f4cb6b2c700 1 --2- 172.21.15.203:0/2632015379 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4cb0151b80 0x7f4cb0066a40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.727+0000 7f4cb6b2c700 1 -- 172.21.15.203:0/2632015379 >> 172.21.15.203:0/2632015379 conn(0x7f4cb0060d80 msgr2=0x7f4cb00661a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:50.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.727+0000 7f4cb6b2c700 1 -- 172.21.15.203:0/2632015379 shutdown_connections 2021-01-06T22:03:50.733 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.727+0000 7f4cb6b2c700 1 -- 172.21.15.203:0/2632015379 wait complete. 2021-01-06T22:03:50.734 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.727+0000 7f4cb6b2c700 1 Processor -- start 2021-01-06T22:03:50.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.727+0000 7f4cb6b2c700 1 -- start start 2021-01-06T22:03:50.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb6b2c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4cb01505a0 0x7f4cb014eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:50.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb6b2c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cb01517a0 0x7f4cb0153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:50.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb6b2c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4cb0151b80 0x7f4cb0160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:50.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb6b2c700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb014f3b0 con 0x7f4cb0151b80 2021-01-06T22:03:50.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb6b2c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb014db70 con 0x7f4cb01517a0 2021-01-06T22:03:50.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb6b2c700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb014aa60 con 0x7f4cb01505a0 2021-01-06T22:03:50.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4caffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cb01517a0 0x7f4cb0153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:50.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4caffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cb01517a0 0x7f4cb0153400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48926/0 (socket says 172.21.15.203:48926) 2021-01-06T22:03:50.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4caffff700 1 -- 172.21.15.203:0/4227324093 learned_addr learned my addr 172.21.15.203:0/4227324093 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:50.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4caffff700 1 -- 172.21.15.203:0/4227324093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cb01517a0 msgr2=0x7f4cb0153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2021-01-06T22:03:50.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4caffff700 1 -- 172.21.15.203:0/4227324093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cb01517a0 msgr2=0x7f4cb0153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2021-01-06T22:03:50.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4caffff700 1 --2- 172.21.15.203:0/4227324093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cb01517a0 0x7f4cb0153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:03:50.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4caffff700 1 --2- 172.21.15.203:0/4227324093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cb01517a0 0x7f4cb0153400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-01-06T22:03:50.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb50c9700 1 --2- 172.21.15.203:0/4227324093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4cb0151b80 0x7f4cb0160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:50.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb48c8700 1 --2- 172.21.15.203:0/4227324093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4cb01505a0 0x7f4cb014eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:50.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb48c8700 1 -- 172.21.15.203:0/4227324093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cb01517a0 msgr2=0x7f4cb0153400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2021-01-06T22:03:50.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb48c8700 1 --2- 172.21.15.203:0/4227324093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cb01517a0 0x7f4cb0153400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb48c8700 1 -- 172.21.15.203:0/4227324093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4cb0151b80 msgr2=0x7f4cb0160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:50.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb48c8700 1 --2- 172.21.15.203:0/4227324093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4cb0151b80 0x7f4cb0160e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb48c8700 1 -- 172.21.15.203:0/4227324093 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb014ad00 con 0x7f4cb01505a0 2021-01-06T22:03:50.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb48c8700 1 --2- 172.21.15.203:0/4227324093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4cb01505a0 0x7f4cb014eaa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f4c9800c6e0 tx=0x7f4c98007b70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:50.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cacff9700 1 -- 172.21.15.203:0/4227324093 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c980072e0 con 0x7f4cb01505a0 2021-01-06T22:03:50.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cacff9700 1 -- 172.21.15.203:0/4227324093 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c98007440 con 0x7f4cb01505a0 2021-01-06T22:03:50.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cacff9700 1 -- 172.21.15.203:0/4227324093 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c980057f0 con 0x7f4cb01505a0 2021-01-06T22:03:50.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb6b2c700 1 -- 172.21.15.203:0/4227324093 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb014d390 con 0x7f4cb01505a0 2021-01-06T22:03:50.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb6b2c700 1 -- 172.21.15.203:0/4227324093 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb014d8d0 con 0x7f4cb01505a0 2021-01-06T22:03:50.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.731+0000 7f4cb6b2c700 1 -- 172.21.15.203:0/4227324093 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb005d600 con 0x7f4cb01505a0 2021-01-06T22:03:50.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.735+0000 7f4cacff9700 1 -- 172.21.15.203:0/4227324093 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f4c98010070 con 0x7f4cb01505a0 2021-01-06T22:03:50.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.735+0000 7f4cacff9700 1 --2- 172.21.15.203:0/4227324093 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4c940563d0 0x7f4c94058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:50.742 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.735+0000 7f4cacff9700 1 -- 172.21.15.203:0/4227324093 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4c9806f1a0 con 0x7f4cb01505a0 2021-01-06T22:03:50.742 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.735+0000 7f4caffff700 1 --2- 172.21.15.203:0/4227324093 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4c940563d0 0x7f4c94058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:50.742 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.735+0000 7f4caffff700 1 --2- 172.21.15.203:0/4227324093 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4c940563d0 0x7f4c94058890 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f4ca0009450 tx=0x7f4ca0016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:50.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.747+0000 7f4cacff9700 1 -- 172.21.15.203:0/4227324093 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f4c980448b0 con 0x7f4cb01505a0 2021-01-06T22:03:50.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.895+0000 7f4cb6b2c700 1 -- 172.21.15.203:0/4227324093 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4cb00008d0 con 0x7f4cb01505a0 2021-01-06T22:03:50.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.899+0000 7f4cacff9700 1 -- 172.21.15.203:0/4227324093 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4c98047ed0 con 0x7f4cb01505a0 2021-01-06T22:03:50.899 INFO:teuthology.orchestra.run.smithi203.stdout:60129542149 2021-01-06T22:03:50.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.899+0000 7f4cb6b2c700 1 -- 172.21.15.203:0/4227324093 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4c940563d0 msgr2=0x7f4c94058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:50.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.899+0000 7f4cb6b2c700 1 --2- 172.21.15.203:0/4227324093 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4c940563d0 0x7f4c94058890 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f4ca0009450 tx=0x7f4ca0016000).stop 2021-01-06T22:03:50.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.899+0000 7f4cb6b2c700 1 -- 172.21.15.203:0/4227324093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4cb01505a0 msgr2=0x7f4cb014eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:50.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.899+0000 7f4cb6b2c700 1 --2- 172.21.15.203:0/4227324093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4cb01505a0 0x7f4cb014eaa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f4c9800c6e0 tx=0x7f4c98007b70).stop 2021-01-06T22:03:50.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.899+0000 7f4cb6b2c700 1 -- 172.21.15.203:0/4227324093 shutdown_connections 2021-01-06T22:03:50.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.899+0000 7f4cb6b2c700 1 --2- 172.21.15.203:0/4227324093 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4c940563d0 0x7f4c94058890 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.899+0000 7f4cb6b2c700 1 --2- 172.21.15.203:0/4227324093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4cb0151b80 0x7f4cb0160e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.899+0000 7f4cb6b2c700 1 --2- 172.21.15.203:0/4227324093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4cb01505a0 0x7f4cb014eaa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.899+0000 7f4cb6b2c700 1 --2- 172.21.15.203:0/4227324093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cb01517a0 0x7f4cb0153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:50.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.899+0000 7f4cb6b2c700 1 -- 172.21.15.203:0/4227324093 >> 172.21.15.203:0/4227324093 conn(0x7f4cb0060d80 msgr2=0x7f4cb00b7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:50.905 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.899+0000 7f4cb6b2c700 1 -- 172.21.15.203:0/4227324093 shutdown_connections 2021-01-06T22:03:50.905 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:50.899+0000 7f4cb6b2c700 1 -- 172.21.15.203:0/4227324093 wait complete. 2021-01-06T22:03:50.914 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.1 2021-01-06T22:03:50.915 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2021-01-06T22:03:51.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea3fff700 1 Processor -- start 2021-01-06T22:03:51.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea3fff700 1 -- start start 2021-01-06T22:03:51.038 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea3fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8ea40610c0 0x7f8ea4062910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.038 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea3fff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea4060d00 con 0x7f8ea40662c0 2021-01-06T22:03:51.039 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea3fff700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea4155a50 con 0x7f8ea4065ee0 2021-01-06T22:03:51.039 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea3fff700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea40a7fd0 con 0x7f8ea40610c0 2021-01-06T22:03:51.039 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea2ffd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f8ea4065ee0 0x7f8ea4152380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47526/0 (socket says 172.21.15.203:47526) 2021-01-06T22:03:51.039 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea2ffd700 1 -- 172.21.15.203:0/2247589443 learned_addr learned my addr 172.21.15.203:0/2247589443 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:51.039 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea27fc700 1 --2- 172.21.15.203:0/2247589443 >> v2:172.21.15.91:3300/0 conn(0x7f8ea40610c0 0x7f8ea4062910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea1ffb700 1 -- 172.21.15.203:0/2247589443 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149336911 0 0) 0x7f8ea4155a50 con 0x7f8ea4065ee0 2021-01-06T22:03:51.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea1ffb700 1 -- 172.21.15.203:0/2247589443 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e84003230 con 0x7f8ea4065ee0 2021-01-06T22:03:51.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea1ffb700 1 -- 172.21.15.203:0/2247589443 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709757345 0 0) 0x7f8ea4060d00 con 0x7f8ea40662c0 2021-01-06T22:03:51.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea1ffb700 1 -- 172.21.15.203:0/2247589443 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ea4155a50 con 0x7f8ea40662c0 2021-01-06T22:03:51.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea27fc700 1 -- 172.21.15.203:0/2247589443 >> v1:172.21.15.203:6789/0 conn(0x7f8ea4065ee0 legacy=0x7f8ea4152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea27fc700 1 -- 172.21.15.203:0/2247589443 >> v1:172.21.15.91:6789/0 conn(0x7f8ea40662c0 legacy=0x7f8ea415cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea27fc700 1 -- 172.21.15.203:0/2247589443 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea4155cf0 con 0x7f8ea40610c0 2021-01-06T22:03:51.042 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.035+0000 7f8ea27fc700 1 --2- 172.21.15.203:0/2247589443 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ea40610c0 0x7f8ea4062910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f8e8c00b980 tx=0x7f8e8c006de0).ready entity=mon.0 client_cookie=a9db1897b6ed5db6 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:51.042 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.039+0000 7f8ea1ffb700 1 -- 172.21.15.203:0/2247589443 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c009640 con 0x7f8ea40610c0 2021-01-06T22:03:51.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.039+0000 7f8ea1ffb700 1 -- 172.21.15.203:0/2247589443 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8c0097a0 con 0x7f8ea40610c0 2021-01-06T22:03:51.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.039+0000 7f8ea1ffb700 1 -- 172.21.15.203:0/2247589443 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c009b20 con 0x7f8ea40610c0 2021-01-06T22:03:51.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.039+0000 7f8ea3fff700 1 -- 172.21.15.203:0/2247589443 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ea40610c0 msgr2=0x7f8ea4062910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.039+0000 7f8ea3fff700 1 --2- 172.21.15.203:0/2247589443 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ea40610c0 0x7f8ea4062910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f8e8c00b980 tx=0x7f8e8c006de0).stop 2021-01-06T22:03:51.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.039+0000 7f8ea3fff700 1 -- 172.21.15.203:0/2247589443 shutdown_connections 2021-01-06T22:03:51.044 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.039+0000 7f8ea3fff700 1 --2- 172.21.15.203:0/2247589443 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ea40610c0 0x7f8ea4062910 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.044 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.039+0000 7f8ea3fff700 1 -- 172.21.15.203:0/2247589443 >> 172.21.15.203:0/2247589443 conn(0x7f8ea4064ae0 msgr2=0x7f8ea4064ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:51.044 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.039+0000 7f8ea3fff700 1 -- 172.21.15.203:0/2247589443 shutdown_connections 2021-01-06T22:03:51.044 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.039+0000 7f8ea3fff700 1 -- 172.21.15.203:0/2247589443 wait complete. 2021-01-06T22:03:51.045 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea3fff700 1 Processor -- start 2021-01-06T22:03:51.045 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea3fff700 1 -- start start 2021-01-06T22:03:51.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea3fff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ea40610c0 0x7f8ea4158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea3fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ea4065ee0 0x7f8ea4157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ea40662c0 0x7f8ea4165660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea3fff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea4157240 con 0x7f8ea40610c0 2021-01-06T22:03:51.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea3fff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea40bc020 con 0x7f8ea40662c0 2021-01-06T22:03:51.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea3fff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea40bc2c0 con 0x7f8ea4065ee0 2021-01-06T22:03:51.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea37fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ea40662c0 0x7f8ea4165660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea37fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ea40662c0 0x7f8ea4165660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48942/0 (socket says 172.21.15.203:48942) 2021-01-06T22:03:51.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea37fe700 1 -- 172.21.15.203:0/2090262452 learned_addr learned my addr 172.21.15.203:0/2090262452 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:51.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea2ffd700 1 --2- 172.21.15.203:0/2090262452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ea40610c0 0x7f8ea4158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea27fc700 1 --2- 172.21.15.203:0/2090262452 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ea4065ee0 0x7f8ea4157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea37fe700 1 -- 172.21.15.203:0/2090262452 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ea4065ee0 msgr2=0x7f8ea4157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea37fe700 1 --2- 172.21.15.203:0/2090262452 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ea4065ee0 0x7f8ea4157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea37fe700 1 -- 172.21.15.203:0/2090262452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ea40610c0 msgr2=0x7f8ea4158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea37fe700 1 --2- 172.21.15.203:0/2090262452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ea40610c0 0x7f8ea4158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea37fe700 1 -- 172.21.15.203:0/2090262452 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea4157470 con 0x7f8ea40662c0 2021-01-06T22:03:51.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea2ffd700 1 --2- 172.21.15.203:0/2090262452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ea40610c0 0x7f8ea4158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:51.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8ea37fe700 1 --2- 172.21.15.203:0/2090262452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ea40662c0 0x7f8ea4165660 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f8e9400c740 tx=0x7f8e94007a10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:51.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8e9affd700 1 -- 172.21.15.203:0/2090262452 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9400e560 con 0x7f8ea40662c0 2021-01-06T22:03:51.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8e9affd700 1 -- 172.21.15.203:0/2090262452 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9400e6c0 con 0x7f8ea40662c0 2021-01-06T22:03:51.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.043+0000 7f8e9affd700 1 -- 172.21.15.203:0/2090262452 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9400ea40 con 0x7f8ea40662c0 2021-01-06T22:03:51.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.047+0000 7f8ea3fff700 1 -- 172.21.15.203:0/2090262452 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea4157710 con 0x7f8ea40662c0 2021-01-06T22:03:51.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.047+0000 7f8ea3fff700 1 -- 172.21.15.203:0/2090262452 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea40bc560 con 0x7f8ea40662c0 2021-01-06T22:03:51.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.047+0000 7f8ea3fff700 1 -- 172.21.15.203:0/2090262452 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea40450a0 con 0x7f8ea40662c0 2021-01-06T22:03:51.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.047+0000 7f8e9affd700 1 -- 172.21.15.203:0/2090262452 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f8e94019030 con 0x7f8ea40662c0 2021-01-06T22:03:51.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.047+0000 7f8e9affd700 1 --2- 172.21.15.203:0/2090262452 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8e880563d0 0x7f8e88058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.047+0000 7f8ea2ffd700 1 --2- 172.21.15.203:0/2090262452 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8e880563d0 0x7f8e88058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.047+0000 7f8e9affd700 1 -- 172.21.15.203:0/2090262452 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8e9406e9e0 con 0x7f8ea40662c0 2021-01-06T22:03:51.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.047+0000 7f8ea2ffd700 1 --2- 172.21.15.203:0/2090262452 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8e880563d0 0x7f8e88058890 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f8ea40b9350 tx=0x7f8e9c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:51.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.051+0000 7f8e9affd700 1 -- 172.21.15.203:0/2090262452 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f8e940442b0 con 0x7f8ea40662c0 2021-01-06T22:03:51.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.207+0000 7f8ea3fff700 1 -- 172.21.15.203:0/2090262452 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8ea40b9380 con 0x7f8ea40662c0 2021-01-06T22:03:51.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.207+0000 7f8e9affd700 1 -- 172.21.15.203:0/2090262452 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8e94017070 con 0x7f8ea40662c0 2021-01-06T22:03:51.209 INFO:teuthology.orchestra.run.smithi203.stdout:60129542150 2021-01-06T22:03:51.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.207+0000 7f8ea3fff700 1 -- 172.21.15.203:0/2090262452 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8e880563d0 msgr2=0x7f8e88058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.207+0000 7f8ea3fff700 1 --2- 172.21.15.203:0/2090262452 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8e880563d0 0x7f8e88058890 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f8ea40b9350 tx=0x7f8e9c009040).stop 2021-01-06T22:03:51.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.207+0000 7f8ea3fff700 1 -- 172.21.15.203:0/2090262452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ea40662c0 msgr2=0x7f8ea4165660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.207+0000 7f8ea3fff700 1 --2- 172.21.15.203:0/2090262452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ea40662c0 0x7f8ea4165660 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f8e9400c740 tx=0x7f8e94007a10).stop 2021-01-06T22:03:51.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.207+0000 7f8ea3fff700 1 -- 172.21.15.203:0/2090262452 shutdown_connections 2021-01-06T22:03:51.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.207+0000 7f8ea3fff700 1 --2- 172.21.15.203:0/2090262452 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8e880563d0 0x7f8e88058890 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.207+0000 7f8ea3fff700 1 --2- 172.21.15.203:0/2090262452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ea40662c0 0x7f8ea4165660 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.207+0000 7f8ea3fff700 1 --2- 172.21.15.203:0/2090262452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ea40610c0 0x7f8ea4158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.207+0000 7f8ea3fff700 1 --2- 172.21.15.203:0/2090262452 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ea4065ee0 0x7f8ea4157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.207+0000 7f8ea3fff700 1 -- 172.21.15.203:0/2090262452 >> 172.21.15.203:0/2090262452 conn(0x7f8ea4064ae0 msgr2=0x7f8ea41537d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:51.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.207+0000 7f8ea3fff700 1 -- 172.21.15.203:0/2090262452 shutdown_connections 2021-01-06T22:03:51.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.207+0000 7f8ea3fff700 1 -- 172.21.15.203:0/2090262452 wait complete. 2021-01-06T22:03:51.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.2 2021-01-06T22:03:51.223 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2021-01-06T22:03:51.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d89066700 1 Processor -- start 2021-01-06T22:03:51.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d89066700 1 -- start start 2021-01-06T22:03:51.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d89066700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d84063c50 0x7f0d8414f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d89066700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0d84156310 0x7f0d8414f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d89066700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d840ba9e0 con 0x7f0d84068370 2021-01-06T22:03:51.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d89066700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8414fda0 con 0x7f0d84156310 2021-01-06T22:03:51.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d89066700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d840bac80 con 0x7f0d84063c50 2021-01-06T22:03:51.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d88865700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0d84156310 0x7f0d8414f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d88865700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0d84156310 0x7f0d8414f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48950/0 (socket says 172.21.15.203:48950) 2021-01-06T22:03:51.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d88865700 1 -- 172.21.15.203:0/568862189 learned_addr learned my addr 172.21.15.203:0/568862189 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:51.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d7b7fe700 1 --1- 172.21.15.203:0/568862189 >> v1:172.21.15.203:6789/0 conn(0x7f0d84068370 0x7f0d84155c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47540/0 (socket says 172.21.15.203:47540) 2021-01-06T22:03:51.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d83fff700 1 --2- 172.21.15.203:0/568862189 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d84063c50 0x7f0d8414f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d88865700 1 -- 172.21.15.203:0/568862189 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d84063c50 msgr2=0x7f0d8414f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:51.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d88865700 1 --2- 172.21.15.203:0/568862189 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d84063c50 0x7f0d8414f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d88865700 1 -- 172.21.15.203:0/568862189 >> v1:172.21.15.203:6789/0 conn(0x7f0d84068370 legacy=0x7f0d84155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d88865700 1 -- 172.21.15.203:0/568862189 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d840baf20 con 0x7f0d84156310 2021-01-06T22:03:51.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.351+0000 7f0d83fff700 1 --2- 172.21.15.203:0/568862189 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d84063c50 0x7f0d8414f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:51.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.355+0000 7f0d88865700 1 --2- 172.21.15.203:0/568862189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d84156310 0x7f0d8414f8b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f0d7c009d50 tx=0x7f0d7c009290).ready entity=mon.1 client_cookie=27ef4b4efeb5de19 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:51.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.355+0000 7f0d837fe700 1 -- 172.21.15.203:0/568862189 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c00bc80 con 0x7f0d84156310 2021-01-06T22:03:51.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.355+0000 7f0d837fe700 1 -- 172.21.15.203:0/568862189 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c00bde0 con 0x7f0d84156310 2021-01-06T22:03:51.359 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.355+0000 7f0d837fe700 1 -- 172.21.15.203:0/568862189 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c011610 con 0x7f0d84156310 2021-01-06T22:03:51.361 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.355+0000 7f0d89066700 1 -- 172.21.15.203:0/568862189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d84156310 msgr2=0x7f0d8414f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.361 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.355+0000 7f0d89066700 1 --2- 172.21.15.203:0/568862189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d84156310 0x7f0d8414f8b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f0d7c009d50 tx=0x7f0d7c009290).stop 2021-01-06T22:03:51.361 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.355+0000 7f0d89066700 1 -- 172.21.15.203:0/568862189 shutdown_connections 2021-01-06T22:03:51.361 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.355+0000 7f0d89066700 1 --2- 172.21.15.203:0/568862189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d84156310 0x7f0d8414f8b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.362 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.355+0000 7f0d89066700 1 --2- 172.21.15.203:0/568862189 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d84063c50 0x7f0d8414f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.362 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.355+0000 7f0d89066700 1 -- 172.21.15.203:0/568862189 >> 172.21.15.203:0/568862189 conn(0x7f0d840631f0 msgr2=0x7f0d840c0040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:51.362 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.355+0000 7f0d89066700 1 -- 172.21.15.203:0/568862189 shutdown_connections 2021-01-06T22:03:51.362 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.355+0000 7f0d89066700 1 -- 172.21.15.203:0/568862189 wait complete. 2021-01-06T22:03:51.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.355+0000 7f0d89066700 1 Processor -- start 2021-01-06T22:03:51.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.355+0000 7f0d89066700 1 -- start start 2021-01-06T22:03:51.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d89066700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d84063c50 0x7f0d84151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d89066700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d84068370 0x7f0d84151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d89066700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d84156310 0x7f0d840b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.364 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d89066700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d84153e90 con 0x7f0d84063c50 2021-01-06T22:03:51.364 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d89066700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d84154130 con 0x7f0d84068370 2021-01-06T22:03:51.364 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d89066700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d841543d0 con 0x7f0d84156310 2021-01-06T22:03:51.364 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d7b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d84068370 0x7f0d84151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.364 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d7b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d84068370 0x7f0d84151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48954/0 (socket says 172.21.15.203:48954) 2021-01-06T22:03:51.365 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d7b7fe700 1 -- 172.21.15.203:0/2202120743 learned_addr learned my addr 172.21.15.203:0/2202120743 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:51.365 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d83fff700 1 --2- 172.21.15.203:0/2202120743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d84063c50 0x7f0d84151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.365 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d88865700 1 --2- 172.21.15.203:0/2202120743 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d84156310 0x7f0d840b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.365 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d7b7fe700 1 -- 172.21.15.203:0/2202120743 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d84156310 msgr2=0x7f0d840b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.365 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d7b7fe700 1 --2- 172.21.15.203:0/2202120743 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d84156310 0x7f0d840b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.366 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d7b7fe700 1 -- 172.21.15.203:0/2202120743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d84063c50 msgr2=0x7f0d84151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.366 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d7b7fe700 1 --2- 172.21.15.203:0/2202120743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d84063c50 0x7f0d84151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.366 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d7b7fe700 1 -- 172.21.15.203:0/2202120743 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d840b7540 con 0x7f0d84068370 2021-01-06T22:03:51.366 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d7b7fe700 1 --2- 172.21.15.203:0/2202120743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d84068370 0x7f0d84151620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f0d6c00c710 tx=0x7f0d6c007b60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:51.366 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d80ff9700 1 -- 172.21.15.203:0/2202120743 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c0072e0 con 0x7f0d84068370 2021-01-06T22:03:51.367 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d89066700 1 -- 172.21.15.203:0/2202120743 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d840b77e0 con 0x7f0d84068370 2021-01-06T22:03:51.367 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d80ff9700 1 -- 172.21.15.203:0/2202120743 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6c005520 con 0x7f0d84068370 2021-01-06T22:03:51.367 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d89066700 1 -- 172.21.15.203:0/2202120743 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d84153b80 con 0x7f0d84068370 2021-01-06T22:03:51.367 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d80ff9700 1 -- 172.21.15.203:0/2202120743 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c0058a0 con 0x7f0d84068370 2021-01-06T22:03:51.368 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d89066700 1 -- 172.21.15.203:0/2202120743 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d840450a0 con 0x7f0d84068370 2021-01-06T22:03:51.368 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d80ff9700 1 -- 172.21.15.203:0/2202120743 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f0d6c010070 con 0x7f0d84068370 2021-01-06T22:03:51.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d80ff9700 1 --2- 172.21.15.203:0/2202120743 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0d680563d0 0x7f0d68058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d80ff9700 1 -- 172.21.15.203:0/2202120743 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0d6c06eca0 con 0x7f0d84068370 2021-01-06T22:03:51.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.359+0000 7f0d83fff700 1 --2- 172.21.15.203:0/2202120743 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0d680563d0 0x7f0d68058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.363+0000 7f0d83fff700 1 --2- 172.21.15.203:0/2202120743 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0d680563d0 0x7f0d68058890 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f0d74006fd0 tx=0x7f0d74008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:51.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.363+0000 7f0d80ff9700 1 -- 172.21.15.203:0/2202120743 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f0d6c073070 con 0x7f0d84068370 2021-01-06T22:03:51.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.515+0000 7f0d89066700 1 -- 172.21.15.203:0/2202120743 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0d840b7c80 con 0x7f0d84068370 2021-01-06T22:03:51.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.519+0000 7f0d80ff9700 1 -- 172.21.15.203:0/2202120743 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0d6c044520 con 0x7f0d84068370 2021-01-06T22:03:51.520 INFO:teuthology.orchestra.run.smithi203.stdout:60129542149 2021-01-06T22:03:51.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.519+0000 7f0d89066700 1 -- 172.21.15.203:0/2202120743 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0d680563d0 msgr2=0x7f0d68058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.519+0000 7f0d89066700 1 --2- 172.21.15.203:0/2202120743 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0d680563d0 0x7f0d68058890 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f0d74006fd0 tx=0x7f0d74008040).stop 2021-01-06T22:03:51.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.519+0000 7f0d89066700 1 -- 172.21.15.203:0/2202120743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d84068370 msgr2=0x7f0d84151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.519+0000 7f0d89066700 1 --2- 172.21.15.203:0/2202120743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d84068370 0x7f0d84151620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f0d6c00c710 tx=0x7f0d6c007b60).stop 2021-01-06T22:03:51.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.519+0000 7f0d89066700 1 -- 172.21.15.203:0/2202120743 shutdown_connections 2021-01-06T22:03:51.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.519+0000 7f0d89066700 1 --2- 172.21.15.203:0/2202120743 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0d680563d0 0x7f0d68058890 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.519+0000 7f0d89066700 1 --2- 172.21.15.203:0/2202120743 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d84156310 0x7f0d840b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.519+0000 7f0d89066700 1 --2- 172.21.15.203:0/2202120743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d84063c50 0x7f0d84151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.519+0000 7f0d89066700 1 --2- 172.21.15.203:0/2202120743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d84068370 0x7f0d84151620 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.519+0000 7f0d89066700 1 -- 172.21.15.203:0/2202120743 >> 172.21.15.203:0/2202120743 conn(0x7f0d840631f0 msgr2=0x7f0d8414c960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:51.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.519+0000 7f0d89066700 1 -- 172.21.15.203:0/2202120743 shutdown_connections 2021-01-06T22:03:51.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.519+0000 7f0d89066700 1 -- 172.21.15.203:0/2202120743 wait complete. 2021-01-06T22:03:51.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.3 2021-01-06T22:03:51.535 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2021-01-06T22:03:51.662 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.659+0000 7f257359e700 1 Processor -- start 2021-01-06T22:03:51.663 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.659+0000 7f257359e700 1 -- start start 2021-01-06T22:03:51.665 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257359e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2574064310 0x7f25740646f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.665 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257359e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f257414a5a0 con 0x7f257414a060 2021-01-06T22:03:51.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257359e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f257414a840 con 0x7f25740650c0 2021-01-06T22:03:51.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257359e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25740bb550 con 0x7f2574064310 2021-01-06T22:03:51.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257259c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2574064310 0x7f25740646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257259c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2574064310 0x7f25740646f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48960/0 (socket says 172.21.15.203:48960) 2021-01-06T22:03:51.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257259c700 1 -- 172.21.15.203:0/193507819 learned_addr learned my addr 172.21.15.203:0/193507819 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:51.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257259c700 1 -- 172.21.15.203:0/193507819 >> v1:172.21.15.203:6789/0 conn(0x7f25740650c0 legacy=0x7f2574147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:51.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257259c700 1 -- 172.21.15.203:0/193507819 >> v1:172.21.15.91:6789/0 conn(0x7f257414a060 legacy=0x7f25740bbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:51.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257259c700 1 -- 172.21.15.203:0/193507819 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257414aae0 con 0x7f2574064310 2021-01-06T22:03:51.668 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257259c700 1 --2- 172.21.15.203:0/193507819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2574064310 0x7f25740646f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f2564009d50 tx=0x7f2564009290).ready entity=mon.1 client_cookie=8d2a5946f9ef7fde server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:51.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257159a700 1 -- 172.21.15.203:0/193507819 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256400bcd0 con 0x7f2574064310 2021-01-06T22:03:51.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257159a700 1 -- 172.21.15.203:0/193507819 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256400be30 con 0x7f2574064310 2021-01-06T22:03:51.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257159a700 1 -- 172.21.15.203:0/193507819 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2564010620 con 0x7f2574064310 2021-01-06T22:03:51.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257359e700 1 -- 172.21.15.203:0/193507819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2574064310 msgr2=0x7f25740646f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257359e700 1 --2- 172.21.15.203:0/193507819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2574064310 0x7f25740646f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f2564009d50 tx=0x7f2564009290).stop 2021-01-06T22:03:51.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257359e700 1 -- 172.21.15.203:0/193507819 shutdown_connections 2021-01-06T22:03:51.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257359e700 1 --2- 172.21.15.203:0/193507819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2574064310 0x7f25740646f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257359e700 1 -- 172.21.15.203:0/193507819 >> 172.21.15.203:0/193507819 conn(0x7f25740c08c0 msgr2=0x7f25740c0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:51.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257359e700 1 -- 172.21.15.203:0/193507819 shutdown_connections 2021-01-06T22:03:51.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.663+0000 7f257359e700 1 -- 172.21.15.203:0/193507819 wait complete. 2021-01-06T22:03:51.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.667+0000 7f257359e700 1 Processor -- start 2021-01-06T22:03:51.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.667+0000 7f257359e700 1 -- start start 2021-01-06T22:03:51.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.667+0000 7f257359e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2574064310 0x7f25740b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.667+0000 7f257359e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25740650c0 0x7f25740bbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.667+0000 7f257359e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f257414a060 0x7f25740bf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.675 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.667+0000 7f257359e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257413c030 con 0x7f25740650c0 2021-01-06T22:03:51.675 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.667+0000 7f257359e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25740b6bf0 con 0x7f257414a060 2021-01-06T22:03:51.675 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.667+0000 7f257359e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25740b6e90 con 0x7f2574064310 2021-01-06T22:03:51.675 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.667+0000 7f2572d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f257414a060 0x7f25740bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.667+0000 7f2572d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f257414a060 0x7f25740bf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48970/0 (socket says 172.21.15.203:48970) 2021-01-06T22:03:51.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.667+0000 7f2572d9d700 1 -- 172.21.15.203:0/2458281577 learned_addr learned my addr 172.21.15.203:0/2458281577 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:51.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.667+0000 7f257259c700 1 --2- 172.21.15.203:0/2458281577 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2574064310 0x7f25740b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.667+0000 7f2571d9b700 1 --2- 172.21.15.203:0/2458281577 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25740650c0 0x7f25740bbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f2572d9d700 1 -- 172.21.15.203:0/2458281577 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2574064310 msgr2=0x7f25740b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f2572d9d700 1 --2- 172.21.15.203:0/2458281577 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2574064310 0x7f25740b5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f2572d9d700 1 -- 172.21.15.203:0/2458281577 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25740650c0 msgr2=0x7f25740bbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f2572d9d700 1 --2- 172.21.15.203:0/2458281577 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25740650c0 0x7f25740bbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f2572d9d700 1 -- 172.21.15.203:0/2458281577 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25740b7130 con 0x7f257414a060 2021-01-06T22:03:51.678 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f257259c700 1 --2- 172.21.15.203:0/2458281577 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2574064310 0x7f25740b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:51.678 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f2571d9b700 1 --2- 172.21.15.203:0/2458281577 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25740650c0 0x7f25740bbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:51.679 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f2572d9d700 1 --2- 172.21.15.203:0/2458281577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f257414a060 0x7f25740bf080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f256c0096e0 tx=0x7f256c006ce0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:51.679 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f25627fc700 1 -- 172.21.15.203:0/2458281577 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c00d560 con 0x7f257414a060 2021-01-06T22:03:51.679 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f25627fc700 1 -- 172.21.15.203:0/2458281577 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256c000e90 con 0x7f257414a060 2021-01-06T22:03:51.679 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f257359e700 1 -- 172.21.15.203:0/2458281577 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25740be940 con 0x7f257414a060 2021-01-06T22:03:51.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f25627fc700 1 -- 172.21.15.203:0/2458281577 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c00d930 con 0x7f257414a060 2021-01-06T22:03:51.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f257359e700 1 -- 172.21.15.203:0/2458281577 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25740c0550 con 0x7f257414a060 2021-01-06T22:03:51.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f257359e700 1 -- 172.21.15.203:0/2458281577 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25740bede0 con 0x7f257414a060 2021-01-06T22:03:51.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f25627fc700 1 -- 172.21.15.203:0/2458281577 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f256c00f030 con 0x7f257414a060 2021-01-06T22:03:51.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f25627fc700 1 --2- 172.21.15.203:0/2458281577 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2550056400 0x7f25500588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f25627fc700 1 -- 172.21.15.203:0/2458281577 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f256c06f110 con 0x7f257414a060 2021-01-06T22:03:51.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.671+0000 7f257259c700 1 --2- 172.21.15.203:0/2458281577 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2550056400 0x7f25500588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.675+0000 7f257259c700 1 --2- 172.21.15.203:0/2458281577 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2550056400 0x7f25500588c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f25640092d0 tx=0x7f2564018000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:51.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.679+0000 7f25627fc700 1 -- 172.21.15.203:0/2458281577 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f25740bede0 con 0x7f257414a060 2021-01-06T22:03:51.833 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.831+0000 7f257359e700 1 -- 172.21.15.203:0/2458281577 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f25740bede0 con 0x7f257414a060 2021-01-06T22:03:51.834 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.831+0000 7f25627fc700 1 -- 172.21.15.203:0/2458281577 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f25740bede0 con 0x7f257414a060 2021-01-06T22:03:51.834 INFO:teuthology.orchestra.run.smithi203.stdout:60129542150 2021-01-06T22:03:51.837 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.835+0000 7f257359e700 1 -- 172.21.15.203:0/2458281577 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2550056400 msgr2=0x7f25500588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.837 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.835+0000 7f257359e700 1 --2- 172.21.15.203:0/2458281577 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2550056400 0x7f25500588c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f25640092d0 tx=0x7f2564018000).stop 2021-01-06T22:03:51.837 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.835+0000 7f257359e700 1 -- 172.21.15.203:0/2458281577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f257414a060 msgr2=0x7f25740bf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.835+0000 7f257359e700 1 --2- 172.21.15.203:0/2458281577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f257414a060 0x7f25740bf080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f256c0096e0 tx=0x7f256c006ce0).stop 2021-01-06T22:03:51.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.835+0000 7f257359e700 1 -- 172.21.15.203:0/2458281577 shutdown_connections 2021-01-06T22:03:51.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.835+0000 7f257359e700 1 --2- 172.21.15.203:0/2458281577 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2550056400 0x7f25500588c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.835+0000 7f257359e700 1 --2- 172.21.15.203:0/2458281577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f257414a060 0x7f25740bf080 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.835+0000 7f257359e700 1 --2- 172.21.15.203:0/2458281577 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2574064310 0x7f25740b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.835+0000 7f257359e700 1 --2- 172.21.15.203:0/2458281577 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25740650c0 0x7f25740bbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.835+0000 7f257359e700 1 -- 172.21.15.203:0/2458281577 >> 172.21.15.203:0/2458281577 conn(0x7f25740c08c0 msgr2=0x7f2574145bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:51.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.835+0000 7f257359e700 1 -- 172.21.15.203:0/2458281577 shutdown_connections 2021-01-06T22:03:51.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.835+0000 7f257359e700 1 -- 172.21.15.203:0/2458281577 wait complete. 2021-01-06T22:03:51.854 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2021-01-06T22:03:51.855 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2021-01-06T22:03:51.983 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.979+0000 7f603a156700 1 Processor -- start 2021-01-06T22:03:51.983 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f603a156700 1 -- start start 2021-01-06T22:03:51.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f603a156700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6034064080 0x7f603414e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f603a156700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f60340bb4c0 0x7f603414e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f603a156700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6034151450 con 0x7f6034064460 2021-01-06T22:03:51.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f603a156700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f603414de80 con 0x7f6034064080 2021-01-06T22:03:51.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f603a156700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60341523a0 con 0x7f60340bb4c0 2021-01-06T22:03:51.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f6033fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f60340bb4c0 0x7f603414e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f6032ffd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f6034064460 0x7f60340badb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47568/0 (socket says 172.21.15.203:47568) 2021-01-06T22:03:51.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f6032ffd700 1 -- 172.21.15.203:0/448518071 learned_addr learned my addr 172.21.15.203:0/448518071 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:51.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f60337fe700 1 --2- 172.21.15.203:0/448518071 >> v2:172.21.15.91:3300/0 conn(0x7f6034064080 0x7f603414e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f6033fff700 1 -- 172.21.15.203:0/448518071 >> v2:172.21.15.91:3300/0 conn(0x7f6034064080 msgr2=0x7f603414e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:51.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f6033fff700 1 --2- 172.21.15.203:0/448518071 >> v2:172.21.15.91:3300/0 conn(0x7f6034064080 0x7f603414e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f6033fff700 1 -- 172.21.15.203:0/448518071 >> v1:172.21.15.203:6789/0 conn(0x7f6034064460 legacy=0x7f60340badb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f60327fc700 1 -- 172.21.15.203:0/448518071 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 717116465 0 0) 0x7f6034151450 con 0x7f6034064460 2021-01-06T22:03:51.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f6033fff700 1 -- 172.21.15.203:0/448518071 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6034151680 con 0x7f60340bb4c0 2021-01-06T22:03:51.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f6033fff700 1 --2- 172.21.15.203:0/448518071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60340bb4c0 0x7f603414e760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f6034062090 tx=0x7f602c0091d0).ready entity=mon.1 client_cookie=93a5db450d2813f0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:51.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f60327fc700 1 -- 172.21.15.203:0/448518071 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c00bad0 con 0x7f60340bb4c0 2021-01-06T22:03:51.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f60327fc700 1 -- 172.21.15.203:0/448518071 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c00bc30 con 0x7f60340bb4c0 2021-01-06T22:03:51.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.983+0000 7f60327fc700 1 -- 172.21.15.203:0/448518071 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c011400 con 0x7f60340bb4c0 2021-01-06T22:03:51.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 -- 172.21.15.203:0/448518071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60340bb4c0 msgr2=0x7f603414e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 --2- 172.21.15.203:0/448518071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60340bb4c0 0x7f603414e760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f6034062090 tx=0x7f602c0091d0).stop 2021-01-06T22:03:51.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 -- 172.21.15.203:0/448518071 shutdown_connections 2021-01-06T22:03:51.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 --2- 172.21.15.203:0/448518071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60340bb4c0 0x7f603414e760 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 --2- 172.21.15.203:0/448518071 >> v2:172.21.15.91:3300/0 conn(0x7f6034064080 0x7f603414e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 -- 172.21.15.203:0/448518071 >> 172.21.15.203:0/448518071 conn(0x7f6034056cd0 msgr2=0x7f60340626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:51.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 -- 172.21.15.203:0/448518071 shutdown_connections 2021-01-06T22:03:51.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 -- 172.21.15.203:0/448518071 wait complete. 2021-01-06T22:03:51.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 Processor -- start 2021-01-06T22:03:51.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 -- start start 2021-01-06T22:03:51.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6034064080 0x7f60340b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6034064460 0x7f603415dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60340bb4c0 0x7f60340bf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:51.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60340b60d0 con 0x7f6034064080 2021-01-06T22:03:51.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60340b6370 con 0x7f60340bb4c0 2021-01-06T22:03:51.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60340b6610 con 0x7f6034064460 2021-01-06T22:03:51.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f6033fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60340bb4c0 0x7f60340bf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f6033fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60340bb4c0 0x7f60340bf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48982/0 (socket says 172.21.15.203:48982) 2021-01-06T22:03:51.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f6033fff700 1 -- 172.21.15.203:0/2485419950 learned_addr learned my addr 172.21.15.203:0/2485419950 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:51.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f60337fe700 1 --2- 172.21.15.203:0/2485419950 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6034064080 0x7f60340b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f6032ffd700 1 --2- 172.21.15.203:0/2485419950 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6034064460 0x7f603415dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:51.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f6033fff700 1 -- 172.21.15.203:0/2485419950 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6034064460 msgr2=0x7f603415dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f6033fff700 1 --2- 172.21.15.203:0/2485419950 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6034064460 0x7f603415dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f6033fff700 1 -- 172.21.15.203:0/2485419950 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6034064080 msgr2=0x7f60340b6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:51.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f6033fff700 1 --2- 172.21.15.203:0/2485419950 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6034064080 0x7f60340b6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:51.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f6033fff700 1 -- 172.21.15.203:0/2485419950 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603415d380 con 0x7f60340bb4c0 2021-01-06T22:03:51.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f6032ffd700 1 --2- 172.21.15.203:0/2485419950 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6034064460 0x7f603415dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:03:51.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f6033fff700 1 --2- 172.21.15.203:0/2485419950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60340bb4c0 0x7f60340bf390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f602c011f80 tx=0x7f602c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:51.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f601b7fe700 1 -- 172.21.15.203:0/2485419950 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c010700 con 0x7f60340bb4c0 2021-01-06T22:03:51.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f601b7fe700 1 -- 172.21.15.203:0/2485419950 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c010860 con 0x7f60340bb4c0 2021-01-06T22:03:51.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 -- 172.21.15.203:0/2485419950 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603415d620 con 0x7f60340bb4c0 2021-01-06T22:03:51.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f601b7fe700 1 -- 172.21.15.203:0/2485419950 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c010c00 con 0x7f60340bb4c0 2021-01-06T22:03:52.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.987+0000 7f603a156700 1 -- 172.21.15.203:0/2485419950 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603415e250 con 0x7f60340bb4c0 2021-01-06T22:03:52.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.991+0000 7f601b7fe700 1 -- 172.21.15.203:0/2485419950 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f602c00f070 con 0x7f60340bb4c0 2021-01-06T22:03:52.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.991+0000 7f601b7fe700 1 --2- 172.21.15.203:0/2485419950 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f60140563d0 0x7f6014058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.991+0000 7f603a156700 1 -- 172.21.15.203:0/2485419950 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603405cfe0 con 0x7f60340bb4c0 2021-01-06T22:03:52.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.991+0000 7f601b7fe700 1 -- 172.21.15.203:0/2485419950 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f602c0712c0 con 0x7f60340bb4c0 2021-01-06T22:03:52.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.991+0000 7f60337fe700 1 --2- 172.21.15.203:0/2485419950 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f60140563d0 0x7f6014058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.991+0000 7f60337fe700 1 --2- 172.21.15.203:0/2485419950 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f60140563d0 0x7f6014058890 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f602800bb50 tx=0x7f6028006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:52.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:51.995+0000 7f601b7fe700 1 -- 172.21.15.203:0/2485419950 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f602c04a5f0 con 0x7f60340bb4c0 2021-01-06T22:03:52.153 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.151+0000 7f603a156700 1 -- 172.21.15.203:0/2485419950 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f60340612c0 con 0x7f60340bb4c0 2021-01-06T22:03:52.154 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.151+0000 7f601b7fe700 1 -- 172.21.15.203:0/2485419950 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f602c04a1e0 con 0x7f60340bb4c0 2021-01-06T22:03:52.154 INFO:teuthology.orchestra.run.smithi203.stdout:55834574855 2021-01-06T22:03:52.156 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.155+0000 7f603a156700 1 -- 172.21.15.203:0/2485419950 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f60140563d0 msgr2=0x7f6014058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.156 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.155+0000 7f603a156700 1 --2- 172.21.15.203:0/2485419950 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f60140563d0 0x7f6014058890 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f602800bb50 tx=0x7f6028006de0).stop 2021-01-06T22:03:52.156 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.155+0000 7f603a156700 1 -- 172.21.15.203:0/2485419950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60340bb4c0 msgr2=0x7f60340bf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.157 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.155+0000 7f603a156700 1 --2- 172.21.15.203:0/2485419950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60340bb4c0 0x7f60340bf390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f602c011f80 tx=0x7f602c017000).stop 2021-01-06T22:03:52.157 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.155+0000 7f603a156700 1 -- 172.21.15.203:0/2485419950 shutdown_connections 2021-01-06T22:03:52.157 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.155+0000 7f603a156700 1 --2- 172.21.15.203:0/2485419950 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f60140563d0 0x7f6014058890 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.157 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.155+0000 7f603a156700 1 --2- 172.21.15.203:0/2485419950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60340bb4c0 0x7f60340bf390 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.157 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.155+0000 7f603a156700 1 --2- 172.21.15.203:0/2485419950 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6034064080 0x7f60340b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.155+0000 7f603a156700 1 --2- 172.21.15.203:0/2485419950 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6034064460 0x7f603415dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.155+0000 7f603a156700 1 -- 172.21.15.203:0/2485419950 >> 172.21.15.203:0/2485419950 conn(0x7f6034056cd0 msgr2=0x7f603414e500 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:52.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.155+0000 7f603a156700 1 -- 172.21.15.203:0/2485419950 shutdown_connections 2021-01-06T22:03:52.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.155+0000 7f603a156700 1 -- 172.21.15.203:0/2485419950 wait complete. 2021-01-06T22:03:52.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.5 2021-01-06T22:03:52.169 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2021-01-06T22:03:52.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15f08ad700 1 Processor -- start 2021-01-06T22:03:52.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15f08ad700 1 -- start start 2021-01-06T22:03:52.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15f08ad700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f15ec0bc320 0x7f15ec0646f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15f08ad700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ec14a5a0 con 0x7f15ec064310 2021-01-06T22:03:52.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15f08ad700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ec14a840 con 0x7f15ec0650c0 2021-01-06T22:03:52.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15f08ad700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ec0bb550 con 0x7f15ec0bc320 2021-01-06T22:03:52.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15ea59c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f15ec0650c0 0x7f15ec0bbc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47586/0 (socket says 172.21.15.203:47586) 2021-01-06T22:03:52.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15ea59c700 1 -- 172.21.15.203:0/4013760587 learned_addr learned my addr 172.21.15.203:0/4013760587 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:52.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15eb59e700 1 --2- 172.21.15.203:0/4013760587 >> v2:172.21.15.91:3300/0 conn(0x7f15ec0bc320 0x7f15ec0646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15e9d9b700 1 -- 172.21.15.203:0/4013760587 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585839238 0 0) 0x7f15ec14a840 con 0x7f15ec0650c0 2021-01-06T22:03:52.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15e9d9b700 1 -- 172.21.15.203:0/4013760587 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15d0003230 con 0x7f15ec0650c0 2021-01-06T22:03:52.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15e9d9b700 1 -- 172.21.15.203:0/4013760587 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590585962 0 0) 0x7f15ec14a5a0 con 0x7f15ec064310 2021-01-06T22:03:52.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15e9d9b700 1 -- 172.21.15.203:0/4013760587 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15ec14a840 con 0x7f15ec064310 2021-01-06T22:03:52.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15eb59e700 1 -- 172.21.15.203:0/4013760587 >> v1:172.21.15.203:6789/0 conn(0x7f15ec0650c0 legacy=0x7f15ec0bbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15eb59e700 1 -- 172.21.15.203:0/4013760587 >> v1:172.21.15.91:6789/0 conn(0x7f15ec064310 legacy=0x7f15ec147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15eb59e700 1 -- 172.21.15.203:0/4013760587 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec14aae0 con 0x7f15ec0bc320 2021-01-06T22:03:52.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15eb59e700 1 --2- 172.21.15.203:0/4013760587 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15ec0bc320 0x7f15ec0646f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f15e400b980 tx=0x7f15e4006de0).ready entity=mon.0 client_cookie=e839ea0ec386d408 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:52.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15e9d9b700 1 -- 172.21.15.203:0/4013760587 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e40096d0 con 0x7f15ec0bc320 2021-01-06T22:03:52.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15e9d9b700 1 -- 172.21.15.203:0/4013760587 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e4009830 con 0x7f15ec0bc320 2021-01-06T22:03:52.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.295+0000 7f15e9d9b700 1 -- 172.21.15.203:0/4013760587 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4009bb0 con 0x7f15ec0bc320 2021-01-06T22:03:52.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 -- 172.21.15.203:0/4013760587 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15ec0bc320 msgr2=0x7f15ec0646f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 --2- 172.21.15.203:0/4013760587 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15ec0bc320 0x7f15ec0646f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f15e400b980 tx=0x7f15e4006de0).stop 2021-01-06T22:03:52.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 -- 172.21.15.203:0/4013760587 shutdown_connections 2021-01-06T22:03:52.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 --2- 172.21.15.203:0/4013760587 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15ec0bc320 0x7f15ec0646f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.304 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 -- 172.21.15.203:0/4013760587 >> 172.21.15.203:0/4013760587 conn(0x7f15ec0c08c0 msgr2=0x7f15ec0c0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:52.304 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 -- 172.21.15.203:0/4013760587 shutdown_connections 2021-01-06T22:03:52.304 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 -- 172.21.15.203:0/4013760587 wait complete. 2021-01-06T22:03:52.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 Processor -- start 2021-01-06T22:03:52.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 -- start start 2021-01-06T22:03:52.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15ec064310 0x7f15ec144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15ec0650c0 0x7f15ec14c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15ec0bc320 0x7f15ec145f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec14ee80 con 0x7f15ec064310 2021-01-06T22:03:52.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec0b7e50 con 0x7f15ec0bc320 2021-01-06T22:03:52.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec145180 con 0x7f15ec0650c0 2021-01-06T22:03:52.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15eb59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15ec0bc320 0x7f15ec145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15eb59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15ec0bc320 0x7f15ec145f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48998/0 (socket says 172.21.15.203:48998) 2021-01-06T22:03:52.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15eb59e700 1 -- 172.21.15.203:0/3288741091 learned_addr learned my addr 172.21.15.203:0/3288741091 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:52.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15ead9d700 1 --2- 172.21.15.203:0/3288741091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15ec064310 0x7f15ec144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15ea59c700 1 --2- 172.21.15.203:0/3288741091 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15ec0650c0 0x7f15ec14c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15eb59e700 1 -- 172.21.15.203:0/3288741091 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15ec0650c0 msgr2=0x7f15ec14c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15eb59e700 1 --2- 172.21.15.203:0/3288741091 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15ec0650c0 0x7f15ec14c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15eb59e700 1 -- 172.21.15.203:0/3288741091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15ec064310 msgr2=0x7f15ec144b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15eb59e700 1 --2- 172.21.15.203:0/3288741091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15ec064310 0x7f15ec144b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15eb59e700 1 -- 172.21.15.203:0/3288741091 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec14f0e0 con 0x7f15ec0bc320 2021-01-06T22:03:52.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15eb59e700 1 --2- 172.21.15.203:0/3288741091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15ec0bc320 0x7f15ec145f90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f15e400a010 tx=0x7f15e4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:52.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15e2ffd700 1 -- 172.21.15.203:0/3288741091 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4021070 con 0x7f15ec0bc320 2021-01-06T22:03:52.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15e2ffd700 1 -- 172.21.15.203:0/3288741091 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e401c5a0 con 0x7f15ec0bc320 2021-01-06T22:03:52.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 -- 172.21.15.203:0/3288741091 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ec14f380 con 0x7f15ec0bc320 2021-01-06T22:03:52.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15e2ffd700 1 -- 172.21.15.203:0/3288741091 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e401c920 con 0x7f15ec0bc320 2021-01-06T22:03:52.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.299+0000 7f15f08ad700 1 -- 172.21.15.203:0/3288741091 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ec0b8550 con 0x7f15ec0bc320 2021-01-06T22:03:52.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.303+0000 7f15f08ad700 1 -- 172.21.15.203:0/3288741091 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ec0450a0 con 0x7f15ec0bc320 2021-01-06T22:03:52.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.303+0000 7f15e2ffd700 1 -- 172.21.15.203:0/3288741091 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f15e400f070 con 0x7f15ec0bc320 2021-01-06T22:03:52.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.303+0000 7f15e2ffd700 1 --2- 172.21.15.203:0/3288741091 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f15c40563d0 0x7f15c4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.303+0000 7f15e2ffd700 1 -- 172.21.15.203:0/3288741091 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f15e4070f80 con 0x7f15ec0bc320 2021-01-06T22:03:52.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.303+0000 7f15ead9d700 1 --2- 172.21.15.203:0/3288741091 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f15c40563d0 0x7f15c4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.303+0000 7f15ead9d700 1 --2- 172.21.15.203:0/3288741091 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f15c40563d0 0x7f15c4058890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f15d4000bc0 tx=0x7f15d4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:52.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.307+0000 7f15e2ffd700 1 -- 172.21.15.203:0/3288741091 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f15e4046880 con 0x7f15ec0bc320 2021-01-06T22:03:52.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.459+0000 7f15f08ad700 1 -- 172.21.15.203:0/3288741091 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f15ec05ca60 con 0x7f15ec0bc320 2021-01-06T22:03:52.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.463+0000 7f15e2ffd700 1 -- 172.21.15.203:0/3288741091 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f15e4049ea0 con 0x7f15ec0bc320 2021-01-06T22:03:52.464 INFO:teuthology.orchestra.run.smithi203.stdout:60129542150 2021-01-06T22:03:52.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.463+0000 7f15f08ad700 1 -- 172.21.15.203:0/3288741091 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f15c40563d0 msgr2=0x7f15c4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.463+0000 7f15f08ad700 1 --2- 172.21.15.203:0/3288741091 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f15c40563d0 0x7f15c4058890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f15d4000bc0 tx=0x7f15d4009040).stop 2021-01-06T22:03:52.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.463+0000 7f15f08ad700 1 -- 172.21.15.203:0/3288741091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15ec0bc320 msgr2=0x7f15ec145f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.463+0000 7f15f08ad700 1 --2- 172.21.15.203:0/3288741091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15ec0bc320 0x7f15ec145f90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f15e400a010 tx=0x7f15e4016000).stop 2021-01-06T22:03:52.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.463+0000 7f15f08ad700 1 -- 172.21.15.203:0/3288741091 shutdown_connections 2021-01-06T22:03:52.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.463+0000 7f15f08ad700 1 --2- 172.21.15.203:0/3288741091 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f15c40563d0 0x7f15c4058890 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.463+0000 7f15f08ad700 1 --2- 172.21.15.203:0/3288741091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15ec0bc320 0x7f15ec145f90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.463+0000 7f15f08ad700 1 --2- 172.21.15.203:0/3288741091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15ec064310 0x7f15ec144b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.463+0000 7f15f08ad700 1 --2- 172.21.15.203:0/3288741091 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15ec0650c0 0x7f15ec14c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.463+0000 7f15f08ad700 1 -- 172.21.15.203:0/3288741091 >> 172.21.15.203:0/3288741091 conn(0x7f15ec0c08c0 msgr2=0x7f15ec146ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:52.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.463+0000 7f15f08ad700 1 -- 172.21.15.203:0/3288741091 shutdown_connections 2021-01-06T22:03:52.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.463+0000 7f15f08ad700 1 -- 172.21.15.203:0/3288741091 wait complete. 2021-01-06T22:03:52.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.6 2021-01-06T22:03:52.481 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2021-01-06T22:03:52.612 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37f59e700 1 Processor -- start 2021-01-06T22:03:52.613 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37f59e700 1 -- start start 2021-01-06T22:03:52.613 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37f59e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe3800bb2d0 0x7fe3800ba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.613 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37f59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe380068820 con 0x7fe3800668b0 2021-01-06T22:03:52.613 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37f59e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38014eaa0 con 0x7fe380066c90 2021-01-06T22:03:52.614 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37f59e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe380064880 con 0x7fe3800bb2d0 2021-01-06T22:03:52.614 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37dd9b700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fe380066c90 0x7fe38014adc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47596/0 (socket says 172.21.15.203:47596) 2021-01-06T22:03:52.614 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37dd9b700 1 -- 172.21.15.203:0/2234898136 learned_addr learned my addr 172.21.15.203:0/2234898136 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:52.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37ed9d700 1 --2- 172.21.15.203:0/2234898136 >> v2:172.21.15.91:3300/0 conn(0x7fe3800bb2d0 0x7fe3800ba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37d59a700 1 -- 172.21.15.203:0/2234898136 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345738104 0 0) 0x7fe38014eaa0 con 0x7fe380066c90 2021-01-06T22:03:52.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37d59a700 1 -- 172.21.15.203:0/2234898136 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe364003230 con 0x7fe380066c90 2021-01-06T22:03:52.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37ed9d700 1 -- 172.21.15.203:0/2234898136 >> v1:172.21.15.203:6789/0 conn(0x7fe380066c90 legacy=0x7fe38014adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37ed9d700 1 -- 172.21.15.203:0/2234898136 >> v1:172.21.15.91:6789/0 conn(0x7fe3800668b0 legacy=0x7fe380064170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37ed9d700 1 -- 172.21.15.203:0/2234898136 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3800602b0 con 0x7fe3800bb2d0 2021-01-06T22:03:52.618 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37ed9d700 1 --2- 172.21.15.203:0/2234898136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3800bb2d0 0x7fe3800ba870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fe37000b980 tx=0x7fe370006de0).ready entity=mon.0 client_cookie=fc5b7d1443777a6b server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:52.618 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37d59a700 1 -- 172.21.15.203:0/2234898136 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370009840 con 0x7fe3800bb2d0 2021-01-06T22:03:52.618 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37d59a700 1 -- 172.21.15.203:0/2234898136 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3700099a0 con 0x7fe3800bb2d0 2021-01-06T22:03:52.618 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.611+0000 7fe37d59a700 1 -- 172.21.15.203:0/2234898136 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370009d20 con 0x7fe3800bb2d0 2021-01-06T22:03:52.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37f59e700 1 -- 172.21.15.203:0/2234898136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3800bb2d0 msgr2=0x7fe3800ba870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37f59e700 1 --2- 172.21.15.203:0/2234898136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3800bb2d0 0x7fe3800ba870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fe37000b980 tx=0x7fe370006de0).stop 2021-01-06T22:03:52.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37f59e700 1 -- 172.21.15.203:0/2234898136 shutdown_connections 2021-01-06T22:03:52.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37f59e700 1 --2- 172.21.15.203:0/2234898136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3800bb2d0 0x7fe3800ba870 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37f59e700 1 -- 172.21.15.203:0/2234898136 >> 172.21.15.203:0/2234898136 conn(0x7fe3800c08c0 msgr2=0x7fe3800c0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:52.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37f59e700 1 -- 172.21.15.203:0/2234898136 shutdown_connections 2021-01-06T22:03:52.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37f59e700 1 -- 172.21.15.203:0/2234898136 wait complete. 2021-01-06T22:03:52.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37f59e700 1 Processor -- start 2021-01-06T22:03:52.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37f59e700 1 -- start start 2021-01-06T22:03:52.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37f59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3800668b0 0x7fe3801488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37f59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe380066c90 0x7fe38014bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37f59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3800bb2d0 0x7fe38014c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37f59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe380147920 con 0x7fe3800668b0 2021-01-06T22:03:52.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37f59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe380148da0 con 0x7fe3800bb2d0 2021-01-06T22:03:52.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37f59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38014c670 con 0x7fe380066c90 2021-01-06T22:03:52.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37ed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3800bb2d0 0x7fe38014c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37e59c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3800668b0 0x7fe3801488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37dd9b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe380066c90 0x7fe38014bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37e59c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3800668b0 0x7fe3801488b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:56856/0 (socket says 172.21.15.203:56856) 2021-01-06T22:03:52.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37e59c700 1 -- 172.21.15.203:0/3991191190 learned_addr learned my addr 172.21.15.203:0/3991191190 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:52.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37ed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3800bb2d0 0x7fe38014c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49012/0 (socket says 172.21.15.203:49012) 2021-01-06T22:03:52.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37ed9d700 1 -- 172.21.15.203:0/3991191190 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe380066c90 msgr2=0x7fe38014bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37ed9d700 1 --2- 172.21.15.203:0/3991191190 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe380066c90 0x7fe38014bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37ed9d700 1 -- 172.21.15.203:0/3991191190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3800668b0 msgr2=0x7fe3801488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37ed9d700 1 --2- 172.21.15.203:0/3991191190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3800668b0 0x7fe3801488b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37ed9d700 1 -- 172.21.15.203:0/3991191190 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38014db80 con 0x7fe3800bb2d0 2021-01-06T22:03:52.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.615+0000 7fe37e59c700 1 --2- 172.21.15.203:0/3991191190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3800668b0 0x7fe3801488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:52.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.619+0000 7fe37ed9d700 1 --2- 172.21.15.203:0/3991191190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3800bb2d0 0x7fe38014c180 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fe37000b980 tx=0x7fe370016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:52.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.619+0000 7fe36e7fc700 1 -- 172.21.15.203:0/3991191190 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370020070 con 0x7fe3800bb2d0 2021-01-06T22:03:52.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.619+0000 7fe36e7fc700 1 -- 172.21.15.203:0/3991191190 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37001c420 con 0x7fe3800bb2d0 2021-01-06T22:03:52.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.619+0000 7fe37f59e700 1 -- 172.21.15.203:0/3991191190 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38014f1f0 con 0x7fe3800bb2d0 2021-01-06T22:03:52.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.619+0000 7fe36e7fc700 1 -- 172.21.15.203:0/3991191190 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370009840 con 0x7fe3800bb2d0 2021-01-06T22:03:52.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.619+0000 7fe37f59e700 1 -- 172.21.15.203:0/3991191190 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3800685f0 con 0x7fe3800bb2d0 2021-01-06T22:03:52.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.619+0000 7fe37f59e700 1 -- 172.21.15.203:0/3991191190 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38014f3b0 con 0x7fe3800bb2d0 2021-01-06T22:03:52.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.619+0000 7fe36e7fc700 1 -- 172.21.15.203:0/3991191190 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fe37000f070 con 0x7fe3800bb2d0 2021-01-06T22:03:52.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.619+0000 7fe36e7fc700 1 --2- 172.21.15.203:0/3991191190 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe35c0563d0 0x7fe35c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.619+0000 7fe36e7fc700 1 -- 172.21.15.203:0/3991191190 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe3700707b0 con 0x7fe3800bb2d0 2021-01-06T22:03:52.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.623+0000 7fe37e59c700 1 --2- 172.21.15.203:0/3991191190 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe35c0563d0 0x7fe35c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.623+0000 7fe37e59c700 1 --2- 172.21.15.203:0/3991191190 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe35c0563d0 0x7fe35c058890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fe374001ba0 tx=0x7fe374009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:52.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.623+0000 7fe36e7fc700 1 -- 172.21.15.203:0/3991191190 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fe370046030 con 0x7fe3800bb2d0 2021-01-06T22:03:52.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.779+0000 7fe37f59e700 1 -- 172.21.15.203:0/3991191190 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe3800450a0 con 0x7fe3800bb2d0 2021-01-06T22:03:52.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.779+0000 7fe36e7fc700 1 -- 172.21.15.203:0/3991191190 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe370049650 con 0x7fe3800bb2d0 2021-01-06T22:03:52.782 INFO:teuthology.orchestra.run.smithi203.stdout:60129542150 2021-01-06T22:03:52.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.783+0000 7fe37f59e700 1 -- 172.21.15.203:0/3991191190 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe35c0563d0 msgr2=0x7fe35c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.783+0000 7fe37f59e700 1 --2- 172.21.15.203:0/3991191190 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe35c0563d0 0x7fe35c058890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fe374001ba0 tx=0x7fe374009040).stop 2021-01-06T22:03:52.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.783+0000 7fe37f59e700 1 -- 172.21.15.203:0/3991191190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3800bb2d0 msgr2=0x7fe38014c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.783+0000 7fe37f59e700 1 --2- 172.21.15.203:0/3991191190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3800bb2d0 0x7fe38014c180 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fe37000b980 tx=0x7fe370016000).stop 2021-01-06T22:03:52.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.783+0000 7fe37f59e700 1 -- 172.21.15.203:0/3991191190 shutdown_connections 2021-01-06T22:03:52.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.783+0000 7fe37f59e700 1 --2- 172.21.15.203:0/3991191190 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe35c0563d0 0x7fe35c058890 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.783+0000 7fe37f59e700 1 --2- 172.21.15.203:0/3991191190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3800bb2d0 0x7fe38014c180 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.783+0000 7fe37f59e700 1 --2- 172.21.15.203:0/3991191190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3800668b0 0x7fe3801488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.783+0000 7fe37f59e700 1 --2- 172.21.15.203:0/3991191190 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe380066c90 0x7fe38014bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.783+0000 7fe37f59e700 1 -- 172.21.15.203:0/3991191190 >> 172.21.15.203:0/3991191190 conn(0x7fe3800c08c0 msgr2=0x7fe380149040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:52.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.783+0000 7fe37f59e700 1 -- 172.21.15.203:0/3991191190 shutdown_connections 2021-01-06T22:03:52.787 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.783+0000 7fe37f59e700 1 -- 172.21.15.203:0/3991191190 wait complete. 2021-01-06T22:03:52.797 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2021-01-06T22:03:52.797 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2021-01-06T22:03:52.798 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-06T22:03:52.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.915+0000 7f8ae7ebf700 1 Processor -- start 2021-01-06T22:03:52.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.915+0000 7f8ae7ebf700 1 -- start start 2021-01-06T22:03:52.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae7ebf700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8ae00c04c0 0x7f8ae00680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae7ebf700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8ae00c08a0 0x7f8ae0063ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae7ebf700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ae0155b60 0x7f8ae0156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae7ebf700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae01522e0 con 0x7f8ae00c04c0 2021-01-06T22:03:52.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae7ebf700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae01531f0 con 0x7f8ae00c08a0 2021-01-06T22:03:52.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae7ebf700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae0156500 con 0x7f8ae0155b60 2021-01-06T22:03:52.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae545a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8ae00c08a0 0x7f8ae0063ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae545a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8ae00c08a0 0x7f8ae0063ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49018/0 (socket says 172.21.15.203:49018) 2021-01-06T22:03:52.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae545a700 1 -- 172.21.15.203:0/2889688093 learned_addr learned my addr 172.21.15.203:0/2889688093 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:52.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae5c5b700 1 --2- 172.21.15.203:0/2889688093 >> v2:172.21.15.91:3300/0 conn(0x7f8ae00c04c0 0x7f8ae00680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae645c700 1 --2- 172.21.15.203:0/2889688093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ae0155b60 0x7f8ae0156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae645c700 1 -- 172.21.15.203:0/2889688093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ae0155b60 msgr2=0x7f8ae0156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2021-01-06T22:03:52.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae645c700 1 -- 172.21.15.203:0/2889688093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ae0155b60 msgr2=0x7f8ae0156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:03:52.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae645c700 1 --2- 172.21.15.203:0/2889688093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ae0155b60 0x7f8ae0156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:03:52.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae645c700 1 --2- 172.21.15.203:0/2889688093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ae0155b60 0x7f8ae0156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-01-06T22:03:52.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae545a700 1 -- 172.21.15.203:0/2889688093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ae0155b60 msgr2=0x7f8ae0156010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-01-06T22:03:52.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae545a700 1 --2- 172.21.15.203:0/2889688093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ae0155b60 0x7f8ae0156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae545a700 1 -- 172.21.15.203:0/2889688093 >> v2:172.21.15.91:3300/0 conn(0x7f8ae00c04c0 msgr2=0x7f8ae00680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:52.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae545a700 1 --2- 172.21.15.203:0/2889688093 >> v2:172.21.15.91:3300/0 conn(0x7f8ae00c04c0 0x7f8ae00680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae545a700 1 -- 172.21.15.203:0/2889688093 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae0063a10 con 0x7f8ae00c08a0 2021-01-06T22:03:52.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae5c5b700 1 --2- 172.21.15.203:0/2889688093 >> v2:172.21.15.91:3300/0 conn(0x7f8ae00c04c0 0x7f8ae00680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:52.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae545a700 1 --2- 172.21.15.203:0/2889688093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae00c08a0 0x7f8ae0063ee0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f8ae01533a0 tx=0x7f8ad4006ea0).ready entity=mon.1 client_cookie=4017153a568a191a server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:52.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae4c59700 1 -- 172.21.15.203:0/2889688093 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4009960 con 0x7f8ae00c08a0 2021-01-06T22:03:52.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae4c59700 1 -- 172.21.15.203:0/2889688093 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad4009ac0 con 0x7f8ae00c08a0 2021-01-06T22:03:52.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae4c59700 1 -- 172.21.15.203:0/2889688093 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4009e40 con 0x7f8ae00c08a0 2021-01-06T22:03:52.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae7ebf700 1 -- 172.21.15.203:0/2889688093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae00c08a0 msgr2=0x7f8ae0063ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae7ebf700 1 --2- 172.21.15.203:0/2889688093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae00c08a0 0x7f8ae0063ee0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f8ae01533a0 tx=0x7f8ad4006ea0).stop 2021-01-06T22:03:52.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae7ebf700 1 -- 172.21.15.203:0/2889688093 shutdown_connections 2021-01-06T22:03:52.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae7ebf700 1 --2- 172.21.15.203:0/2889688093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ae0155b60 0x7f8ae0156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae7ebf700 1 --2- 172.21.15.203:0/2889688093 >> v2:172.21.15.91:3300/0 conn(0x7f8ae00c04c0 0x7f8ae00680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae7ebf700 1 --2- 172.21.15.203:0/2889688093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae00c08a0 0x7f8ae0063ee0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae7ebf700 1 -- 172.21.15.203:0/2889688093 >> 172.21.15.203:0/2889688093 conn(0x7f8ae0056cd0 msgr2=0x7f8ae00626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:52.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae7ebf700 1 -- 172.21.15.203:0/2889688093 shutdown_connections 2021-01-06T22:03:52.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.919+0000 7f8ae7ebf700 1 -- 172.21.15.203:0/2889688093 wait complete. 2021-01-06T22:03:52.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae7ebf700 1 Processor -- start 2021-01-06T22:03:52.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae7ebf700 1 -- start start 2021-01-06T22:03:52.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae7ebf700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ae00c04c0 0x7f8ae014bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae7ebf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae00c08a0 0x7f8ae0150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae7ebf700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ae0155b60 0x7f8ae00b8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae7ebf700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae0149be0 con 0x7f8ae00c04c0 2021-01-06T22:03:52.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae7ebf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae0149e80 con 0x7f8ae00c08a0 2021-01-06T22:03:52.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae7ebf700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae014a120 con 0x7f8ae0155b60 2021-01-06T22:03:52.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae545a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae00c08a0 0x7f8ae0150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae545a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae00c08a0 0x7f8ae0150e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49022/0 (socket says 172.21.15.203:49022) 2021-01-06T22:03:52.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae545a700 1 -- 172.21.15.203:0/998002202 learned_addr learned my addr 172.21.15.203:0/998002202 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:52.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae5c5b700 1 --2- 172.21.15.203:0/998002202 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ae00c04c0 0x7f8ae014bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae645c700 1 --2- 172.21.15.203:0/998002202 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ae0155b60 0x7f8ae00b8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae545a700 1 -- 172.21.15.203:0/998002202 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ae0155b60 msgr2=0x7f8ae00b8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae545a700 1 --2- 172.21.15.203:0/998002202 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ae0155b60 0x7f8ae00b8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae545a700 1 -- 172.21.15.203:0/998002202 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ae00c04c0 msgr2=0x7f8ae014bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:52.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae545a700 1 --2- 172.21.15.203:0/998002202 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ae00c04c0 0x7f8ae014bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:52.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae545a700 1 -- 172.21.15.203:0/998002202 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae014a3c0 con 0x7f8ae00c08a0 2021-01-06T22:03:52.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae645c700 1 --2- 172.21.15.203:0/998002202 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ae0155b60 0x7f8ae00b8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:03:52.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae545a700 1 --2- 172.21.15.203:0/998002202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae00c08a0 0x7f8ae0150e40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8ad400b190 tx=0x7f8ad4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:52.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ad1ffb700 1 -- 172.21.15.203:0/998002202 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad401b410 con 0x7f8ae00c08a0 2021-01-06T22:03:52.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ad1ffb700 1 -- 172.21.15.203:0/998002202 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad401b570 con 0x7f8ae00c08a0 2021-01-06T22:03:52.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae7ebf700 1 -- 172.21.15.203:0/998002202 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae014a660 con 0x7f8ae00c08a0 2021-01-06T22:03:52.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ad1ffb700 1 -- 172.21.15.203:0/998002202 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad401b910 con 0x7f8ae00c08a0 2021-01-06T22:03:52.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.923+0000 7f8ae7ebf700 1 -- 172.21.15.203:0/998002202 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae00bb590 con 0x7f8ae00c08a0 2021-01-06T22:03:52.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.927+0000 7f8ae7ebf700 1 -- 172.21.15.203:0/998002202 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae014a7f0 con 0x7f8ae00c08a0 2021-01-06T22:03:52.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.927+0000 7f8ad1ffb700 1 -- 172.21.15.203:0/998002202 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f8ad4021070 con 0x7f8ae00c08a0 2021-01-06T22:03:52.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.927+0000 7f8ad1ffb700 1 --2- 172.21.15.203:0/998002202 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8ac0056400 0x7f8ac00588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:52.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.927+0000 7f8ad1ffb700 1 -- 172.21.15.203:0/998002202 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8ad4070df0 con 0x7f8ae00c08a0 2021-01-06T22:03:52.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.927+0000 7f8ae5c5b700 1 --2- 172.21.15.203:0/998002202 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8ac0056400 0x7f8ac00588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:52.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.927+0000 7f8ae5c5b700 1 --2- 172.21.15.203:0/998002202 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8ac0056400 0x7f8ac00588c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f8adc006fd0 tx=0x7f8adc008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:52.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:52.939+0000 7f8ad1ffb700 1 -- 172.21.15.203:0/998002202 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f8ad4046670 con 0x7f8ae00c08a0 2021-01-06T22:03:53.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.079+0000 7f8ae7ebf700 1 -- 172.21.15.203:0/998002202 --> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ae0064e60 con 0x7f8ac0056400 2021-01-06T22:03:53.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.091+0000 7f8ad1ffb700 1 -- 172.21.15.203:0/998002202 <== mgr.4102 v2:172.21.15.203:6800/18146 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82162 (secure 0 0 0) 0x7f8ae0064e60 con 0x7f8ac0056400 2021-01-06T22:03:53.092 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:53.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.091+0000 7f8ae7ebf700 1 -- 172.21.15.203:0/998002202 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8ac0056400 msgr2=0x7f8ac00588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.091+0000 7f8ae7ebf700 1 --2- 172.21.15.203:0/998002202 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8ac0056400 0x7f8ac00588c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f8adc006fd0 tx=0x7f8adc008040).stop 2021-01-06T22:03:53.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.091+0000 7f8ae7ebf700 1 -- 172.21.15.203:0/998002202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae00c08a0 msgr2=0x7f8ae0150e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.091+0000 7f8ae7ebf700 1 --2- 172.21.15.203:0/998002202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae00c08a0 0x7f8ae0150e40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8ad400b190 tx=0x7f8ad4016000).stop 2021-01-06T22:03:53.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.091+0000 7f8ae7ebf700 1 -- 172.21.15.203:0/998002202 shutdown_connections 2021-01-06T22:03:53.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.091+0000 7f8ae7ebf700 1 --2- 172.21.15.203:0/998002202 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8ac0056400 0x7f8ac00588c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.091+0000 7f8ae7ebf700 1 --2- 172.21.15.203:0/998002202 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ae0155b60 0x7f8ae00b8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.091+0000 7f8ae7ebf700 1 --2- 172.21.15.203:0/998002202 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ae00c04c0 0x7f8ae014bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.091+0000 7f8ae7ebf700 1 --2- 172.21.15.203:0/998002202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae00c08a0 0x7f8ae0150e40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.091+0000 7f8ae7ebf700 1 -- 172.21.15.203:0/998002202 >> 172.21.15.203:0/998002202 conn(0x7f8ae0056cd0 msgr2=0x7f8ae00b99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:53.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.091+0000 7f8ae7ebf700 1 -- 172.21.15.203:0/998002202 shutdown_connections 2021-01-06T22:03:53.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.091+0000 7f8ae7ebf700 1 -- 172.21.15.203:0/998002202 wait complete. 2021-01-06T22:03:53.096 INFO:teuthology.orchestra.run.smithi203.stderr:dumped all 2021-01-06T22:03:53.109 INFO:teuthology.orchestra.run.smithi203.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2021-01-06T22:03:51.094184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4962,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":40988,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":38912,"kb_avail":754933732,"statfs":{"total":773094113280,"available":773052141568,"internally_reserved":0,"allocated":2060288,"data_stored":741074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39845888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4928,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.340217"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.758566+0000","last_change":"2021-01-06T22:03:39.757200+0000","last_active":"2021-01-06T22:03:39.758566+0000","last_peered":"2021-01-06T22:03:39.758566+0000","last_clean":"2021-01-06T22:03:39.758566+0000","last_became_active":"2021-01-06T22:03:39.756706+0000","last_became_peered":"2021-01-06T22:03:39.756706+0000","last_unstale":"2021-01-06T22:03:39.758566+0000","last_undegraded":"2021-01-06T22:03:39.758566+0000","last_fullsized":"2021-01-06T22:03:39.758566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.755461+0000","last_change":"2021-01-06T22:03:36.733731+0000","last_active":"2021-01-06T22:03:39.755461+0000","last_peered":"2021-01-06T22:03:39.755461+0000","last_clean":"2021-01-06T22:03:39.755461+0000","last_became_active":"2021-01-06T22:03:36.733026+0000","last_became_peered":"2021-01-06T22:03:36.733026+0000","last_unstale":"2021-01-06T22:03:39.755461+0000","last_undegraded":"2021-01-06T22:03:39.755461+0000","last_fullsized":"2021-01-06T22:03:39.755461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"23'10","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.806682+0000","last_change":"2021-01-06T22:03:39.372339+0000","last_active":"2021-01-06T22:03:43.806682+0000","last_peered":"2021-01-06T22:03:43.806682+0000","last_clean":"2021-01-06T22:03:43.806682+0000","last_became_active":"2021-01-06T22:03:39.372000+0000","last_became_peered":"2021-01-06T22:03:39.372000+0000","last_unstale":"2021-01-06T22:03:43.806682+0000","last_undegraded":"2021-01-06T22:03:43.806682+0000","last_fullsized":"2021-01-06T22:03:43.806682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":735,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.758556+0000","last_change":"2021-01-06T22:03:39.757152+0000","last_active":"2021-01-06T22:03:39.758556+0000","last_peered":"2021-01-06T22:03:39.758556+0000","last_clean":"2021-01-06T22:03:39.758556+0000","last_became_active":"2021-01-06T22:03:39.756673+0000","last_became_peered":"2021-01-06T22:03:39.756673+0000","last_unstale":"2021-01-06T22:03:39.758556+0000","last_undegraded":"2021-01-06T22:03:39.758556+0000","last_fullsized":"2021-01-06T22:03:39.758556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747142+0000","last_change":"2021-01-06T22:03:36.731052+0000","last_active":"2021-01-06T22:03:39.747142+0000","last_peered":"2021-01-06T22:03:39.747142+0000","last_clean":"2021-01-06T22:03:39.747142+0000","last_became_active":"2021-01-06T22:03:36.730252+0000","last_became_peered":"2021-01-06T22:03:36.730252+0000","last_unstale":"2021-01-06T22:03:39.747142+0000","last_undegraded":"2021-01-06T22:03:39.747142+0000","last_fullsized":"2021-01-06T22:03:39.747142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"23'18","reported_seq":"28","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.807276+0000","last_change":"2021-01-06T22:03:39.371415+0000","last_active":"2021-01-06T22:03:43.807276+0000","last_peered":"2021-01-06T22:03:43.807276+0000","last_clean":"2021-01-06T22:03:43.807276+0000","last_became_active":"2021-01-06T22:03:39.370981+0000","last_became_peered":"2021-01-06T22:03:39.370981+0000","last_unstale":"2021-01-06T22:03:43.807276+0000","last_undegraded":"2021-01-06T22:03:43.807276+0000","last_fullsized":"2021-01-06T22:03:43.807276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":642,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.776586+0000","last_change":"2021-01-06T22:03:39.775845+0000","last_active":"2021-01-06T22:03:39.776586+0000","last_peered":"2021-01-06T22:03:39.776586+0000","last_clean":"2021-01-06T22:03:39.776586+0000","last_became_active":"2021-01-06T22:03:39.775329+0000","last_became_peered":"2021-01-06T22:03:39.775329+0000","last_unstale":"2021-01-06T22:03:39.776586+0000","last_undegraded":"2021-01-06T22:03:39.776586+0000","last_fullsized":"2021-01-06T22:03:39.776586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.751998+0000","last_change":"2021-01-06T22:03:36.209909+0000","last_active":"2021-01-06T22:03:39.751998+0000","last_peered":"2021-01-06T22:03:39.751998+0000","last_clean":"2021-01-06T22:03:39.751998+0000","last_became_active":"2021-01-06T22:03:36.209405+0000","last_became_peered":"2021-01-06T22:03:36.209405+0000","last_unstale":"2021-01-06T22:03:39.751998+0000","last_undegraded":"2021-01-06T22:03:39.751998+0000","last_fullsized":"2021-01-06T22:03:39.751998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"23'21","reported_seq":"31","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.819440+0000","last_change":"2021-01-06T22:03:39.374989+0000","last_active":"2021-01-06T22:03:43.819440+0000","last_peered":"2021-01-06T22:03:43.819440+0000","last_clean":"2021-01-06T22:03:43.819440+0000","last_became_active":"2021-01-06T22:03:39.374506+0000","last_became_peered":"2021-01-06T22:03:39.374506+0000","last_unstale":"2021-01-06T22:03:43.819440+0000","last_undegraded":"2021-01-06T22:03:43.819440+0000","last_fullsized":"2021-01-06T22:03:43.819440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":112,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:40.227173+0000","last_change":"2021-01-06T22:03:40.226400+0000","last_active":"2021-01-06T22:03:40.227173+0000","last_peered":"2021-01-06T22:03:40.227173+0000","last_clean":"2021-01-06T22:03:40.227173+0000","last_became_active":"2021-01-06T22:03:40.225932+0000","last_became_peered":"2021-01-06T22:03:40.225932+0000","last_unstale":"2021-01-06T22:03:40.227173+0000","last_undegraded":"2021-01-06T22:03:40.227173+0000","last_fullsized":"2021-01-06T22:03:40.227173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747380+0000","last_change":"2021-01-06T22:03:36.243205+0000","last_active":"2021-01-06T22:03:39.747380+0000","last_peered":"2021-01-06T22:03:39.747380+0000","last_clean":"2021-01-06T22:03:39.747380+0000","last_became_active":"2021-01-06T22:03:36.242610+0000","last_became_peered":"2021-01-06T22:03:36.242610+0000","last_unstale":"2021-01-06T22:03:39.747380+0000","last_undegraded":"2021-01-06T22:03:39.747380+0000","last_fullsized":"2021-01-06T22:03:39.747380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"17","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:40.222155+0000","last_change":"2021-01-06T22:03:32.172575+0000","last_active":"2021-01-06T22:03:40.222155+0000","last_peered":"2021-01-06T22:03:40.222155+0000","last_clean":"2021-01-06T22:03:40.222155+0000","last_became_active":"2021-01-06T22:03:32.171025+0000","last_became_peered":"2021-01-06T22:03:32.171025+0000","last_unstale":"2021-01-06T22:03:40.222155+0000","last_undegraded":"2021-01-06T22:03:40.222155+0000","last_fullsized":"2021-01-06T22:03:40.222155+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":8,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:25.162227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:25.162227+0000","last_clean_scrub_stamp":"2021-01-06T22:03:25.162227+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"23'8","reported_seq":"18","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.815252+0000","last_change":"2021-01-06T22:03:39.374385+0000","last_active":"2021-01-06T22:03:43.815252+0000","last_peered":"2021-01-06T22:03:43.815252+0000","last_clean":"2021-01-06T22:03:43.815252+0000","last_became_active":"2021-01-06T22:03:39.374029+0000","last_became_peered":"2021-01-06T22:03:39.374029+0000","last_unstale":"2021-01-06T22:03:43.815252+0000","last_undegraded":"2021-01-06T22:03:43.815252+0000","last_fullsized":"2021-01-06T22:03:43.815252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.765901+0000","last_change":"2021-01-06T22:03:39.762496+0000","last_active":"2021-01-06T22:03:39.765901+0000","last_peered":"2021-01-06T22:03:39.765901+0000","last_clean":"2021-01-06T22:03:39.765901+0000","last_became_active":"2021-01-06T22:03:39.761903+0000","last_became_peered":"2021-01-06T22:03:39.761903+0000","last_unstale":"2021-01-06T22:03:39.765901+0000","last_undegraded":"2021-01-06T22:03:39.765901+0000","last_fullsized":"2021-01-06T22:03:39.765901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747295+0000","last_change":"2021-01-06T22:03:36.206609+0000","last_active":"2021-01-06T22:03:39.747295+0000","last_peered":"2021-01-06T22:03:39.747295+0000","last_clean":"2021-01-06T22:03:39.747295+0000","last_became_active":"2021-01-06T22:03:36.206123+0000","last_became_peered":"2021-01-06T22:03:36.206123+0000","last_unstale":"2021-01-06T22:03:39.747295+0000","last_undegraded":"2021-01-06T22:03:39.747295+0000","last_fullsized":"2021-01-06T22:03:39.747295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"23'12","reported_seq":"22","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.821871+0000","last_change":"2021-01-06T22:03:39.377897+0000","last_active":"2021-01-06T22:03:43.821871+0000","last_peered":"2021-01-06T22:03:43.821871+0000","last_clean":"2021-01-06T22:03:43.821871+0000","last_became_active":"2021-01-06T22:03:39.376836+0000","last_became_peered":"2021-01-06T22:03:39.376836+0000","last_unstale":"2021-01-06T22:03:43.821871+0000","last_undegraded":"2021-01-06T22:03:43.821871+0000","last_fullsized":"2021-01-06T22:03:43.821871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":720,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.778763+0000","last_change":"2021-01-06T22:03:39.777501+0000","last_active":"2021-01-06T22:03:39.778763+0000","last_peered":"2021-01-06T22:03:39.778763+0000","last_clean":"2021-01-06T22:03:39.778763+0000","last_became_active":"2021-01-06T22:03:39.777158+0000","last_became_peered":"2021-01-06T22:03:39.777158+0000","last_unstale":"2021-01-06T22:03:39.778763+0000","last_undegraded":"2021-01-06T22:03:39.778763+0000","last_fullsized":"2021-01-06T22:03:39.778763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747656+0000","last_change":"2021-01-06T22:03:36.208974+0000","last_active":"2021-01-06T22:03:39.747656+0000","last_peered":"2021-01-06T22:03:39.747656+0000","last_clean":"2021-01-06T22:03:39.747656+0000","last_became_active":"2021-01-06T22:03:36.208576+0000","last_became_peered":"2021-01-06T22:03:36.208576+0000","last_unstale":"2021-01-06T22:03:39.747656+0000","last_undegraded":"2021-01-06T22:03:39.747656+0000","last_fullsized":"2021-01-06T22:03:39.747656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"23'13","reported_seq":"23","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.816002+0000","last_change":"2021-01-06T22:03:39.375127+0000","last_active":"2021-01-06T22:03:43.816002+0000","last_peered":"2021-01-06T22:03:43.816002+0000","last_clean":"2021-01-06T22:03:43.816002+0000","last_became_active":"2021-01-06T22:03:39.374751+0000","last_became_peered":"2021-01-06T22:03:39.374751+0000","last_unstale":"2021-01-06T22:03:43.816002+0000","last_undegraded":"2021-01-06T22:03:43.816002+0000","last_fullsized":"2021-01-06T22:03:43.816002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2088,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:40.232168+0000","last_change":"2021-01-06T22:03:40.230887+0000","last_active":"2021-01-06T22:03:40.232168+0000","last_peered":"2021-01-06T22:03:40.232168+0000","last_clean":"2021-01-06T22:03:40.232168+0000","last_became_active":"2021-01-06T22:03:40.230100+0000","last_became_peered":"2021-01-06T22:03:40.230100+0000","last_unstale":"2021-01-06T22:03:40.232168+0000","last_undegraded":"2021-01-06T22:03:40.232168+0000","last_fullsized":"2021-01-06T22:03:40.232168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"23'17","reported_seq":"27","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.818982+0000","last_change":"2021-01-06T22:03:39.377489+0000","last_active":"2021-01-06T22:03:43.818982+0000","last_peered":"2021-01-06T22:03:43.818982+0000","last_clean":"2021-01-06T22:03:43.818982+0000","last_became_active":"2021-01-06T22:03:39.377092+0000","last_became_peered":"2021-01-06T22:03:39.377092+0000","last_unstale":"2021-01-06T22:03:43.818982+0000","last_undegraded":"2021-01-06T22:03:43.818982+0000","last_fullsized":"2021-01-06T22:03:43.818982+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":665,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.759458+0000","last_change":"2021-01-06T22:03:36.213113+0000","last_active":"2021-01-06T22:03:39.759458+0000","last_peered":"2021-01-06T22:03:39.759458+0000","last_clean":"2021-01-06T22:03:39.759458+0000","last_became_active":"2021-01-06T22:03:36.212790+0000","last_became_peered":"2021-01-06T22:03:36.212790+0000","last_unstale":"2021-01-06T22:03:39.759458+0000","last_undegraded":"2021-01-06T22:03:39.759458+0000","last_fullsized":"2021-01-06T22:03:39.759458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.764030+0000","last_change":"2021-01-06T22:03:39.759582+0000","last_active":"2021-01-06T22:03:39.764030+0000","last_peered":"2021-01-06T22:03:39.764030+0000","last_clean":"2021-01-06T22:03:39.764030+0000","last_became_active":"2021-01-06T22:03:39.759212+0000","last_became_peered":"2021-01-06T22:03:39.759212+0000","last_unstale":"2021-01-06T22:03:39.764030+0000","last_undegraded":"2021-01-06T22:03:39.764030+0000","last_fullsized":"2021-01-06T22:03:39.764030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747367+0000","last_change":"2021-01-06T22:03:36.207481+0000","last_active":"2021-01-06T22:03:39.747367+0000","last_peered":"2021-01-06T22:03:39.747367+0000","last_clean":"2021-01-06T22:03:39.747367+0000","last_became_active":"2021-01-06T22:03:36.206991+0000","last_became_peered":"2021-01-06T22:03:36.206991+0000","last_unstale":"2021-01-06T22:03:39.747367+0000","last_undegraded":"2021-01-06T22:03:39.747367+0000","last_fullsized":"2021-01-06T22:03:39.747367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'9","reported_seq":"19","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.813735+0000","last_change":"2021-01-06T22:03:39.370350+0000","last_active":"2021-01-06T22:03:43.813735+0000","last_peered":"2021-01-06T22:03:43.813735+0000","last_clean":"2021-01-06T22:03:43.813735+0000","last_became_active":"2021-01-06T22:03:39.369929+0000","last_became_peered":"2021-01-06T22:03:39.369929+0000","last_unstale":"2021-01-06T22:03:43.813735+0000","last_undegraded":"2021-01-06T22:03:43.813735+0000","last_fullsized":"2021-01-06T22:03:43.813735+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":4962,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":9924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5112,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366728,"statfs":{"total":96636764160,"available":96631529472,"internally_reserved":0,"allocated":245760,"data_stored":91578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3330000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.913}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1789999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2490000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8290000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3610000000000002}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5224,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366616,"statfs":{"total":96636764160,"available":96631414784,"internally_reserved":0,"allocated":294912,"data_stored":95674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9570000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2440000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4990000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3069999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8260000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8690000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9239999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5128,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366712,"statfs":{"total":96636764160,"available":96631513088,"internally_reserved":0,"allocated":262144,"data_stored":93666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.758}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4380000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6139999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8520000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6219999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7839999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8690000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5116,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366724,"statfs":{"total":96636764160,"available":96631525376,"internally_reserved":0,"allocated":249856,"data_stored":91530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.613}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.056}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.74}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.452}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.548}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6110000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7869999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":237568,"data_stored":91466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7490000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5549999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.889}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":126.69499999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3980000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5990000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5148,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366692,"statfs":{"total":96636764160,"available":96631492608,"internally_reserved":0,"allocated":282624,"data_stored":93563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.722}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0270000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9239999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8420000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9409999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.806}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5104,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366736,"statfs":{"total":96636764160,"available":96631537664,"internally_reserved":0,"allocated":237568,"data_stored":91466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.407}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":264.72500000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.355}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.024}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7029999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9489999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5116,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366724,"statfs":{"total":96636764160,"available":96631525376,"internally_reserved":0,"allocated":249856,"data_stored":92131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.419}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9530000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8660000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4140000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9590000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.556}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5209999999999999}]}]}]}} 2021-01-06T22:03:53.111 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-06T22:03:53.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a3d1ed700 1 Processor -- start 2021-01-06T22:03:53.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a3d1ed700 1 -- start start 2021-01-06T22:03:53.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a3d1ed700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6a380c44f0 0x7f6a380c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a3d1ed700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a380c48d0 0x7f6a380c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a3d1ed700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a380c0f50 con 0x7f6a38060fc0 2021-01-06T22:03:53.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a3d1ed700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a380c1d60 con 0x7f6a380c44f0 2021-01-06T22:03:53.234 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a3d1ed700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a38066cf0 con 0x7f6a380c48d0 2021-01-06T22:03:53.235 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a377fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a380c48d0 0x7f6a380c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.235 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a37fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6a380c44f0 0x7f6a380c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.236 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a3c9ec700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f6a38060fc0 0x7f6a38150730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:42282/0 (socket says 172.21.15.203:42282) 2021-01-06T22:03:53.236 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a3c9ec700 1 -- 172.21.15.203:0/2544141134 learned_addr learned my addr 172.21.15.203:0/2544141134 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:53.236 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a37fff700 1 -- 172.21.15.203:0/2544141134 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a380c48d0 msgr2=0x7f6a380c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:53.236 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a37fff700 1 --2- 172.21.15.203:0/2544141134 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a380c48d0 0x7f6a380c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.237 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a37fff700 1 -- 172.21.15.203:0/2544141134 >> v1:172.21.15.91:6789/0 conn(0x7f6a38060fc0 legacy=0x7f6a38150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.237 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a37fff700 1 -- 172.21.15.203:0/2544141134 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a38060750 con 0x7f6a380c44f0 2021-01-06T22:03:53.237 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a36ffd700 1 -- 172.21.15.203:0/2544141134 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562665804 0 0) 0x7f6a380c0f50 con 0x7f6a38060fc0 2021-01-06T22:03:53.237 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a377fe700 1 --2- 172.21.15.203:0/2544141134 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a380c48d0 0x7f6a380c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:53.238 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a37fff700 1 --2- 172.21.15.203:0/2544141134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a380c44f0 0x7f6a380c1380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f6a28009c90 tx=0x7f6a280091d0).ready entity=mon.0 client_cookie=107354bf45b4d062 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:53.238 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a36ffd700 1 -- 172.21.15.203:0/2544141134 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2800ba70 con 0x7f6a380c44f0 2021-01-06T22:03:53.238 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a36ffd700 1 -- 172.21.15.203:0/2544141134 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2800bbd0 con 0x7f6a380c44f0 2021-01-06T22:03:53.238 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.231+0000 7f6a36ffd700 1 -- 172.21.15.203:0/2544141134 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28011400 con 0x7f6a380c44f0 2021-01-06T22:03:53.238 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3d1ed700 1 -- 172.21.15.203:0/2544141134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a380c44f0 msgr2=0x7f6a380c1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3d1ed700 1 --2- 172.21.15.203:0/2544141134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a380c44f0 0x7f6a380c1380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f6a28009c90 tx=0x7f6a280091d0).stop 2021-01-06T22:03:53.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3d1ed700 1 -- 172.21.15.203:0/2544141134 shutdown_connections 2021-01-06T22:03:53.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3d1ed700 1 --2- 172.21.15.203:0/2544141134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a380c44f0 0x7f6a380c1380 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3d1ed700 1 --2- 172.21.15.203:0/2544141134 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a380c48d0 0x7f6a380c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.240 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3d1ed700 1 -- 172.21.15.203:0/2544141134 >> 172.21.15.203:0/2544141134 conn(0x7f6a38056cd0 msgr2=0x7f6a38061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:53.240 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3d1ed700 1 -- 172.21.15.203:0/2544141134 shutdown_connections 2021-01-06T22:03:53.240 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3d1ed700 1 -- 172.21.15.203:0/2544141134 wait complete. 2021-01-06T22:03:53.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3d1ed700 1 Processor -- start 2021-01-06T22:03:53.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3d1ed700 1 -- start start 2021-01-06T22:03:53.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3d1ed700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a38060fc0 0x7f6a38151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3d1ed700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a380c44f0 0x7f6a380b9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3d1ed700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a380c48d0 0x7f6a3814a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3d1ed700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a380b9c10 con 0x7f6a38060fc0 2021-01-06T22:03:53.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3d1ed700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a38153d00 con 0x7f6a380c48d0 2021-01-06T22:03:53.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3d1ed700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a38153fa0 con 0x7f6a380c44f0 2021-01-06T22:03:53.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3c9ec700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a380c48d0 0x7f6a3814a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3c9ec700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a380c48d0 0x7f6a3814a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49040/0 (socket says 172.21.15.203:49040) 2021-01-06T22:03:53.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.235+0000 7f6a3c9ec700 1 -- 172.21.15.203:0/4148138198 learned_addr learned my addr 172.21.15.203:0/4148138198 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:53.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.239+0000 7f6a37fff700 1 --2- 172.21.15.203:0/4148138198 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a38060fc0 0x7f6a38151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.239+0000 7f6a377fe700 1 --2- 172.21.15.203:0/4148138198 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a380c44f0 0x7f6a380b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.239+0000 7f6a3c9ec700 1 -- 172.21.15.203:0/4148138198 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a380c44f0 msgr2=0x7f6a380b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.239+0000 7f6a3c9ec700 1 --2- 172.21.15.203:0/4148138198 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a380c44f0 0x7f6a380b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.239+0000 7f6a3c9ec700 1 -- 172.21.15.203:0/4148138198 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a38060fc0 msgr2=0x7f6a38151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.239+0000 7f6a3c9ec700 1 --2- 172.21.15.203:0/4148138198 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a38060fc0 0x7f6a38151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.239+0000 7f6a3c9ec700 1 -- 172.21.15.203:0/4148138198 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a38154240 con 0x7f6a380c48d0 2021-01-06T22:03:53.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.239+0000 7f6a3c9ec700 1 --2- 172.21.15.203:0/4148138198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a380c48d0 0x7f6a3814a390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f6a3000c760 tx=0x7f6a300079f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:53.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.239+0000 7f6a2ffff700 1 -- 172.21.15.203:0/4148138198 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a300072e0 con 0x7f6a380c48d0 2021-01-06T22:03:53.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.239+0000 7f6a2ffff700 1 -- 172.21.15.203:0/4148138198 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a30007440 con 0x7f6a380c48d0 2021-01-06T22:03:53.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.239+0000 7f6a3d1ed700 1 -- 172.21.15.203:0/4148138198 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a38152ed0 con 0x7f6a380c48d0 2021-01-06T22:03:53.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.239+0000 7f6a2ffff700 1 -- 172.21.15.203:0/4148138198 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a300057f0 con 0x7f6a380c48d0 2021-01-06T22:03:53.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.239+0000 7f6a3d1ed700 1 -- 172.21.15.203:0/4148138198 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a38150590 con 0x7f6a380c48d0 2021-01-06T22:03:53.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.239+0000 7f6a3d1ed700 1 -- 172.21.15.203:0/4148138198 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a380450a0 con 0x7f6a380c48d0 2021-01-06T22:03:53.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.239+0000 7f6a2ffff700 1 -- 172.21.15.203:0/4148138198 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f6a30010070 con 0x7f6a380c48d0 2021-01-06T22:03:53.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.239+0000 7f6a2ffff700 1 --2- 172.21.15.203:0/4148138198 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a1c0563d0 0x7f6a1c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.243+0000 7f6a2ffff700 1 -- 172.21.15.203:0/4148138198 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6a3006ec20 con 0x7f6a380c48d0 2021-01-06T22:03:53.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.243+0000 7f6a37fff700 1 --2- 172.21.15.203:0/4148138198 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a1c0563d0 0x7f6a1c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.243+0000 7f6a37fff700 1 --2- 172.21.15.203:0/4148138198 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a1c0563d0 0x7f6a1c058890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f6a2800b770 tx=0x7f6a28017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:53.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.247+0000 7f6a2ffff700 1 -- 172.21.15.203:0/4148138198 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f6a300444a0 con 0x7f6a380c48d0 2021-01-06T22:03:53.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.395+0000 7f6a3d1ed700 1 -- 172.21.15.203:0/4148138198 --> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a3814ff60 con 0x7f6a1c0563d0 2021-01-06T22:03:53.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.399+0000 7f6a2ffff700 1 -- 172.21.15.203:0/4148138198 <== mgr.4102 v2:172.21.15.203:6800/18146 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82153 (secure 0 0 0) 0x7f6a3814ff60 con 0x7f6a1c0563d0 2021-01-06T22:03:53.401 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:53.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.399+0000 7f6a3d1ed700 1 -- 172.21.15.203:0/4148138198 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a1c0563d0 msgr2=0x7f6a1c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.399+0000 7f6a3d1ed700 1 --2- 172.21.15.203:0/4148138198 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a1c0563d0 0x7f6a1c058890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f6a2800b770 tx=0x7f6a28017000).stop 2021-01-06T22:03:53.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.399+0000 7f6a3d1ed700 1 -- 172.21.15.203:0/4148138198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a380c48d0 msgr2=0x7f6a3814a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.399+0000 7f6a3d1ed700 1 --2- 172.21.15.203:0/4148138198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a380c48d0 0x7f6a3814a390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f6a3000c760 tx=0x7f6a300079f0).stop 2021-01-06T22:03:53.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.399+0000 7f6a3d1ed700 1 -- 172.21.15.203:0/4148138198 shutdown_connections 2021-01-06T22:03:53.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.399+0000 7f6a3d1ed700 1 --2- 172.21.15.203:0/4148138198 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a1c0563d0 0x7f6a1c058890 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.399+0000 7f6a3d1ed700 1 --2- 172.21.15.203:0/4148138198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a380c48d0 0x7f6a3814a390 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.399+0000 7f6a3d1ed700 1 --2- 172.21.15.203:0/4148138198 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a38060fc0 0x7f6a38151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.399+0000 7f6a3d1ed700 1 --2- 172.21.15.203:0/4148138198 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a380c44f0 0x7f6a380b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.399+0000 7f6a3d1ed700 1 -- 172.21.15.203:0/4148138198 >> 172.21.15.203:0/4148138198 conn(0x7f6a38056cd0 msgr2=0x7f6a380bc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:53.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.399+0000 7f6a3d1ed700 1 -- 172.21.15.203:0/4148138198 shutdown_connections 2021-01-06T22:03:53.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.399+0000 7f6a3d1ed700 1 -- 172.21.15.203:0/4148138198 wait complete. 2021-01-06T22:03:53.406 INFO:teuthology.orchestra.run.smithi203.stderr:dumped all 2021-01-06T22:03:53.415 INFO:teuthology.orchestra.run.smithi203.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2021-01-06T22:03:53.094775+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4962,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":40988,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":38912,"kb_avail":754933732,"statfs":{"total":773094113280,"available":773052141568,"internally_reserved":0,"allocated":2060288,"data_stored":741074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39845888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4928,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003701"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.758566+0000","last_change":"2021-01-06T22:03:39.757200+0000","last_active":"2021-01-06T22:03:39.758566+0000","last_peered":"2021-01-06T22:03:39.758566+0000","last_clean":"2021-01-06T22:03:39.758566+0000","last_became_active":"2021-01-06T22:03:39.756706+0000","last_became_peered":"2021-01-06T22:03:39.756706+0000","last_unstale":"2021-01-06T22:03:39.758566+0000","last_undegraded":"2021-01-06T22:03:39.758566+0000","last_fullsized":"2021-01-06T22:03:39.758566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.755461+0000","last_change":"2021-01-06T22:03:36.733731+0000","last_active":"2021-01-06T22:03:39.755461+0000","last_peered":"2021-01-06T22:03:39.755461+0000","last_clean":"2021-01-06T22:03:39.755461+0000","last_became_active":"2021-01-06T22:03:36.733026+0000","last_became_peered":"2021-01-06T22:03:36.733026+0000","last_unstale":"2021-01-06T22:03:39.755461+0000","last_undegraded":"2021-01-06T22:03:39.755461+0000","last_fullsized":"2021-01-06T22:03:39.755461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"23'10","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.806682+0000","last_change":"2021-01-06T22:03:39.372339+0000","last_active":"2021-01-06T22:03:43.806682+0000","last_peered":"2021-01-06T22:03:43.806682+0000","last_clean":"2021-01-06T22:03:43.806682+0000","last_became_active":"2021-01-06T22:03:39.372000+0000","last_became_peered":"2021-01-06T22:03:39.372000+0000","last_unstale":"2021-01-06T22:03:43.806682+0000","last_undegraded":"2021-01-06T22:03:43.806682+0000","last_fullsized":"2021-01-06T22:03:43.806682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":735,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.758556+0000","last_change":"2021-01-06T22:03:39.757152+0000","last_active":"2021-01-06T22:03:39.758556+0000","last_peered":"2021-01-06T22:03:39.758556+0000","last_clean":"2021-01-06T22:03:39.758556+0000","last_became_active":"2021-01-06T22:03:39.756673+0000","last_became_peered":"2021-01-06T22:03:39.756673+0000","last_unstale":"2021-01-06T22:03:39.758556+0000","last_undegraded":"2021-01-06T22:03:39.758556+0000","last_fullsized":"2021-01-06T22:03:39.758556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747142+0000","last_change":"2021-01-06T22:03:36.731052+0000","last_active":"2021-01-06T22:03:39.747142+0000","last_peered":"2021-01-06T22:03:39.747142+0000","last_clean":"2021-01-06T22:03:39.747142+0000","last_became_active":"2021-01-06T22:03:36.730252+0000","last_became_peered":"2021-01-06T22:03:36.730252+0000","last_unstale":"2021-01-06T22:03:39.747142+0000","last_undegraded":"2021-01-06T22:03:39.747142+0000","last_fullsized":"2021-01-06T22:03:39.747142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"23'18","reported_seq":"28","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.807276+0000","last_change":"2021-01-06T22:03:39.371415+0000","last_active":"2021-01-06T22:03:43.807276+0000","last_peered":"2021-01-06T22:03:43.807276+0000","last_clean":"2021-01-06T22:03:43.807276+0000","last_became_active":"2021-01-06T22:03:39.370981+0000","last_became_peered":"2021-01-06T22:03:39.370981+0000","last_unstale":"2021-01-06T22:03:43.807276+0000","last_undegraded":"2021-01-06T22:03:43.807276+0000","last_fullsized":"2021-01-06T22:03:43.807276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":642,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.776586+0000","last_change":"2021-01-06T22:03:39.775845+0000","last_active":"2021-01-06T22:03:39.776586+0000","last_peered":"2021-01-06T22:03:39.776586+0000","last_clean":"2021-01-06T22:03:39.776586+0000","last_became_active":"2021-01-06T22:03:39.775329+0000","last_became_peered":"2021-01-06T22:03:39.775329+0000","last_unstale":"2021-01-06T22:03:39.776586+0000","last_undegraded":"2021-01-06T22:03:39.776586+0000","last_fullsized":"2021-01-06T22:03:39.776586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.751998+0000","last_change":"2021-01-06T22:03:36.209909+0000","last_active":"2021-01-06T22:03:39.751998+0000","last_peered":"2021-01-06T22:03:39.751998+0000","last_clean":"2021-01-06T22:03:39.751998+0000","last_became_active":"2021-01-06T22:03:36.209405+0000","last_became_peered":"2021-01-06T22:03:36.209405+0000","last_unstale":"2021-01-06T22:03:39.751998+0000","last_undegraded":"2021-01-06T22:03:39.751998+0000","last_fullsized":"2021-01-06T22:03:39.751998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"23'21","reported_seq":"31","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.819440+0000","last_change":"2021-01-06T22:03:39.374989+0000","last_active":"2021-01-06T22:03:43.819440+0000","last_peered":"2021-01-06T22:03:43.819440+0000","last_clean":"2021-01-06T22:03:43.819440+0000","last_became_active":"2021-01-06T22:03:39.374506+0000","last_became_peered":"2021-01-06T22:03:39.374506+0000","last_unstale":"2021-01-06T22:03:43.819440+0000","last_undegraded":"2021-01-06T22:03:43.819440+0000","last_fullsized":"2021-01-06T22:03:43.819440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":112,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:40.227173+0000","last_change":"2021-01-06T22:03:40.226400+0000","last_active":"2021-01-06T22:03:40.227173+0000","last_peered":"2021-01-06T22:03:40.227173+0000","last_clean":"2021-01-06T22:03:40.227173+0000","last_became_active":"2021-01-06T22:03:40.225932+0000","last_became_peered":"2021-01-06T22:03:40.225932+0000","last_unstale":"2021-01-06T22:03:40.227173+0000","last_undegraded":"2021-01-06T22:03:40.227173+0000","last_fullsized":"2021-01-06T22:03:40.227173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747380+0000","last_change":"2021-01-06T22:03:36.243205+0000","last_active":"2021-01-06T22:03:39.747380+0000","last_peered":"2021-01-06T22:03:39.747380+0000","last_clean":"2021-01-06T22:03:39.747380+0000","last_became_active":"2021-01-06T22:03:36.242610+0000","last_became_peered":"2021-01-06T22:03:36.242610+0000","last_unstale":"2021-01-06T22:03:39.747380+0000","last_undegraded":"2021-01-06T22:03:39.747380+0000","last_fullsized":"2021-01-06T22:03:39.747380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"17","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:40.222155+0000","last_change":"2021-01-06T22:03:32.172575+0000","last_active":"2021-01-06T22:03:40.222155+0000","last_peered":"2021-01-06T22:03:40.222155+0000","last_clean":"2021-01-06T22:03:40.222155+0000","last_became_active":"2021-01-06T22:03:32.171025+0000","last_became_peered":"2021-01-06T22:03:32.171025+0000","last_unstale":"2021-01-06T22:03:40.222155+0000","last_undegraded":"2021-01-06T22:03:40.222155+0000","last_fullsized":"2021-01-06T22:03:40.222155+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":8,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:25.162227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:25.162227+0000","last_clean_scrub_stamp":"2021-01-06T22:03:25.162227+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"23'8","reported_seq":"18","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.815252+0000","last_change":"2021-01-06T22:03:39.374385+0000","last_active":"2021-01-06T22:03:43.815252+0000","last_peered":"2021-01-06T22:03:43.815252+0000","last_clean":"2021-01-06T22:03:43.815252+0000","last_became_active":"2021-01-06T22:03:39.374029+0000","last_became_peered":"2021-01-06T22:03:39.374029+0000","last_unstale":"2021-01-06T22:03:43.815252+0000","last_undegraded":"2021-01-06T22:03:43.815252+0000","last_fullsized":"2021-01-06T22:03:43.815252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.765901+0000","last_change":"2021-01-06T22:03:39.762496+0000","last_active":"2021-01-06T22:03:39.765901+0000","last_peered":"2021-01-06T22:03:39.765901+0000","last_clean":"2021-01-06T22:03:39.765901+0000","last_became_active":"2021-01-06T22:03:39.761903+0000","last_became_peered":"2021-01-06T22:03:39.761903+0000","last_unstale":"2021-01-06T22:03:39.765901+0000","last_undegraded":"2021-01-06T22:03:39.765901+0000","last_fullsized":"2021-01-06T22:03:39.765901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747295+0000","last_change":"2021-01-06T22:03:36.206609+0000","last_active":"2021-01-06T22:03:39.747295+0000","last_peered":"2021-01-06T22:03:39.747295+0000","last_clean":"2021-01-06T22:03:39.747295+0000","last_became_active":"2021-01-06T22:03:36.206123+0000","last_became_peered":"2021-01-06T22:03:36.206123+0000","last_unstale":"2021-01-06T22:03:39.747295+0000","last_undegraded":"2021-01-06T22:03:39.747295+0000","last_fullsized":"2021-01-06T22:03:39.747295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"23'12","reported_seq":"22","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.821871+0000","last_change":"2021-01-06T22:03:39.377897+0000","last_active":"2021-01-06T22:03:43.821871+0000","last_peered":"2021-01-06T22:03:43.821871+0000","last_clean":"2021-01-06T22:03:43.821871+0000","last_became_active":"2021-01-06T22:03:39.376836+0000","last_became_peered":"2021-01-06T22:03:39.376836+0000","last_unstale":"2021-01-06T22:03:43.821871+0000","last_undegraded":"2021-01-06T22:03:43.821871+0000","last_fullsized":"2021-01-06T22:03:43.821871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":720,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.778763+0000","last_change":"2021-01-06T22:03:39.777501+0000","last_active":"2021-01-06T22:03:39.778763+0000","last_peered":"2021-01-06T22:03:39.778763+0000","last_clean":"2021-01-06T22:03:39.778763+0000","last_became_active":"2021-01-06T22:03:39.777158+0000","last_became_peered":"2021-01-06T22:03:39.777158+0000","last_unstale":"2021-01-06T22:03:39.778763+0000","last_undegraded":"2021-01-06T22:03:39.778763+0000","last_fullsized":"2021-01-06T22:03:39.778763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747656+0000","last_change":"2021-01-06T22:03:36.208974+0000","last_active":"2021-01-06T22:03:39.747656+0000","last_peered":"2021-01-06T22:03:39.747656+0000","last_clean":"2021-01-06T22:03:39.747656+0000","last_became_active":"2021-01-06T22:03:36.208576+0000","last_became_peered":"2021-01-06T22:03:36.208576+0000","last_unstale":"2021-01-06T22:03:39.747656+0000","last_undegraded":"2021-01-06T22:03:39.747656+0000","last_fullsized":"2021-01-06T22:03:39.747656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"23'13","reported_seq":"23","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.816002+0000","last_change":"2021-01-06T22:03:39.375127+0000","last_active":"2021-01-06T22:03:43.816002+0000","last_peered":"2021-01-06T22:03:43.816002+0000","last_clean":"2021-01-06T22:03:43.816002+0000","last_became_active":"2021-01-06T22:03:39.374751+0000","last_became_peered":"2021-01-06T22:03:39.374751+0000","last_unstale":"2021-01-06T22:03:43.816002+0000","last_undegraded":"2021-01-06T22:03:43.816002+0000","last_fullsized":"2021-01-06T22:03:43.816002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2088,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:40.232168+0000","last_change":"2021-01-06T22:03:40.230887+0000","last_active":"2021-01-06T22:03:40.232168+0000","last_peered":"2021-01-06T22:03:40.232168+0000","last_clean":"2021-01-06T22:03:40.232168+0000","last_became_active":"2021-01-06T22:03:40.230100+0000","last_became_peered":"2021-01-06T22:03:40.230100+0000","last_unstale":"2021-01-06T22:03:40.232168+0000","last_undegraded":"2021-01-06T22:03:40.232168+0000","last_fullsized":"2021-01-06T22:03:40.232168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"23'17","reported_seq":"27","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.818982+0000","last_change":"2021-01-06T22:03:39.377489+0000","last_active":"2021-01-06T22:03:43.818982+0000","last_peered":"2021-01-06T22:03:43.818982+0000","last_clean":"2021-01-06T22:03:43.818982+0000","last_became_active":"2021-01-06T22:03:39.377092+0000","last_became_peered":"2021-01-06T22:03:39.377092+0000","last_unstale":"2021-01-06T22:03:43.818982+0000","last_undegraded":"2021-01-06T22:03:43.818982+0000","last_fullsized":"2021-01-06T22:03:43.818982+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":665,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.759458+0000","last_change":"2021-01-06T22:03:36.213113+0000","last_active":"2021-01-06T22:03:39.759458+0000","last_peered":"2021-01-06T22:03:39.759458+0000","last_clean":"2021-01-06T22:03:39.759458+0000","last_became_active":"2021-01-06T22:03:36.212790+0000","last_became_peered":"2021-01-06T22:03:36.212790+0000","last_unstale":"2021-01-06T22:03:39.759458+0000","last_undegraded":"2021-01-06T22:03:39.759458+0000","last_fullsized":"2021-01-06T22:03:39.759458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.764030+0000","last_change":"2021-01-06T22:03:39.759582+0000","last_active":"2021-01-06T22:03:39.764030+0000","last_peered":"2021-01-06T22:03:39.764030+0000","last_clean":"2021-01-06T22:03:39.764030+0000","last_became_active":"2021-01-06T22:03:39.759212+0000","last_became_peered":"2021-01-06T22:03:39.759212+0000","last_unstale":"2021-01-06T22:03:39.764030+0000","last_undegraded":"2021-01-06T22:03:39.764030+0000","last_fullsized":"2021-01-06T22:03:39.764030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747367+0000","last_change":"2021-01-06T22:03:36.207481+0000","last_active":"2021-01-06T22:03:39.747367+0000","last_peered":"2021-01-06T22:03:39.747367+0000","last_clean":"2021-01-06T22:03:39.747367+0000","last_became_active":"2021-01-06T22:03:36.206991+0000","last_became_peered":"2021-01-06T22:03:36.206991+0000","last_unstale":"2021-01-06T22:03:39.747367+0000","last_undegraded":"2021-01-06T22:03:39.747367+0000","last_fullsized":"2021-01-06T22:03:39.747367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'9","reported_seq":"19","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.813735+0000","last_change":"2021-01-06T22:03:39.370350+0000","last_active":"2021-01-06T22:03:43.813735+0000","last_peered":"2021-01-06T22:03:43.813735+0000","last_clean":"2021-01-06T22:03:43.813735+0000","last_became_active":"2021-01-06T22:03:39.369929+0000","last_became_peered":"2021-01-06T22:03:39.369929+0000","last_unstale":"2021-01-06T22:03:43.813735+0000","last_undegraded":"2021-01-06T22:03:43.813735+0000","last_fullsized":"2021-01-06T22:03:43.813735+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":4962,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":9924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5112,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366728,"statfs":{"total":96636764160,"available":96631529472,"internally_reserved":0,"allocated":245760,"data_stored":91578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3460000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6829999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.532}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1190000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2189999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5224,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366616,"statfs":{"total":96636764160,"available":96631414784,"internally_reserved":0,"allocated":294912,"data_stored":95674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9570000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2440000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4990000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3069999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8260000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8690000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9239999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5128,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366712,"statfs":{"total":96636764160,"available":96631513088,"internally_reserved":0,"allocated":262144,"data_stored":93666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.758}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4380000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6139999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8520000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6219999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7839999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8690000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5116,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366724,"statfs":{"total":96636764160,"available":96631525376,"internally_reserved":0,"allocated":249856,"data_stored":91530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.613}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.056}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.74}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.452}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.548}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6110000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7869999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":237568,"data_stored":91466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9770000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.772}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3370000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.633}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8450000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9089999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.956}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5148,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366692,"statfs":{"total":96636764160,"available":96631492608,"internally_reserved":0,"allocated":282624,"data_stored":93563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.762}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7030000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4129999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.972}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8450000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0710000000000002}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5104,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366736,"statfs":{"total":96636764160,"available":96631537664,"internally_reserved":0,"allocated":237568,"data_stored":91466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.036}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.262}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1240000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3330000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3740000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.879}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1819999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5116,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366724,"statfs":{"total":96636764160,"available":96631525376,"internally_reserved":0,"allocated":249856,"data_stored":92131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.419}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9530000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8660000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4140000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9590000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.556}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5209999999999999}]}]}]}} 2021-01-06T22:03:53.417 INFO:tasks.ceph.ceph_manager.ceph:clean! 2021-01-06T22:03:53.417 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2021-01-06T22:03:53.418 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2021-01-06T22:03:53.418 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2021-01-06T22:03:53.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0759e700 1 Processor -- start 2021-01-06T22:03:53.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0759e700 1 -- start start 2021-01-06T22:03:53.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0759e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fed080602b0 0x7fed080c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0759e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fed08149150 0x7fed0814bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0759e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed0814f0a0 con 0x7fed08148d70 2021-01-06T22:03:53.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0759e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed081495a0 con 0x7fed080602b0 2021-01-06T22:03:53.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0759e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed0814c3c0 con 0x7fed08149150 2021-01-06T22:03:53.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0659c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fed08148d70 0x7fed0814fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47638/0 (socket says 172.21.15.203:47638) 2021-01-06T22:03:53.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0659c700 1 -- 172.21.15.203:0/2920604002 learned_addr learned my addr 172.21.15.203:0/2920604002 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:53.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed06d9d700 1 --2- 172.21.15.203:0/2920604002 >> v2:172.21.15.203:3300/0 conn(0x7fed08149150 0x7fed0814bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed05d9b700 1 --2- 172.21.15.203:0/2920604002 >> v2:172.21.15.91:3300/0 conn(0x7fed080602b0 0x7fed080c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0559a700 1 -- 172.21.15.203:0/2920604002 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913876490 0 0) 0x7fed0814f0a0 con 0x7fed08148d70 2021-01-06T22:03:53.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0559a700 1 -- 172.21.15.203:0/2920604002 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecec003230 con 0x7fed08148d70 2021-01-06T22:03:53.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0559a700 1 -- 172.21.15.203:0/2920604002 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3670864704 0 0) 0x7fecec003230 con 0x7fed08148d70 2021-01-06T22:03:53.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0559a700 1 -- 172.21.15.203:0/2920604002 >> v2:172.21.15.203:3300/0 conn(0x7fed08149150 msgr2=0x7fed0814bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:53.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0559a700 1 --2- 172.21.15.203:0/2920604002 >> v2:172.21.15.203:3300/0 conn(0x7fed08149150 0x7fed0814bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0559a700 1 -- 172.21.15.203:0/2920604002 >> v2:172.21.15.91:3300/0 conn(0x7fed080602b0 msgr2=0x7fed080c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:53.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0559a700 1 --2- 172.21.15.203:0/2920604002 >> v2:172.21.15.91:3300/0 conn(0x7fed080602b0 0x7fed080c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0559a700 1 -- 172.21.15.203:0/2920604002 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0814c660 con 0x7fed08148d70 2021-01-06T22:03:53.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0559a700 1 -- 172.21.15.203:0/2920604002 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7fecf00014e0 con 0x7fed08148d70 2021-01-06T22:03:53.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0559a700 1 -- 172.21.15.203:0/2920604002 >> v1:172.21.15.203:6789/0 conn(0x7fed08148d70 legacy=0x7fed0814fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0559a700 1 --2- 172.21.15.203:0/2920604002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecec0037f0 0x7fecec003c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0559a700 1 --2- 172.21.15.203:0/2920604002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecec0044d0 0x7fecec004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0559a700 1 --2- 172.21.15.203:0/2920604002 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fecec004e40 0x7fecec00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0559a700 1 -- 172.21.15.203:0/2920604002 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed081495a0 con 0x7fecec0044d0 2021-01-06T22:03:53.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0559a700 1 -- 172.21.15.203:0/2920604002 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0814c3c0 con 0x7fecec0037f0 2021-01-06T22:03:53.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0559a700 1 -- 172.21.15.203:0/2920604002 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec005370 con 0x7fecec004e40 2021-01-06T22:03:53.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0659c700 1 --2- 172.21.15.203:0/2920604002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecec0037f0 0x7fecec003c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed05d9b700 1 --2- 172.21.15.203:0/2920604002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecec0044d0 0x7fecec004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0659c700 1 -- 172.21.15.203:0/2920604002 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fecec004e40 msgr2=0x7fecec00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:53.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0659c700 1 --2- 172.21.15.203:0/2920604002 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fecec004e40 0x7fecec00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:03:53.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0659c700 1 -- 172.21.15.203:0/2920604002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecec0044d0 msgr2=0x7fecec004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:53.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0659c700 1 --2- 172.21.15.203:0/2920604002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecec0044d0 0x7fecec004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0659c700 1 -- 172.21.15.203:0/2920604002 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec003230 con 0x7fecec0037f0 2021-01-06T22:03:53.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed05d9b700 1 --2- 172.21.15.203:0/2920604002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecec0044d0 0x7fecec004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:03:53.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed06d9d700 1 -- 172.21.15.203:0/2920604002 reap_dead start 2021-01-06T22:03:53.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.539+0000 7fed0659c700 1 --2- 172.21.15.203:0/2920604002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecec0037f0 0x7fecec003c10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fecf000c240 tx=0x7fed0814c7f0).ready entity=mon.1 client_cookie=e0bacc0e0c6ddeb3 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:53.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.543+0000 7fed0559a700 1 -- 172.21.15.203:0/2920604002 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf00076e0 con 0x7fecec0037f0 2021-01-06T22:03:53.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.543+0000 7fed0559a700 1 -- 172.21.15.203:0/2920604002 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf00070d0 con 0x7fecec0037f0 2021-01-06T22:03:53.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.543+0000 7fed0559a700 1 -- 172.21.15.203:0/2920604002 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0007490 con 0x7fecec0037f0 2021-01-06T22:03:53.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.543+0000 7fed0759e700 1 -- 172.21.15.203:0/2920604002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecec0037f0 msgr2=0x7fecec003c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.543+0000 7fed0759e700 1 --2- 172.21.15.203:0/2920604002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecec0037f0 0x7fecec003c10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fecf000c240 tx=0x7fed0814c7f0).stop 2021-01-06T22:03:53.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.543+0000 7fed0759e700 1 -- 172.21.15.203:0/2920604002 shutdown_connections 2021-01-06T22:03:53.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.543+0000 7fed0759e700 1 --2- 172.21.15.203:0/2920604002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecec0037f0 0x7fecec003c10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.543+0000 7fed0759e700 1 -- 172.21.15.203:0/2920604002 >> 172.21.15.203:0/2920604002 conn(0x7fed080667b0 msgr2=0x7fed080c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:53.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.543+0000 7fed0759e700 1 -- 172.21.15.203:0/2920604002 shutdown_connections 2021-01-06T22:03:53.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.543+0000 7fed0759e700 1 -- 172.21.15.203:0/2920604002 wait complete. 2021-01-06T22:03:53.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.543+0000 7fed0759e700 1 Processor -- start 2021-01-06T22:03:53.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.543+0000 7fed0759e700 1 -- start start 2021-01-06T22:03:53.555 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.543+0000 7fed0759e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecec0037f0 0x7fed0815e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.555 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.543+0000 7fed0759e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed08149cf0 0x7fed0814a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.555 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed0759e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed08144180 0x7fed081445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.555 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed0759e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed080b1ca0 con 0x7fed08144180 2021-01-06T22:03:53.556 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed0759e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0814a660 con 0x7fecec0037f0 2021-01-06T22:03:53.556 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed0759e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed080b80b0 con 0x7fed08149cf0 2021-01-06T22:03:53.556 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed0659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecec0037f0 0x7fed0815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.556 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed0659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecec0037f0 0x7fed0815e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49056/0 (socket says 172.21.15.203:49056) 2021-01-06T22:03:53.556 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed0659c700 1 -- 172.21.15.203:0/814455190 learned_addr learned my addr 172.21.15.203:0/814455190 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:53.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed06d9d700 1 --2- 172.21.15.203:0/814455190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed08144180 0x7fed081445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed05d9b700 1 --2- 172.21.15.203:0/814455190 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed08149cf0 0x7fed0814a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed0659c700 1 -- 172.21.15.203:0/814455190 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed08149cf0 msgr2=0x7fed0814a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed0659c700 1 --2- 172.21.15.203:0/814455190 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed08149cf0 0x7fed0814a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed0659c700 1 -- 172.21.15.203:0/814455190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed08144180 msgr2=0x7fed081445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed0659c700 1 --2- 172.21.15.203:0/814455190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed08144180 0x7fed081445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed0659c700 1 -- 172.21.15.203:0/814455190 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed080b14c0 con 0x7fecec0037f0 2021-01-06T22:03:53.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed0659c700 1 --2- 172.21.15.203:0/814455190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecec0037f0 0x7fed0815e170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fecf0007570 tx=0x7fecf0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:53.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fecfe7fc700 1 -- 172.21.15.203:0/814455190 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0000cd0 con 0x7fecec0037f0 2021-01-06T22:03:53.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fecfe7fc700 1 -- 172.21.15.203:0/814455190 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf0006e70 con 0x7fecec0037f0 2021-01-06T22:03:53.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fecfe7fc700 1 -- 172.21.15.203:0/814455190 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0013660 con 0x7fecec0037f0 2021-01-06T22:03:53.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed0759e700 1 -- 172.21.15.203:0/814455190 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed080b1760 con 0x7fecec0037f0 2021-01-06T22:03:53.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed0759e700 1 -- 172.21.15.203:0/814455190 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed08149850 con 0x7fecec0037f0 2021-01-06T22:03:53.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed0759e700 1 -- 172.21.15.203:0/814455190 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed0815e660 con 0x7fecec0037f0 2021-01-06T22:03:53.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fecfe7fc700 1 -- 172.21.15.203:0/814455190 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fecf0026020 con 0x7fecec0037f0 2021-01-06T22:03:53.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fecfe7fc700 1 --2- 172.21.15.203:0/814455190 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fece0056400 0x7fece00588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fecfe7fc700 1 -- 172.21.15.203:0/814455190 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fecf0070ba0 con 0x7fecec0037f0 2021-01-06T22:03:53.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.547+0000 7fed05d9b700 1 --2- 172.21.15.203:0/814455190 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fece0056400 0x7fece00588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.551+0000 7fed05d9b700 1 --2- 172.21.15.203:0/814455190 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fece0056400 0x7fece00588c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fecf8006cf0 tx=0x7fecf8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:53.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.555+0000 7fecfe7fc700 1 -- 172.21.15.203:0/814455190 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fecf0046420 con 0x7fecec0037f0 2021-01-06T22:03:53.765 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.763+0000 7fed0759e700 1 -- 172.21.15.203:0/814455190 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fed080450a0 con 0x7fecec0037f0 2021-01-06T22:03:53.766 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.763+0000 7fecfe7fc700 1 -- 172.21.15.203:0/814455190 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7fecf0049a40 con 0x7fecec0037f0 2021-01-06T22:03:53.766 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:53.769 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.767+0000 7fed0759e700 1 -- 172.21.15.203:0/814455190 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fece0056400 msgr2=0x7fece00588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.769 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.767+0000 7fed0759e700 1 --2- 172.21.15.203:0/814455190 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fece0056400 0x7fece00588c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fecf8006cf0 tx=0x7fecf8008040).stop 2021-01-06T22:03:53.769 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.767+0000 7fed0759e700 1 -- 172.21.15.203:0/814455190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecec0037f0 msgr2=0x7fed0815e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.767+0000 7fed0759e700 1 --2- 172.21.15.203:0/814455190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecec0037f0 0x7fed0815e170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fecf0007570 tx=0x7fecf0015000).stop 2021-01-06T22:03:53.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.767+0000 7fed0759e700 1 -- 172.21.15.203:0/814455190 shutdown_connections 2021-01-06T22:03:53.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.767+0000 7fed0759e700 1 --2- 172.21.15.203:0/814455190 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fece0056400 0x7fece00588c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.767+0000 7fed0759e700 1 --2- 172.21.15.203:0/814455190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed08144180 0x7fed081445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.767+0000 7fed0759e700 1 --2- 172.21.15.203:0/814455190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecec0037f0 0x7fed0815e170 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.767+0000 7fed0759e700 1 --2- 172.21.15.203:0/814455190 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed08149cf0 0x7fed0814a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.767+0000 7fed0759e700 1 -- 172.21.15.203:0/814455190 >> 172.21.15.203:0/814455190 conn(0x7fed080667b0 msgr2=0x7fed08067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:53.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.767+0000 7fed0759e700 1 -- 172.21.15.203:0/814455190 shutdown_connections 2021-01-06T22:03:53.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.767+0000 7fed0759e700 1 -- 172.21.15.203:0/814455190 wait complete. 2021-01-06T22:03:53.786 INFO:teuthology.orchestra.run.smithi203.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-01-06T22:03:53.787 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2021-01-06T22:03:53.788 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-06T22:03:53.909 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.907+0000 7fd99d4aa700 1 Processor -- start 2021-01-06T22:03:53.910 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.907+0000 7fd99d4aa700 1 -- start start 2021-01-06T22:03:53.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd99d4aa700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd998067720 0x7fd998068110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd99d4aa700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd998148080 0x7fd9980602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd99d4aa700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9980bba10 con 0x7fd998067340 2021-01-06T22:03:53.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd99d4aa700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9980bb230 con 0x7fd998148080 2021-01-06T22:03:53.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd99d4aa700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9980bb4d0 con 0x7fd998067720 2021-01-06T22:03:53.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd997fff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fd998067340 0x7fd998147970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47658/0 (socket says 172.21.15.203:47658) 2021-01-06T22:03:53.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd997fff700 1 -- 172.21.15.203:0/3350467141 learned_addr learned my addr 172.21.15.203:0/3350467141 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:53.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd9977fe700 1 --2- 172.21.15.203:0/3350467141 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd998067720 0x7fd998068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd99cca9700 1 --2- 172.21.15.203:0/3350467141 >> v2:172.21.15.91:3300/0 conn(0x7fd998148080 0x7fd9980602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd996ffd700 1 -- 172.21.15.203:0/3350467141 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281352134 0 0) 0x7fd9980bba10 con 0x7fd998067340 2021-01-06T22:03:53.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd996ffd700 1 -- 172.21.15.203:0/3350467141 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd984003230 con 0x7fd998067340 2021-01-06T22:03:53.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd9977fe700 1 -- 172.21.15.203:0/3350467141 >> v2:172.21.15.91:3300/0 conn(0x7fd998148080 msgr2=0x7fd9980602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:53.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd9977fe700 1 --2- 172.21.15.203:0/3350467141 >> v2:172.21.15.91:3300/0 conn(0x7fd998148080 0x7fd9980602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd9977fe700 1 -- 172.21.15.203:0/3350467141 >> v1:172.21.15.203:6789/0 conn(0x7fd998067340 legacy=0x7fd998147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd9977fe700 1 -- 172.21.15.203:0/3350467141 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9980bb770 con 0x7fd998067720 2021-01-06T22:03:53.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd9977fe700 1 --2- 172.21.15.203:0/3350467141 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd998067720 0x7fd998068110 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fd98c00b980 tx=0x7fd98c006de0).ready entity=mon.2 client_cookie=cda2b03c313908c3 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:53.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd996ffd700 1 -- 172.21.15.203:0/3350467141 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0096d0 con 0x7fd998067720 2021-01-06T22:03:53.918 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd996ffd700 1 -- 172.21.15.203:0/3350467141 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c009830 con 0x7fd998067720 2021-01-06T22:03:53.918 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd996ffd700 1 -- 172.21.15.203:0/3350467141 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c009bb0 con 0x7fd998067720 2021-01-06T22:03:53.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd99d4aa700 1 -- 172.21.15.203:0/3350467141 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd998067720 msgr2=0x7fd998068110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd99d4aa700 1 --2- 172.21.15.203:0/3350467141 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd998067720 0x7fd998068110 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fd98c00b980 tx=0x7fd98c006de0).stop 2021-01-06T22:03:53.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd99d4aa700 1 -- 172.21.15.203:0/3350467141 shutdown_connections 2021-01-06T22:03:53.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd99d4aa700 1 --2- 172.21.15.203:0/3350467141 >> v2:172.21.15.91:3300/0 conn(0x7fd998148080 0x7fd9980602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd99d4aa700 1 --2- 172.21.15.203:0/3350467141 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd998067720 0x7fd998068110 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd99d4aa700 1 -- 172.21.15.203:0/3350467141 >> 172.21.15.203:0/3350467141 conn(0x7fd9980667b0 msgr2=0x7fd9980c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:53.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd99d4aa700 1 -- 172.21.15.203:0/3350467141 shutdown_connections 2021-01-06T22:03:53.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.911+0000 7fd99d4aa700 1 -- 172.21.15.203:0/3350467141 wait complete. 2021-01-06T22:03:53.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd99d4aa700 1 Processor -- start 2021-01-06T22:03:53.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd99d4aa700 1 -- start start 2021-01-06T22:03:53.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd99d4aa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd998067340 0x7fd9980bf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.928 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd99d4aa700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd998067720 0x7fd9981400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.928 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd99d4aa700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd998148080 0x7fd99815b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd99d4aa700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9980be950 con 0x7fd998148080 2021-01-06T22:03:53.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd99d4aa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99815bf40 con 0x7fd998067340 2021-01-06T22:03:53.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd99d4aa700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99815c1e0 con 0x7fd998067720 2021-01-06T22:03:53.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd997fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd998067340 0x7fd9980bf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd997fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd998067340 0x7fd9980bf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49068/0 (socket says 172.21.15.203:49068) 2021-01-06T22:03:53.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd997fff700 1 -- 172.21.15.203:0/3294957704 learned_addr learned my addr 172.21.15.203:0/3294957704 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:53.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd9977fe700 1 --2- 172.21.15.203:0/3294957704 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd998067720 0x7fd9981400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd99cca9700 1 --2- 172.21.15.203:0/3294957704 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd998148080 0x7fd99815b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd997fff700 1 -- 172.21.15.203:0/3294957704 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd998067720 msgr2=0x7fd9981400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd997fff700 1 --2- 172.21.15.203:0/3294957704 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd998067720 0x7fd9981400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd997fff700 1 -- 172.21.15.203:0/3294957704 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd998148080 msgr2=0x7fd99815b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:53.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd997fff700 1 --2- 172.21.15.203:0/3294957704 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd998148080 0x7fd99815b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:53.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd997fff700 1 -- 172.21.15.203:0/3294957704 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9980beb40 con 0x7fd998067340 2021-01-06T22:03:53.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd99cca9700 1 --2- 172.21.15.203:0/3294957704 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd998148080 0x7fd99815b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:03:53.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd997fff700 1 --2- 172.21.15.203:0/3294957704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd998067340 0x7fd9980bf520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fd98800c6e0 tx=0x7fd988007ab0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:53.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd97ffff700 1 -- 172.21.15.203:0/3294957704 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9880072e0 con 0x7fd998067340 2021-01-06T22:03:53.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd97ffff700 1 -- 172.21.15.203:0/3294957704 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd988007440 con 0x7fd998067340 2021-01-06T22:03:53.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd99d4aa700 1 -- 172.21.15.203:0/3294957704 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9980bede0 con 0x7fd998067340 2021-01-06T22:03:53.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd97ffff700 1 -- 172.21.15.203:0/3294957704 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988005810 con 0x7fd998067340 2021-01-06T22:03:53.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.915+0000 7fd99d4aa700 1 -- 172.21.15.203:0/3294957704 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9980bfa10 con 0x7fd998067340 2021-01-06T22:03:53.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.919+0000 7fd99d4aa700 1 -- 172.21.15.203:0/3294957704 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9980450a0 con 0x7fd998067340 2021-01-06T22:03:53.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.919+0000 7fd97ffff700 1 -- 172.21.15.203:0/3294957704 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fd988010070 con 0x7fd998067340 2021-01-06T22:03:53.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.919+0000 7fd97ffff700 1 --2- 172.21.15.203:0/3294957704 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9780563d0 0x7fd978058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:53.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.919+0000 7fd97ffff700 1 -- 172.21.15.203:0/3294957704 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd98806eb90 con 0x7fd998067340 2021-01-06T22:03:53.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.919+0000 7fd9977fe700 1 --2- 172.21.15.203:0/3294957704 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9780563d0 0x7fd978058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:53.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.919+0000 7fd9977fe700 1 --2- 172.21.15.203:0/3294957704 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9780563d0 0x7fd978058890 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fd98c00b130 tx=0x7fd98c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:53.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:53.927+0000 7fd97ffff700 1 -- 172.21.15.203:0/3294957704 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fd988044410 con 0x7fd998067340 2021-01-06T22:03:54.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.083+0000 7fd99d4aa700 1 -- 172.21.15.203:0/3294957704 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd99805ca60 con 0x7fd998067340 2021-01-06T22:03:54.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.087+0000 7fd97ffff700 1 -- 172.21.15.203:0/3294957704 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16826 (secure 0 0 0) 0x7fd988047a30 con 0x7fd998067340 2021-01-06T22:03:54.088 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:54.088 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":23,"fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","created":"2021-01-06T22:03:21.322671+0000","modified":"2021-01-06T22:03:40.744826+0000","last_up_change":"2021-01-06T22:03:31.115611+0000","last_in_change":"2021-01-06T22:03:31.115611+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-06T22:03:25.161465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-06T22:03:35.070364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-01-06T22:03:37.862354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-01-06T22:03:38.738416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e0b35c09-147c-4ccc-b566-c48b81e25412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":18347},{"type":"v1","addr":"172.21.15.203:6815","nonce":18347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":18347},{"type":"v1","addr":"172.21.15.203:6818","nonce":18347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":18347},{"type":"v1","addr":"172.21.15.203:6831","nonce":18347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6821","nonce":18347},{"type":"v1","addr":"172.21.15.203:6827","nonce":18347}]},"public_addr":"172.21.15.203:6815/18347","cluster_addr":"172.21.15.203:6818/18347","heartbeat_back_addr":"172.21.15.203:6831/18347","heartbeat_front_addr":"172.21.15.203:6827/18347","state":["exists","up"]},{"osd":1,"uuid":"8add59a0-2d58-4c1b-a8d1-62049ee9d85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":18337},{"type":"v1","addr":"172.21.15.203:6803","nonce":18337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":18337},{"type":"v1","addr":"172.21.15.203:6805","nonce":18337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":18337},{"type":"v1","addr":"172.21.15.203:6809","nonce":18337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":18337},{"type":"v1","addr":"172.21.15.203:6807","nonce":18337}]},"public_addr":"172.21.15.203:6803/18337","cluster_addr":"172.21.15.203:6805/18337","heartbeat_back_addr":"172.21.15.203:6809/18337","heartbeat_front_addr":"172.21.15.203:6807/18337","state":["exists","up"]},{"osd":2,"uuid":"b294a411-c7c9-477e-8eae-3b7bd0563ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":18354},{"type":"v1","addr":"172.21.15.203:6811","nonce":18354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":18354},{"type":"v1","addr":"172.21.15.203:6819","nonce":18354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6825","nonce":18354},{"type":"v1","addr":"172.21.15.203:6828","nonce":18354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6823","nonce":18354},{"type":"v1","addr":"172.21.15.203:6824","nonce":18354}]},"public_addr":"172.21.15.203:6811/18354","cluster_addr":"172.21.15.203:6819/18354","heartbeat_back_addr":"172.21.15.203:6828/18354","heartbeat_front_addr":"172.21.15.203:6824/18354","state":["exists","up"]},{"osd":3,"uuid":"5878a8f7-1a42-41c3-9810-0ac41a54dd6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":18346},{"type":"v1","addr":"172.21.15.203:6817","nonce":18346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":18346},{"type":"v1","addr":"172.21.15.203:6822","nonce":18346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":18346},{"type":"v1","addr":"172.21.15.203:6833","nonce":18346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":18346},{"type":"v1","addr":"172.21.15.203:6829","nonce":18346}]},"public_addr":"172.21.15.203:6817/18346","cluster_addr":"172.21.15.203:6822/18346","heartbeat_back_addr":"172.21.15.203:6833/18346","heartbeat_front_addr":"172.21.15.203:6829/18346","state":["exists","up"]},{"osd":4,"uuid":"f641a345-81e4-4400-a44c-d2f82fef7f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":18370},{"type":"v1","addr":"172.21.15.91:6809","nonce":18370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":18370},{"type":"v1","addr":"172.21.15.91:6811","nonce":18370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":18370},{"type":"v1","addr":"172.21.15.91:6816","nonce":18370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":18370},{"type":"v1","addr":"172.21.15.91:6813","nonce":18370}]},"public_addr":"172.21.15.91:6809/18370","cluster_addr":"172.21.15.91:6811/18370","heartbeat_back_addr":"172.21.15.91:6816/18370","heartbeat_front_addr":"172.21.15.91:6813/18370","state":["exists","up"]},{"osd":5,"uuid":"c1ff150d-68c6-4e6b-abd4-76af27bb2103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":18372},{"type":"v1","addr":"172.21.15.91:6801","nonce":18372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":18372},{"type":"v1","addr":"172.21.15.91:6803","nonce":18372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":18372},{"type":"v1","addr":"172.21.15.91:6807","nonce":18372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":18372},{"type":"v1","addr":"172.21.15.91:6805","nonce":18372}]},"public_addr":"172.21.15.91:6801/18372","cluster_addr":"172.21.15.91:6803/18372","heartbeat_back_addr":"172.21.15.91:6807/18372","heartbeat_front_addr":"172.21.15.91:6805/18372","state":["exists","up"]},{"osd":6,"uuid":"35a58e1a-0a0a-4f02-a477-adc029038d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6815","nonce":18371},{"type":"v1","addr":"172.21.15.91:6817","nonce":18371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":18371},{"type":"v1","addr":"172.21.15.91:6819","nonce":18371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":18371},{"type":"v1","addr":"172.21.15.91:6823","nonce":18371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":18371},{"type":"v1","addr":"172.21.15.91:6821","nonce":18371}]},"public_addr":"172.21.15.91:6817/18371","cluster_addr":"172.21.15.91:6819/18371","heartbeat_back_addr":"172.21.15.91:6823/18371","heartbeat_front_addr":"172.21.15.91:6821/18371","state":["exists","up"]},{"osd":7,"uuid":"809a2d89-4303-482a-8558-aef907814234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":18373},{"type":"v1","addr":"172.21.15.91:6825","nonce":18373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":18373},{"type":"v1","addr":"172.21.15.91:6827","nonce":18373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":18373},{"type":"v1","addr":"172.21.15.91:6831","nonce":18373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":18373},{"type":"v1","addr":"172.21.15.91:6829","nonce":18373}]},"public_addr":"172.21.15.91:6825/18373","cluster_addr":"172.21.15.91:6827/18373","heartbeat_back_addr":"172.21.15.91:6831/18373","heartbeat_front_addr":"172.21.15.91:6829/18373","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.875052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.103604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.179225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.158512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.206073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.894840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.229708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.248192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-06T22:03:54.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.087+0000 7fd99d4aa700 1 -- 172.21.15.203:0/3294957704 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9780563d0 msgr2=0x7fd978058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.087+0000 7fd99d4aa700 1 --2- 172.21.15.203:0/3294957704 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9780563d0 0x7fd978058890 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fd98c00b130 tx=0x7fd98c016000).stop 2021-01-06T22:03:54.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.087+0000 7fd99d4aa700 1 -- 172.21.15.203:0/3294957704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd998067340 msgr2=0x7fd9980bf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.087+0000 7fd99d4aa700 1 --2- 172.21.15.203:0/3294957704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd998067340 0x7fd9980bf520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fd98800c6e0 tx=0x7fd988007ab0).stop 2021-01-06T22:03:54.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.087+0000 7fd99d4aa700 1 -- 172.21.15.203:0/3294957704 shutdown_connections 2021-01-06T22:03:54.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.087+0000 7fd99d4aa700 1 --2- 172.21.15.203:0/3294957704 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9780563d0 0x7fd978058890 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.087+0000 7fd99d4aa700 1 --2- 172.21.15.203:0/3294957704 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd998148080 0x7fd99815b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.087+0000 7fd99d4aa700 1 --2- 172.21.15.203:0/3294957704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd998067340 0x7fd9980bf520 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.087+0000 7fd99d4aa700 1 --2- 172.21.15.203:0/3294957704 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd998067720 0x7fd9981400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.087+0000 7fd99d4aa700 1 -- 172.21.15.203:0/3294957704 >> 172.21.15.203:0/3294957704 conn(0x7fd9980667b0 msgr2=0x7fd9980bb970 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:54.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.087+0000 7fd99d4aa700 1 -- 172.21.15.203:0/3294957704 shutdown_connections 2021-01-06T22:03:54.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.087+0000 7fd99d4aa700 1 -- 172.21.15.203:0/3294957704 wait complete. 2021-01-06T22:03:54.103 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-01-06T22:03:25.161465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-01-06T22:03:35.070364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-01-06T22:03:37.862354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-01-06T22:03:38.738416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-01-06T22:03:54.104 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-01-06T22:03:54.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f818120f700 1 Processor -- start 2021-01-06T22:03:54.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f818120f700 1 -- start start 2021-01-06T22:03:54.236 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f818120f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f817c063010 0x7f817c14e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.236 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f818120f700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f817c0633f0 0x7f817c064b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.237 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f818120f700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817c0be990 con 0x7f817c0be1b0 2021-01-06T22:03:54.237 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f818120f700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f817c14ec60 con 0x7f817c0633f0 2021-01-06T22:03:54.238 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f818120f700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f817c0be6f0 con 0x7f817c063010 2021-01-06T22:03:54.238 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f817bfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f817c063010 0x7f817c14e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.238 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f817bfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f817c063010 0x7f817c14e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49076/0 (socket says 172.21.15.203:49076) 2021-01-06T22:03:54.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f817bfff700 1 -- 172.21.15.203:0/2879427081 learned_addr learned my addr 172.21.15.203:0/2879427081 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:54.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f817b7fe700 1 --2- 172.21.15.203:0/2879427081 >> v2:172.21.15.91:3300/0 conn(0x7f817c0633f0 0x7f817c064b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.240 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f817bfff700 1 -- 172.21.15.203:0/2879427081 >> v2:172.21.15.91:3300/0 conn(0x7f817c0633f0 msgr2=0x7f817c064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:54.240 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f817bfff700 1 --2- 172.21.15.203:0/2879427081 >> v2:172.21.15.91:3300/0 conn(0x7f817c0633f0 0x7f817c064b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.240 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f817bfff700 1 -- 172.21.15.203:0/2879427081 >> v1:172.21.15.91:6789/0 conn(0x7f817c0be1b0 legacy=0x7f817c147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:54.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f817bfff700 1 -- 172.21.15.203:0/2879427081 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c0c1050 con 0x7f817c063010 2021-01-06T22:03:54.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f817bfff700 1 --2- 172.21.15.203:0/2879427081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f817c063010 0x7f817c14e770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f817c0667e0 tx=0x7f816c006ea0).ready entity=mon.1 client_cookie=8544d78a40344022 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:54.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f817affd700 1 -- 172.21.15.203:0/2879427081 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c0098d0 con 0x7f817c063010 2021-01-06T22:03:54.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f817affd700 1 -- 172.21.15.203:0/2879427081 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816c009a30 con 0x7f817c063010 2021-01-06T22:03:54.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.231+0000 7f817affd700 1 -- 172.21.15.203:0/2879427081 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c009db0 con 0x7f817c063010 2021-01-06T22:03:54.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f818120f700 1 -- 172.21.15.203:0/2879427081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f817c063010 msgr2=0x7f817c14e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f818120f700 1 --2- 172.21.15.203:0/2879427081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f817c063010 0x7f817c14e770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f817c0667e0 tx=0x7f816c006ea0).stop 2021-01-06T22:03:54.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f818120f700 1 -- 172.21.15.203:0/2879427081 shutdown_connections 2021-01-06T22:03:54.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f818120f700 1 --2- 172.21.15.203:0/2879427081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f817c063010 0x7f817c14e770 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f818120f700 1 --2- 172.21.15.203:0/2879427081 >> v2:172.21.15.91:3300/0 conn(0x7f817c0633f0 0x7f817c064b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f818120f700 1 -- 172.21.15.203:0/2879427081 >> 172.21.15.203:0/2879427081 conn(0x7f817c067e20 msgr2=0x7f817c068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:54.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f818120f700 1 -- 172.21.15.203:0/2879427081 shutdown_connections 2021-01-06T22:03:54.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f818120f700 1 -- 172.21.15.203:0/2879427081 wait complete. 2021-01-06T22:03:54.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f818120f700 1 Processor -- start 2021-01-06T22:03:54.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f818120f700 1 -- start start 2021-01-06T22:03:54.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f818120f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f817c063010 0x7f817c146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f818120f700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f817c0633f0 0x7f817c14a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f818120f700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f817c0be1b0 0x7f817c145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f818120f700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c148280 con 0x7f817c0be1b0 2021-01-06T22:03:54.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f818120f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c14b1e0 con 0x7f817c063010 2021-01-06T22:03:54.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f818120f700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817c147d40 con 0x7f817c0633f0 2021-01-06T22:03:54.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f817bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f817c063010 0x7f817c146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f817bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f817c063010 0x7f817c146e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49082/0 (socket says 172.21.15.203:49082) 2021-01-06T22:03:54.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f817bfff700 1 -- 172.21.15.203:0/2595975252 learned_addr learned my addr 172.21.15.203:0/2595975252 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:54.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f817b7fe700 1 --2- 172.21.15.203:0/2595975252 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f817c0633f0 0x7f817c14a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f8180a0e700 1 --2- 172.21.15.203:0/2595975252 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f817c0be1b0 0x7f817c145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f817bfff700 1 -- 172.21.15.203:0/2595975252 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f817c0633f0 msgr2=0x7f817c14a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f817bfff700 1 --2- 172.21.15.203:0/2595975252 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f817c0633f0 0x7f817c14a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f817bfff700 1 -- 172.21.15.203:0/2595975252 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f817c0be1b0 msgr2=0x7f817c145590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f817bfff700 1 --2- 172.21.15.203:0/2595975252 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f817c0be1b0 0x7f817c145590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f817bfff700 1 -- 172.21.15.203:0/2595975252 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c147fe0 con 0x7f817c063010 2021-01-06T22:03:54.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.235+0000 7f817bfff700 1 --2- 172.21.15.203:0/2595975252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f817c063010 0x7f817c146e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f816c011940 tx=0x7f816c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:54.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.239+0000 7f8163fff700 1 -- 172.21.15.203:0/2595975252 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c011ea0 con 0x7f817c063010 2021-01-06T22:03:54.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.239+0000 7f8163fff700 1 -- 172.21.15.203:0/2595975252 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816c00be80 con 0x7f817c063010 2021-01-06T22:03:54.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.239+0000 7f818120f700 1 -- 172.21.15.203:0/2595975252 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817c14d1d0 con 0x7f817c063010 2021-01-06T22:03:54.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.239+0000 7f8163fff700 1 -- 172.21.15.203:0/2595975252 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c01d660 con 0x7f817c063010 2021-01-06T22:03:54.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.239+0000 7f818120f700 1 -- 172.21.15.203:0/2595975252 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817c14add0 con 0x7f817c063010 2021-01-06T22:03:54.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.239+0000 7f818120f700 1 -- 172.21.15.203:0/2595975252 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817c05d810 con 0x7f817c063010 2021-01-06T22:03:54.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.239+0000 7f8163fff700 1 -- 172.21.15.203:0/2595975252 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f816c00f070 con 0x7f817c063010 2021-01-06T22:03:54.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.239+0000 7f8163fff700 1 --2- 172.21.15.203:0/2595975252 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f815c0563d0 0x7f815c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.239+0000 7f817b7fe700 1 --2- 172.21.15.203:0/2595975252 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f815c0563d0 0x7f815c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.239+0000 7f8163fff700 1 -- 172.21.15.203:0/2595975252 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f816c070fa0 con 0x7f817c063010 2021-01-06T22:03:54.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.239+0000 7f817b7fe700 1 --2- 172.21.15.203:0/2595975252 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f815c0563d0 0x7f815c058890 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f8170006fd0 tx=0x7f8170008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:54.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.243+0000 7f8163fff700 1 -- 172.21.15.203:0/2595975252 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f816c046820 con 0x7f817c063010 2021-01-06T22:03:54.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.399+0000 7f818120f700 1 -- 172.21.15.203:0/2595975252 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f817c14d670 con 0x7f817c063010 2021-01-06T22:03:54.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.399+0000 7f8163fff700 1 -- 172.21.15.203:0/2595975252 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f816c049e40 con 0x7f817c063010 2021-01-06T22:03:54.403 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 1 2021-01-06T22:03:54.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.403+0000 7f818120f700 1 -- 172.21.15.203:0/2595975252 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f815c0563d0 msgr2=0x7f815c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.403+0000 7f818120f700 1 --2- 172.21.15.203:0/2595975252 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f815c0563d0 0x7f815c058890 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f8170006fd0 tx=0x7f8170008040).stop 2021-01-06T22:03:54.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.403+0000 7f818120f700 1 -- 172.21.15.203:0/2595975252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f817c063010 msgr2=0x7f817c146e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.403+0000 7f818120f700 1 --2- 172.21.15.203:0/2595975252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f817c063010 0x7f817c146e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f816c011940 tx=0x7f816c016000).stop 2021-01-06T22:03:54.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.403+0000 7f818120f700 1 -- 172.21.15.203:0/2595975252 shutdown_connections 2021-01-06T22:03:54.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.403+0000 7f818120f700 1 --2- 172.21.15.203:0/2595975252 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f815c0563d0 0x7f815c058890 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.403+0000 7f818120f700 1 --2- 172.21.15.203:0/2595975252 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f817c0be1b0 0x7f817c145590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.403+0000 7f818120f700 1 --2- 172.21.15.203:0/2595975252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f817c063010 0x7f817c146e20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.403+0000 7f818120f700 1 --2- 172.21.15.203:0/2595975252 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f817c0633f0 0x7f817c14a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.403+0000 7f818120f700 1 -- 172.21.15.203:0/2595975252 >> 172.21.15.203:0/2595975252 conn(0x7f817c067e20 msgr2=0x7f817c14e550 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:54.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.403+0000 7f818120f700 1 -- 172.21.15.203:0/2595975252 shutdown_connections 2021-01-06T22:03:54.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.403+0000 7f818120f700 1 -- 172.21.15.203:0/2595975252 wait complete. 2021-01-06T22:03:54.422 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-01-06T22:03:54.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9cad133700 1 Processor -- start 2021-01-06T22:03:54.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9cad133700 1 -- start start 2021-01-06T22:03:54.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9cad133700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9ca8151560 0x7f9ca80bf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9cad133700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca8152460 con 0x7f9ca8064bc0 2021-01-06T22:03:54.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9cad133700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca814e740 con 0x7f9ca80bf100 2021-01-06T22:03:54.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9cad133700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca814e9e0 con 0x7f9ca8151560 2021-01-06T22:03:54.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca6d9d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f9ca80bf100 0x7f9ca814df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47684/0 (socket says 172.21.15.203:47684) 2021-01-06T22:03:54.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca6d9d700 1 -- 172.21.15.203:0/242748769 learned_addr learned my addr 172.21.15.203:0/242748769 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:54.555 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca759e700 1 --2- 172.21.15.203:0/242748769 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9ca8151560 0x7f9ca80bf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca5d9b700 1 -- 172.21.15.203:0/242748769 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323293269 0 0) 0x7f9ca814e740 con 0x7f9ca80bf100 2021-01-06T22:03:54.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca5d9b700 1 -- 172.21.15.203:0/242748769 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c94003230 con 0x7f9ca80bf100 2021-01-06T22:03:54.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca5d9b700 1 -- 172.21.15.203:0/242748769 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3878512161 0 0) 0x7f9c94003230 con 0x7f9ca80bf100 2021-01-06T22:03:54.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca5d9b700 1 -- 172.21.15.203:0/242748769 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9ca8151560 msgr2=0x7f9ca80bf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:54.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca5d9b700 1 --2- 172.21.15.203:0/242748769 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9ca8151560 0x7f9ca80bf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca5d9b700 1 -- 172.21.15.203:0/242748769 >> v1:172.21.15.91:6789/0 conn(0x7f9ca8064bc0 legacy=0x7f9ca8147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca5d9b700 1 -- 172.21.15.203:0/242748769 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca814ec80 con 0x7f9ca80bf100 2021-01-06T22:03:54.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca5d9b700 1 -- 172.21.15.203:0/242748769 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f9c980014e0 con 0x7f9ca80bf100 2021-01-06T22:03:54.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca5d9b700 1 -- 172.21.15.203:0/242748769 >> v1:172.21.15.203:6789/0 conn(0x7f9ca80bf100 legacy=0x7f9ca814df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca5d9b700 1 --2- 172.21.15.203:0/242748769 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9ca8151560 0x7f9c94003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca5d9b700 1 --2- 172.21.15.203:0/242748769 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c94003c30 0x7f9c940044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca5d9b700 1 --2- 172.21.15.203:0/242748769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c940049c0 0x7f9c94004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca5d9b700 1 -- 172.21.15.203:0/242748769 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca814e9e0 con 0x7f9c94003c30 2021-01-06T22:03:54.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca5d9b700 1 -- 172.21.15.203:0/242748769 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c940041c0 con 0x7f9c940049c0 2021-01-06T22:03:54.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca5d9b700 1 -- 172.21.15.203:0/242748769 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9400b720 con 0x7f9ca8151560 2021-01-06T22:03:54.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.551+0000 7f9ca759e700 1 --2- 172.21.15.203:0/242748769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c940049c0 0x7f9c94004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9ca6d9d700 1 --2- 172.21.15.203:0/242748769 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9ca8151560 0x7f9c94003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9ca659c700 1 --2- 172.21.15.203:0/242748769 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c94003c30 0x7f9c940044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9ca759e700 1 -- 172.21.15.203:0/242748769 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9ca8151560 msgr2=0x7f9c94003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:54.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9ca759e700 1 --2- 172.21.15.203:0/242748769 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9ca8151560 0x7f9c94003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9ca759e700 1 -- 172.21.15.203:0/242748769 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c94003c30 msgr2=0x7f9c940044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:54.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9ca759e700 1 --2- 172.21.15.203:0/242748769 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c94003c30 0x7f9c940044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9ca759e700 1 -- 172.21.15.203:0/242748769 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c94003230 con 0x7f9c940049c0 2021-01-06T22:03:54.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9ca759e700 1 --2- 172.21.15.203:0/242748769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c940049c0 0x7f9c94004e40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f9ca0009e60 tx=0x7f9ca00091d0).ready entity=mon.1 client_cookie=571e07840c88181d server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:54.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9ca5d9b700 1 -- 172.21.15.203:0/242748769 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca000ed60 con 0x7f9c940049c0 2021-01-06T22:03:54.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9ca5d9b700 1 -- 172.21.15.203:0/242748769 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca000eec0 con 0x7f9c940049c0 2021-01-06T22:03:54.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9ca5d9b700 1 -- 172.21.15.203:0/242748769 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0012600 con 0x7f9c940049c0 2021-01-06T22:03:54.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9cad133700 1 -- 172.21.15.203:0/242748769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c940049c0 msgr2=0x7f9c94004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9cad133700 1 --2- 172.21.15.203:0/242748769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c940049c0 0x7f9c94004e40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f9ca0009e60 tx=0x7f9ca00091d0).stop 2021-01-06T22:03:54.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9ca759e700 1 -- 172.21.15.203:0/242748769 reap_dead start 2021-01-06T22:03:54.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9cad133700 1 -- 172.21.15.203:0/242748769 shutdown_connections 2021-01-06T22:03:54.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9cad133700 1 -- 172.21.15.203:0/242748769 >> 172.21.15.203:0/242748769 conn(0x7f9ca80602b0 msgr2=0x7f9ca80606c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:54.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9cad133700 1 -- 172.21.15.203:0/242748769 shutdown_connections 2021-01-06T22:03:54.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9cad133700 1 -- 172.21.15.203:0/242748769 wait complete. 2021-01-06T22:03:54.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9cad133700 1 Processor -- start 2021-01-06T22:03:54.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.555+0000 7f9cad133700 1 -- start start 2021-01-06T22:03:54.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9cad133700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9ca8063fb0 0x7f9ca8148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9cad133700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ca8064390 0x7f9ca815df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9cad133700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ca815e3f0 0x7f9ca814cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.568 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9cad133700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca80b2830 con 0x7f9ca815e3f0 2021-01-06T22:03:54.568 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9cad133700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca8161bf0 con 0x7f9ca8064390 2021-01-06T22:03:54.568 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9cad133700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca8161e50 con 0x7f9ca8063fb0 2021-01-06T22:03:54.568 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9ca659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ca8064390 0x7f9ca815df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.568 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9ca659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ca8064390 0x7f9ca815df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49106/0 (socket says 172.21.15.203:49106) 2021-01-06T22:03:54.569 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9ca659c700 1 -- 172.21.15.203:0/2746762075 learned_addr learned my addr 172.21.15.203:0/2746762075 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:54.569 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9ca759e700 1 --2- 172.21.15.203:0/2746762075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ca815e3f0 0x7f9ca814cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.569 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9ca6d9d700 1 --2- 172.21.15.203:0/2746762075 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9ca8063fb0 0x7f9ca8148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.569 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9ca659c700 1 -- 172.21.15.203:0/2746762075 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9ca8063fb0 msgr2=0x7f9ca8148ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.569 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9ca659c700 1 --2- 172.21.15.203:0/2746762075 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9ca8063fb0 0x7f9ca8148ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9ca659c700 1 -- 172.21.15.203:0/2746762075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ca815e3f0 msgr2=0x7f9ca814cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9ca659c700 1 --2- 172.21.15.203:0/2746762075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ca815e3f0 0x7f9ca814cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9ca659c700 1 -- 172.21.15.203:0/2746762075 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca80b2a60 con 0x7f9ca8064390 2021-01-06T22:03:54.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9ca6d9d700 1 --2- 172.21.15.203:0/2746762075 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9ca8063fb0 0x7f9ca8148ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:54.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9ca659c700 1 --2- 172.21.15.203:0/2746762075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ca8064390 0x7f9ca815df00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f9c9c00e6e0 tx=0x7f9c9c009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:54.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9c8effd700 1 -- 172.21.15.203:0/2746762075 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c010550 con 0x7f9ca8064390 2021-01-06T22:03:54.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9c8effd700 1 -- 172.21.15.203:0/2746762075 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9c010040 con 0x7f9ca8064390 2021-01-06T22:03:54.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9cad133700 1 -- 172.21.15.203:0/2746762075 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca80b2d00 con 0x7f9ca8064390 2021-01-06T22:03:54.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9c8effd700 1 -- 172.21.15.203:0/2746762075 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c0108c0 con 0x7f9ca8064390 2021-01-06T22:03:54.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9cad133700 1 -- 172.21.15.203:0/2746762075 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca815e850 con 0x7f9ca8064390 2021-01-06T22:03:54.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.559+0000 7f9cad133700 1 -- 172.21.15.203:0/2746762075 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca815dc40 con 0x7f9ca8064390 2021-01-06T22:03:54.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.563+0000 7f9c8effd700 1 -- 172.21.15.203:0/2746762075 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f9c9c010c30 con 0x7f9ca8064390 2021-01-06T22:03:54.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.563+0000 7f9c8effd700 1 --2- 172.21.15.203:0/2746762075 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9c88056400 0x7f9c880588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.563+0000 7f9c8effd700 1 -- 172.21.15.203:0/2746762075 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9c9c06f5b0 con 0x7f9ca8064390 2021-01-06T22:03:54.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.563+0000 7f9ca6d9d700 1 --2- 172.21.15.203:0/2746762075 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9c88056400 0x7f9c880588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.563+0000 7f9ca6d9d700 1 --2- 172.21.15.203:0/2746762075 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9c88056400 0x7f9c880588c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f9c980058d0 tx=0x7f9c98009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:54.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.567+0000 7f9c8effd700 1 -- 172.21.15.203:0/2746762075 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f9ca815dc40 con 0x7f9ca8064390 2021-01-06T22:03:54.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.719+0000 7f9cad133700 1 -- 172.21.15.203:0/2746762075 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9ca815dc40 con 0x7f9ca8064390 2021-01-06T22:03:54.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.719+0000 7f9c8effd700 1 -- 172.21.15.203:0/2746762075 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9ca815dc40 con 0x7f9ca8064390 2021-01-06T22:03:54.720 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:03:54.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.719+0000 7f9cad133700 1 -- 172.21.15.203:0/2746762075 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9c88056400 msgr2=0x7f9c880588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.719+0000 7f9cad133700 1 --2- 172.21.15.203:0/2746762075 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9c88056400 0x7f9c880588c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f9c980058d0 tx=0x7f9c98009040).stop 2021-01-06T22:03:54.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.719+0000 7f9cad133700 1 -- 172.21.15.203:0/2746762075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ca8064390 msgr2=0x7f9ca815df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.719+0000 7f9cad133700 1 --2- 172.21.15.203:0/2746762075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ca8064390 0x7f9ca815df00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f9c9c00e6e0 tx=0x7f9c9c009040).stop 2021-01-06T22:03:54.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.719+0000 7f9cad133700 1 -- 172.21.15.203:0/2746762075 shutdown_connections 2021-01-06T22:03:54.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.719+0000 7f9cad133700 1 --2- 172.21.15.203:0/2746762075 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9c88056400 0x7f9c880588c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.719+0000 7f9cad133700 1 --2- 172.21.15.203:0/2746762075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ca815e3f0 0x7f9ca814cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.719+0000 7f9cad133700 1 --2- 172.21.15.203:0/2746762075 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9ca8063fb0 0x7f9ca8148ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.719+0000 7f9cad133700 1 --2- 172.21.15.203:0/2746762075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ca8064390 0x7f9ca815df00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.719+0000 7f9cad133700 1 -- 172.21.15.203:0/2746762075 >> 172.21.15.203:0/2746762075 conn(0x7f9ca80602b0 msgr2=0x7f9ca814f390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:54.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.719+0000 7f9cad133700 1 -- 172.21.15.203:0/2746762075 shutdown_connections 2021-01-06T22:03:54.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.719+0000 7f9cad133700 1 -- 172.21.15.203:0/2746762075 wait complete. 2021-01-06T22:03:54.735 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-01-06T22:03:54.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd89dc68700 1 Processor -- start 2021-01-06T22:03:54.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd89dc68700 1 -- start start 2021-01-06T22:03:54.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd89dc68700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd8980c45d0 0x7fd8980c1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd89dc68700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd8980c49b0 0x7fd8981511c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd89dc68700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd898066940 0x7fd8980bdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd89dc68700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd898151ac0 con 0x7fd8980c45d0 2021-01-06T22:03:54.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd89dc68700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd898151d60 con 0x7fd8980c49b0 2021-01-06T22:03:54.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd89dc68700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd898152000 con 0x7fd898066940 2021-01-06T22:03:54.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd897fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd8980c49b0 0x7fd8981511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd897fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd8980c49b0 0x7fd8981511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49110/0 (socket says 172.21.15.203:49110) 2021-01-06T22:03:54.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd897fff700 1 -- 172.21.15.203:0/2224947694 learned_addr learned my addr 172.21.15.203:0/2224947694 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:54.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd89cc66700 1 --2- 172.21.15.203:0/2224947694 >> v2:172.21.15.91:3300/0 conn(0x7fd8980c45d0 0x7fd8980c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd89d467700 1 --2- 172.21.15.203:0/2224947694 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd898066940 0x7fd8980bdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd897fff700 1 -- 172.21.15.203:0/2224947694 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd898066940 msgr2=0x7fd8980bdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:54.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd897fff700 1 --2- 172.21.15.203:0/2224947694 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd898066940 0x7fd8980bdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd897fff700 1 -- 172.21.15.203:0/2224947694 >> v2:172.21.15.91:3300/0 conn(0x7fd8980c45d0 msgr2=0x7fd8980c1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:54.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd897fff700 1 --2- 172.21.15.203:0/2224947694 >> v2:172.21.15.91:3300/0 conn(0x7fd8980c45d0 0x7fd8980c1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd897fff700 1 -- 172.21.15.203:0/2224947694 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8980c0c60 con 0x7fd8980c49b0 2021-01-06T22:03:54.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd897fff700 1 --2- 172.21.15.203:0/2224947694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8980c49b0 0x7fd8981511c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fd8980638a0 tx=0x7fd88c006ea0).ready entity=mon.1 client_cookie=b27850a7a80fcb2c server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:54.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd8977fe700 1 -- 172.21.15.203:0/2224947694 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c0098d0 con 0x7fd8980c49b0 2021-01-06T22:03:54.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd8977fe700 1 -- 172.21.15.203:0/2224947694 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88c009a30 con 0x7fd8980c49b0 2021-01-06T22:03:54.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.863+0000 7fd8977fe700 1 -- 172.21.15.203:0/2224947694 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c009db0 con 0x7fd8980c49b0 2021-01-06T22:03:54.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 -- 172.21.15.203:0/2224947694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8980c49b0 msgr2=0x7fd8981511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 --2- 172.21.15.203:0/2224947694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8980c49b0 0x7fd8981511c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fd8980638a0 tx=0x7fd88c006ea0).stop 2021-01-06T22:03:54.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 -- 172.21.15.203:0/2224947694 shutdown_connections 2021-01-06T22:03:54.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 --2- 172.21.15.203:0/2224947694 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd898066940 0x7fd8980bdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 --2- 172.21.15.203:0/2224947694 >> v2:172.21.15.91:3300/0 conn(0x7fd8980c45d0 0x7fd8980c1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 --2- 172.21.15.203:0/2224947694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8980c49b0 0x7fd8981511c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 -- 172.21.15.203:0/2224947694 >> 172.21.15.203:0/2224947694 conn(0x7fd8980626e0 msgr2=0x7fd898062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:54.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 -- 172.21.15.203:0/2224947694 shutdown_connections 2021-01-06T22:03:54.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 -- 172.21.15.203:0/2224947694 wait complete. 2021-01-06T22:03:54.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 Processor -- start 2021-01-06T22:03:54.878 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 -- start start 2021-01-06T22:03:54.878 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd898066940 0x7fd89814e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.878 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd8980c45d0 0x7fd89815dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.878 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8980c49b0 0x7fd8981510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.878 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89814cfb0 con 0x7fd898066940 2021-01-06T22:03:54.879 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89814d1e0 con 0x7fd8980c49b0 2021-01-06T22:03:54.879 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89814d480 con 0x7fd8980c45d0 2021-01-06T22:03:54.879 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89d467700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8980c49b0 0x7fd8981510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.879 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89d467700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8980c49b0 0x7fd8981510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49120/0 (socket says 172.21.15.203:49120) 2021-01-06T22:03:54.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89d467700 1 -- 172.21.15.203:0/3695077891 learned_addr learned my addr 172.21.15.203:0/3695077891 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:54.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89cc66700 1 --2- 172.21.15.203:0/3695077891 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd898066940 0x7fd89814e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd897fff700 1 --2- 172.21.15.203:0/3695077891 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd8980c45d0 0x7fd89815dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89d467700 1 -- 172.21.15.203:0/3695077891 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd8980c45d0 msgr2=0x7fd89815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89d467700 1 --2- 172.21.15.203:0/3695077891 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd8980c45d0 0x7fd89815dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89d467700 1 -- 172.21.15.203:0/3695077891 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd898066940 msgr2=0x7fd89814e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:54.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89d467700 1 --2- 172.21.15.203:0/3695077891 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd898066940 0x7fd89814e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:54.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89d467700 1 -- 172.21.15.203:0/3695077891 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89814d720 con 0x7fd8980c49b0 2021-01-06T22:03:54.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89d467700 1 --2- 172.21.15.203:0/3695077891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8980c49b0 0x7fd8981510c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fd89000d930 tx=0x7fd890008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:54.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd894ff9700 1 -- 172.21.15.203:0/3695077891 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8900096a0 con 0x7fd8980c49b0 2021-01-06T22:03:54.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd894ff9700 1 -- 172.21.15.203:0/3695077891 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd890009800 con 0x7fd8980c49b0 2021-01-06T22:03:54.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 -- 172.21.15.203:0/3695077891 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89814d9c0 con 0x7fd8980c49b0 2021-01-06T22:03:54.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd894ff9700 1 -- 172.21.15.203:0/3695077891 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890009ba0 con 0x7fd8980c49b0 2021-01-06T22:03:54.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.867+0000 7fd89dc68700 1 -- 172.21.15.203:0/3695077891 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8981515b0 con 0x7fd8980c49b0 2021-01-06T22:03:54.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.871+0000 7fd89dc68700 1 -- 172.21.15.203:0/3695077891 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89814de60 con 0x7fd8980c49b0 2021-01-06T22:03:54.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.871+0000 7fd894ff9700 1 -- 172.21.15.203:0/3695077891 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fd890013480 con 0x7fd8980c49b0 2021-01-06T22:03:54.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.871+0000 7fd894ff9700 1 --2- 172.21.15.203:0/3695077891 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd878056400 0x7fd8780588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:54.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.871+0000 7fd894ff9700 1 -- 172.21.15.203:0/3695077891 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd89006faf0 con 0x7fd8980c49b0 2021-01-06T22:03:54.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.871+0000 7fd89cc66700 1 --2- 172.21.15.203:0/3695077891 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd878056400 0x7fd8780588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:54.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.871+0000 7fd89cc66700 1 --2- 172.21.15.203:0/3695077891 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd878056400 0x7fd8780588c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fd888006fd0 tx=0x7fd888008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:54.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:54.875+0000 7fd894ff9700 1 -- 172.21.15.203:0/3695077891 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fd89814de60 con 0x7fd8980c49b0 2021-01-06T22:03:55.032 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.031+0000 7fd89dc68700 1 -- 172.21.15.203:0/3695077891 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd89814de60 con 0x7fd8980c49b0 2021-01-06T22:03:55.033 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.031+0000 7fd894ff9700 1 -- 172.21.15.203:0/3695077891 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd89814de60 con 0x7fd8980c49b0 2021-01-06T22:03:55.033 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:03:55.035 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.035+0000 7fd89dc68700 1 -- 172.21.15.203:0/3695077891 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd878056400 msgr2=0x7fd8780588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:55.035 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.035+0000 7fd89dc68700 1 --2- 172.21.15.203:0/3695077891 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd878056400 0x7fd8780588c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fd888006fd0 tx=0x7fd888008040).stop 2021-01-06T22:03:55.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.035+0000 7fd89dc68700 1 -- 172.21.15.203:0/3695077891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8980c49b0 msgr2=0x7fd8981510c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:55.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.035+0000 7fd89dc68700 1 --2- 172.21.15.203:0/3695077891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8980c49b0 0x7fd8981510c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fd89000d930 tx=0x7fd890008040).stop 2021-01-06T22:03:55.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.035+0000 7fd89dc68700 1 -- 172.21.15.203:0/3695077891 shutdown_connections 2021-01-06T22:03:55.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.035+0000 7fd89dc68700 1 --2- 172.21.15.203:0/3695077891 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd878056400 0x7fd8780588c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.035+0000 7fd89dc68700 1 --2- 172.21.15.203:0/3695077891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8980c49b0 0x7fd8981510c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.035+0000 7fd89dc68700 1 --2- 172.21.15.203:0/3695077891 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd898066940 0x7fd89814e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.035+0000 7fd89dc68700 1 --2- 172.21.15.203:0/3695077891 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd8980c45d0 0x7fd89815dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.038 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.035+0000 7fd89dc68700 1 -- 172.21.15.203:0/3695077891 >> 172.21.15.203:0/3695077891 conn(0x7fd8980626e0 msgr2=0x7fd8980c3450 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:55.039 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.035+0000 7fd89dc68700 1 -- 172.21.15.203:0/3695077891 shutdown_connections 2021-01-06T22:03:55.039 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.035+0000 7fd89dc68700 1 -- 172.21.15.203:0/3695077891 wait complete. 2021-01-06T22:03:55.048 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-01-06T22:03:55.177 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.175+0000 7f5368a19700 1 Processor -- start 2021-01-06T22:03:55.178 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.175+0000 7f5368a19700 1 -- start start 2021-01-06T22:03:55.180 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.175+0000 7f5368a19700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53640b97a0 0x7f53640b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:55.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5368a19700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f53640bd420 0x7f5364065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:55.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5368a19700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53640be410 con 0x7f53640bed80 2021-01-06T22:03:55.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5368a19700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5364066510 con 0x7f53640bd420 2021-01-06T22:03:55.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5368a19700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53640667b0 con 0x7f53640b97a0 2021-01-06T22:03:55.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5362ffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f53640bd420 0x7f5364065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:55.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5362ffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f53640bd420 0x7f5364065c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49126/0 (socket says 172.21.15.203:49126) 2021-01-06T22:03:55.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5362ffd700 1 -- 172.21.15.203:0/70859157 learned_addr learned my addr 172.21.15.203:0/70859157 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:55.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f53637fe700 1 --2- 172.21.15.203:0/70859157 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53640b97a0 0x7f53640b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:55.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5362ffd700 1 -- 172.21.15.203:0/70859157 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53640b97a0 msgr2=0x7f53640b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:55.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5362ffd700 1 --2- 172.21.15.203:0/70859157 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53640b97a0 0x7f53640b9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5362ffd700 1 -- 172.21.15.203:0/70859157 >> v1:172.21.15.91:6789/0 conn(0x7f53640bed80 legacy=0x7f536414f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:55.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5362ffd700 1 -- 172.21.15.203:0/70859157 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5364066a50 con 0x7f53640bd420 2021-01-06T22:03:55.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5362ffd700 1 --2- 172.21.15.203:0/70859157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53640bd420 0x7f5364065c80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f535400b9e0 tx=0x7f5354006ea0).ready entity=mon.1 client_cookie=9acc15e3ae346a3c server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:55.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f53627fc700 1 -- 172.21.15.203:0/70859157 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53540098a0 con 0x7f53640bd420 2021-01-06T22:03:55.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f53627fc700 1 -- 172.21.15.203:0/70859157 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5354009a00 con 0x7f53640bd420 2021-01-06T22:03:55.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f53627fc700 1 -- 172.21.15.203:0/70859157 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354009d80 con 0x7f53640bd420 2021-01-06T22:03:55.187 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5368a19700 1 -- 172.21.15.203:0/70859157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53640bd420 msgr2=0x7f5364065c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:55.187 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5368a19700 1 --2- 172.21.15.203:0/70859157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53640bd420 0x7f5364065c80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f535400b9e0 tx=0x7f5354006ea0).stop 2021-01-06T22:03:55.187 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5368a19700 1 -- 172.21.15.203:0/70859157 shutdown_connections 2021-01-06T22:03:55.188 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5368a19700 1 --2- 172.21.15.203:0/70859157 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53640b97a0 0x7f53640b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.188 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5368a19700 1 --2- 172.21.15.203:0/70859157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53640bd420 0x7f5364065c80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.188 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5368a19700 1 -- 172.21.15.203:0/70859157 >> 172.21.15.203:0/70859157 conn(0x7f53640648f0 msgr2=0x7f5364064cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:55.188 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5368a19700 1 -- 172.21.15.203:0/70859157 shutdown_connections 2021-01-06T22:03:55.189 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.179+0000 7f5368a19700 1 -- 172.21.15.203:0/70859157 wait complete. 2021-01-06T22:03:55.189 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5368a19700 1 Processor -- start 2021-01-06T22:03:55.189 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5368a19700 1 -- start start 2021-01-06T22:03:55.189 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5368a19700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53640b97a0 0x7f53640b7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:55.189 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5368a19700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53640bd420 0x7f53640b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:55.190 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5368a19700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53640bed80 0x7f53640b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:55.190 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5368a19700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53640b6eb0 con 0x7f53640bed80 2021-01-06T22:03:55.190 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5368a19700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53640b9ed0 con 0x7f53640bd420 2021-01-06T22:03:55.190 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5368a19700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53640b82a0 con 0x7f53640b97a0 2021-01-06T22:03:55.190 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5362ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53640bd420 0x7f53640b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:55.191 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5362ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53640bd420 0x7f53640b64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49132/0 (socket says 172.21.15.203:49132) 2021-01-06T22:03:55.191 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5362ffd700 1 -- 172.21.15.203:0/1383500887 learned_addr learned my addr 172.21.15.203:0/1383500887 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:55.191 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f53637fe700 1 --2- 172.21.15.203:0/1383500887 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53640b97a0 0x7f53640b7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:55.191 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5363fff700 1 --2- 172.21.15.203:0/1383500887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53640bed80 0x7f53640b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:55.191 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5362ffd700 1 -- 172.21.15.203:0/1383500887 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53640b97a0 msgr2=0x7f53640b7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:55.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5362ffd700 1 --2- 172.21.15.203:0/1383500887 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53640b97a0 0x7f53640b7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5362ffd700 1 -- 172.21.15.203:0/1383500887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53640bed80 msgr2=0x7f53640b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:55.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5362ffd700 1 --2- 172.21.15.203:0/1383500887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53640bed80 0x7f53640b69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5362ffd700 1 -- 172.21.15.203:0/1383500887 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53640a33c0 con 0x7f53640bd420 2021-01-06T22:03:55.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5362ffd700 1 --2- 172.21.15.203:0/1383500887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53640bd420 0x7f53640b64d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f5364057c10 tx=0x7f5354016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:55.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f535b7fe700 1 -- 172.21.15.203:0/1383500887 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354012ce0 con 0x7f53640bd420 2021-01-06T22:03:55.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f535b7fe700 1 -- 172.21.15.203:0/1383500887 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5354012e40 con 0x7f53640bd420 2021-01-06T22:03:55.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.183+0000 7f5368a19700 1 -- 172.21.15.203:0/1383500887 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53640a3660 con 0x7f53640bd420 2021-01-06T22:03:55.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.187+0000 7f535b7fe700 1 -- 172.21.15.203:0/1383500887 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535401c650 con 0x7f53640bd420 2021-01-06T22:03:55.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.187+0000 7f5368a19700 1 -- 172.21.15.203:0/1383500887 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53640b6030 con 0x7f53640bd420 2021-01-06T22:03:55.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.187+0000 7f5368a19700 1 -- 172.21.15.203:0/1383500887 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53640a3b00 con 0x7f53640bd420 2021-01-06T22:03:55.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.187+0000 7f535b7fe700 1 -- 172.21.15.203:0/1383500887 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f535400f070 con 0x7f53640bd420 2021-01-06T22:03:55.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.187+0000 7f535b7fe700 1 --2- 172.21.15.203:0/1383500887 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f53480563d0 0x7f5348058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:55.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.187+0000 7f535b7fe700 1 -- 172.21.15.203:0/1383500887 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5354070ce0 con 0x7f53640bd420 2021-01-06T22:03:55.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.187+0000 7f53637fe700 1 --2- 172.21.15.203:0/1383500887 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f53480563d0 0x7f5348058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:55.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.187+0000 7f53637fe700 1 --2- 172.21.15.203:0/1383500887 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f53480563d0 0x7f5348058890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f534c00bb50 tx=0x7f534c006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:55.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.195+0000 7f535b7fe700 1 -- 172.21.15.203:0/1383500887 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f53640a3b00 con 0x7f53640bd420 2021-01-06T22:03:55.348 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.347+0000 7f5368a19700 1 -- 172.21.15.203:0/1383500887 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f53640a3b00 con 0x7f53640bd420 2021-01-06T22:03:55.348 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.347+0000 7f535b7fe700 1 -- 172.21.15.203:0/1383500887 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f53640a3b00 con 0x7f53640bd420 2021-01-06T22:03:55.349 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:03:55.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.347+0000 7f5368a19700 1 -- 172.21.15.203:0/1383500887 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f53480563d0 msgr2=0x7f5348058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:55.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.347+0000 7f5368a19700 1 --2- 172.21.15.203:0/1383500887 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f53480563d0 0x7f5348058890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f534c00bb50 tx=0x7f534c006de0).stop 2021-01-06T22:03:55.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.347+0000 7f5368a19700 1 -- 172.21.15.203:0/1383500887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53640bd420 msgr2=0x7f53640b64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:55.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.347+0000 7f5368a19700 1 --2- 172.21.15.203:0/1383500887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53640bd420 0x7f53640b64d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f5364057c10 tx=0x7f5354016000).stop 2021-01-06T22:03:55.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.347+0000 7f5368a19700 1 -- 172.21.15.203:0/1383500887 shutdown_connections 2021-01-06T22:03:55.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.347+0000 7f5368a19700 1 --2- 172.21.15.203:0/1383500887 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f53480563d0 0x7f5348058890 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.347+0000 7f5368a19700 1 --2- 172.21.15.203:0/1383500887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53640bed80 0x7f53640b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.347+0000 7f5368a19700 1 --2- 172.21.15.203:0/1383500887 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53640b97a0 0x7f53640b7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.347+0000 7f5368a19700 1 --2- 172.21.15.203:0/1383500887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53640bd420 0x7f53640b64d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.347+0000 7f5368a19700 1 -- 172.21.15.203:0/1383500887 >> 172.21.15.203:0/1383500887 conn(0x7f53640648f0 msgr2=0x7f53640bb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:55.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.347+0000 7f5368a19700 1 -- 172.21.15.203:0/1383500887 shutdown_connections 2021-01-06T22:03:55.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.347+0000 7f5368a19700 1 -- 172.21.15.203:0/1383500887 wait complete. 2021-01-06T22:03:55.363 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:03:55.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d13fff700 1 Processor -- start 2021-01-06T22:03:55.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d13fff700 1 -- start start 2021-01-06T22:03:55.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d13fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9d140c44f0 0x7f9d140c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:55.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d13fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9d140c48d0 0x7f9d140c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:55.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d13fff700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d140c0f50 con 0x7f9d14060fc0 2021-01-06T22:03:55.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d13fff700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d140c1d60 con 0x7f9d140c44f0 2021-01-06T22:03:55.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d13fff700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d14066cf0 con 0x7f9d140c48d0 2021-01-06T22:03:55.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d0bfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9d140c48d0 0x7f9d140c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:55.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d137fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f9d14060fc0 0x7f9d14150730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47734/0 (socket says 172.21.15.203:47734) 2021-01-06T22:03:55.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d0bfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9d140c48d0 0x7f9d140c1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49140/0 (socket says 172.21.15.203:49140) 2021-01-06T22:03:55.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d137fe700 1 -- 172.21.15.203:0/25845731 learned_addr learned my addr 172.21.15.203:0/25845731 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:55.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d12ffd700 1 --2- 172.21.15.203:0/25845731 >> v2:172.21.15.91:3300/0 conn(0x7f9d140c44f0 0x7f9d140c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:55.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d0bfff700 1 -- 172.21.15.203:0/25845731 >> v2:172.21.15.91:3300/0 conn(0x7f9d140c44f0 msgr2=0x7f9d140c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:55.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d0bfff700 1 --2- 172.21.15.203:0/25845731 >> v2:172.21.15.91:3300/0 conn(0x7f9d140c44f0 0x7f9d140c1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d0bfff700 1 -- 172.21.15.203:0/25845731 >> v1:172.21.15.203:6789/0 conn(0x7f9d14060fc0 legacy=0x7f9d14150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:55.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d0bfff700 1 -- 172.21.15.203:0/25845731 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d14060750 con 0x7f9d140c48d0 2021-01-06T22:03:55.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d0bfff700 1 --2- 172.21.15.203:0/25845731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d140c48d0 0x7f9d140c1870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f9d14061480 tx=0x7f9cfc006de0).ready entity=mon.1 client_cookie=91f097204ce3b223 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:55.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d127fc700 1 -- 172.21.15.203:0/25845731 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0096c0 con 0x7f9d140c48d0 2021-01-06T22:03:55.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d127fc700 1 -- 172.21.15.203:0/25845731 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc009820 con 0x7f9d140c48d0 2021-01-06T22:03:55.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d127fc700 1 -- 172.21.15.203:0/25845731 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc009ba0 con 0x7f9d140c48d0 2021-01-06T22:03:55.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d13fff700 1 -- 172.21.15.203:0/25845731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d140c48d0 msgr2=0x7f9d140c1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:55.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d13fff700 1 --2- 172.21.15.203:0/25845731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d140c48d0 0x7f9d140c1870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f9d14061480 tx=0x7f9cfc006de0).stop 2021-01-06T22:03:55.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d13fff700 1 -- 172.21.15.203:0/25845731 shutdown_connections 2021-01-06T22:03:55.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d13fff700 1 --2- 172.21.15.203:0/25845731 >> v2:172.21.15.91:3300/0 conn(0x7f9d140c44f0 0x7f9d140c1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d13fff700 1 --2- 172.21.15.203:0/25845731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d140c48d0 0x7f9d140c1870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d13fff700 1 -- 172.21.15.203:0/25845731 >> 172.21.15.203:0/25845731 conn(0x7f9d14056cd0 msgr2=0x7f9d14061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:55.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d13fff700 1 -- 172.21.15.203:0/25845731 shutdown_connections 2021-01-06T22:03:55.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.487+0000 7f9d13fff700 1 -- 172.21.15.203:0/25845731 wait complete. 2021-01-06T22:03:55.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d13fff700 1 Processor -- start 2021-01-06T22:03:55.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d13fff700 1 -- start start 2021-01-06T22:03:55.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d13fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9d14060fc0 0x7f9d1414a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:55.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d13fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d140c44f0 0x7f9d140b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:55.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d13fff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9d140c48d0 0x7f9d1414ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:55.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d13fff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d14153470 con 0x7f9d140c48d0 2021-01-06T22:03:55.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d13fff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d141543e0 con 0x7f9d140c44f0 2021-01-06T22:03:55.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d13fff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d141531d0 con 0x7f9d14060fc0 2021-01-06T22:03:55.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d0bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d140c44f0 0x7f9d140b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:55.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d0bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d140c44f0 0x7f9d140b9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49148/0 (socket says 172.21.15.203:49148) 2021-01-06T22:03:55.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d0bfff700 1 -- 172.21.15.203:0/2312307258 learned_addr learned my addr 172.21.15.203:0/2312307258 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:55.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d137fe700 1 --2- 172.21.15.203:0/2312307258 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9d140c48d0 0x7f9d1414ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:55.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d12ffd700 1 --2- 172.21.15.203:0/2312307258 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9d14060fc0 0x7f9d1414a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:55.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d0bfff700 1 -- 172.21.15.203:0/2312307258 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9d14060fc0 msgr2=0x7f9d1414a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:55.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d0bfff700 1 --2- 172.21.15.203:0/2312307258 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9d14060fc0 0x7f9d1414a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d0bfff700 1 -- 172.21.15.203:0/2312307258 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9d140c48d0 msgr2=0x7f9d1414ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:55.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d0bfff700 1 --2- 172.21.15.203:0/2312307258 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9d140c48d0 0x7f9d1414ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d0bfff700 1 -- 172.21.15.203:0/2312307258 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d14153760 con 0x7f9d140c44f0 2021-01-06T22:03:55.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d0bfff700 1 --2- 172.21.15.203:0/2312307258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d140c44f0 0x7f9d140b9e30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f9cfc006e20 tx=0x7f9cfc016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:55.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d0affd700 1 -- 172.21.15.203:0/2312307258 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc01c630 con 0x7f9d140c44f0 2021-01-06T22:03:55.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d0affd700 1 -- 172.21.15.203:0/2312307258 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc01c790 con 0x7f9d140c44f0 2021-01-06T22:03:55.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d13fff700 1 -- 172.21.15.203:0/2312307258 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d14153a00 con 0x7f9d140c44f0 2021-01-06T22:03:55.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d0affd700 1 -- 172.21.15.203:0/2312307258 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc01cb30 con 0x7f9d140c44f0 2021-01-06T22:03:55.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.491+0000 7f9d13fff700 1 -- 172.21.15.203:0/2312307258 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d14153f40 con 0x7f9d140c44f0 2021-01-06T22:03:55.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.495+0000 7f9d13fff700 1 -- 172.21.15.203:0/2312307258 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d140450a0 con 0x7f9d140c44f0 2021-01-06T22:03:55.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.495+0000 7f9d0affd700 1 -- 172.21.15.203:0/2312307258 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f9cfc00f070 con 0x7f9d140c44f0 2021-01-06T22:03:55.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.495+0000 7f9d0affd700 1 --2- 172.21.15.203:0/2312307258 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9cf80563d0 0x7f9cf8058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:55.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.495+0000 7f9d0affd700 1 -- 172.21.15.203:0/2312307258 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9cfc070e70 con 0x7f9d140c44f0 2021-01-06T22:03:55.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.495+0000 7f9d12ffd700 1 --2- 172.21.15.203:0/2312307258 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9cf80563d0 0x7f9cf8058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:55.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.495+0000 7f9d12ffd700 1 --2- 172.21.15.203:0/2312307258 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9cf80563d0 0x7f9cf8058890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f9d140be9e0 tx=0x7f9d04008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:55.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.499+0000 7f9d0affd700 1 -- 172.21.15.203:0/2312307258 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f9cfc0466f0 con 0x7f9d140c44f0 2021-01-06T22:03:55.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.699+0000 7f9d13fff700 1 -- 172.21.15.203:0/2312307258 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d1405ca60 con 0x7f9d140c44f0 2021-01-06T22:03:55.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.699+0000 7f9d0affd700 1 -- 172.21.15.203:0/2312307258 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+3574 (secure 0 0 0) 0x7f9cfc049d10 con 0x7f9d140c44f0 2021-01-06T22:03:55.703 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:55.703 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":5}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:03:44.792784+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1116012025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1116012025},{"type":"v1","addr":"172.21.15.203:6835","nonce":1116012025}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:03:55.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.703+0000 7f9d13fff700 1 -- 172.21.15.203:0/2312307258 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9cf80563d0 msgr2=0x7f9cf8058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:55.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.703+0000 7f9d13fff700 1 --2- 172.21.15.203:0/2312307258 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9cf80563d0 0x7f9cf8058890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f9d140be9e0 tx=0x7f9d04008040).stop 2021-01-06T22:03:55.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.703+0000 7f9d13fff700 1 -- 172.21.15.203:0/2312307258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d140c44f0 msgr2=0x7f9d140b9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:55.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.703+0000 7f9d13fff700 1 --2- 172.21.15.203:0/2312307258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d140c44f0 0x7f9d140b9e30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f9cfc006e20 tx=0x7f9cfc016000).stop 2021-01-06T22:03:55.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.703+0000 7f9d13fff700 1 -- 172.21.15.203:0/2312307258 shutdown_connections 2021-01-06T22:03:55.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.703+0000 7f9d13fff700 1 --2- 172.21.15.203:0/2312307258 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9cf80563d0 0x7f9cf8058890 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.703+0000 7f9d13fff700 1 --2- 172.21.15.203:0/2312307258 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9d140c48d0 0x7f9d1414ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.703+0000 7f9d13fff700 1 --2- 172.21.15.203:0/2312307258 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9d14060fc0 0x7f9d1414a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.703+0000 7f9d13fff700 1 --2- 172.21.15.203:0/2312307258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d140c44f0 0x7f9d140b9e30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:55.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.703+0000 7f9d13fff700 1 -- 172.21.15.203:0/2312307258 >> 172.21.15.203:0/2312307258 conn(0x7f9d14056cd0 msgr2=0x7f9d1414fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:55.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.703+0000 7f9d13fff700 1 -- 172.21.15.203:0/2312307258 shutdown_connections 2021-01-06T22:03:55.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:55.703+0000 7f9d13fff700 1 -- 172.21.15.203:0/2312307258 wait complete. 2021-01-06T22:03:55.709 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 12 2021-01-06T22:03:56.327 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 12, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-01-06T22:03:39.741327+0000', 'modified': '2021-01-06T22:03:44.792784+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4274, 'mds_1': 4273, 'mds_2': 4270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4270': {'gid': 4270, 'name': 'c', 'rank': 2, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6835/1116012025', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 1116012025}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 1116012025}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4273': {'gid': 4273, 'name': 'a', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6837/3277561222', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 3277561222}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 3277561222}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4274': {'gid': 4274, 'name': 'f', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.91:6837/3888667247', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.91:6835', 'nonce': 3888667247}, {'type': 'v1', 'addr': '172.21.15.91:6837', 'nonce': 3888667247}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-01-06T22:03:56.327 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2021-01-06T22:03:56.328 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-01-06T22:03:56.469 INFO:teuthology.orchestra.run.smithi203.stdout:{"cluster_fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","whoami":2,"id":4270,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":12.667376207,"mdsmap_epoch":12,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":20.169299487} 2021-01-06T22:03:56.469 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "f96dbde2-6d40-40dc-b34a-9fb254ce805f", "fs_name": "cephfs", "id": 4270, "mdsmap_epoch": 12, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 12.667376207, "state": "up:active", "uptime": 20.169299487, "want_state": "up:active", "whoami": 2 } 2021-01-06T22:03:56.470 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-01-06T22:03:56.600 INFO:teuthology.orchestra.run.smithi203.stdout:{"cluster_fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","whoami":1,"id":4273,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":13.814937992000001,"mdsmap_epoch":12,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":20.299267793999999} 2021-01-06T22:03:56.600 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "f96dbde2-6d40-40dc-b34a-9fb254ce805f", "fs_name": "cephfs", "id": 4273, "mdsmap_epoch": 12, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 13.814937992, "state": "up:active", "uptime": 20.299267794, "want_state": "up:active", "whoami": 1 } 2021-01-06T22:03:56.601 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.f.asok --format=json status 2021-01-06T22:03:56.742 INFO:teuthology.orchestra.run.smithi091.stdout:{"cluster_fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","whoami":0,"id":4274,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":16.989840410999999,"mdsmap_epoch":12,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":20.418382287} 2021-01-06T22:03:56.743 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "f96dbde2-6d40-40dc-b34a-9fb254ce805f", "fs_name": "cephfs", "id": 4274, "mdsmap_epoch": 12, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 16.989840411, "state": "up:active", "uptime": 20.418382287, "want_state": "up:active", "whoami": 0 } 2021-01-06T22:03:56.743 INFO:teuthology.run_tasks:Running task ceph-fuse... 2021-01-06T22:03:56.765 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2021-01-06T22:03:56.766 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2021-01-06T22:03:56.766 INFO:tasks.ceph_fuse:client.0 config is {} 2021-01-06T22:03:56.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-01-06T22:03:56.767 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2021-01-06T22:03:56.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-01-06T22:03:56.777 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2021-01-06T22:03:56.839 INFO:teuthology.orchestra.run.smithi203.stderr:Cannot find device "ceph-brx" 2021-01-06T22:03:56.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-01-06T22:03:56.841 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2021-01-06T22:03:56.841 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2021-01-06T22:03:56.842 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2021-01-06T22:03:56.842 INFO:tasks.cephfs.mount:self.client.name = client.0 2021-01-06T22:03:56.842 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2021-01-06T22:03:56.843 INFO:tasks.cephfs.mount:self.cephfs_name = None 2021-01-06T22:03:56.843 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2021-01-06T22:03:56.843 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2021-01-06T22:03:56.844 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2021-01-06T22:03:56.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-01-06T22:03:56.844 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2021-01-06T22:03:56.889 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2021-01-06T22:03:56.889 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2021-01-06T22:03:56.889 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2021-01-06T22:03:56.890 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2021-01-06T22:03:56.890 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2021-01-06T22:03:56.890 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2021-01-06T22:03:56.890 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2021-01-06T22:03:56.891 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2021-01-06T22:03:56.891 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2021-01-06T22:03:56.891 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2021-01-06T22:03:56.892 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2021-01-06T22:03:56.892 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2021-01-06T22:03:56.892 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2021-01-06T22:03:56.892 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2021-01-06T22:03:56.893 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global enp3s0f1 2021-01-06T22:03:56.893 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2021-01-06T22:03:56.893 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::ec4:7aff:fe8f:4a23/64 scope link 2021-01-06T22:03:56.895 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2021-01-06T22:03:56.895 INFO:teuthology.orchestra.run.smithi203.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2021-01-06T22:03:56.895 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 02:42:d2:a2:a5:fb brd ff:ff:ff:ff:ff:ff 2021-01-06T22:03:56.895 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2021-01-06T22:03:56.896 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2021-01-06T22:03:56.897 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2021-01-06T22:03:56.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-01-06T22:03:56.897 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec sudo bash -c ' 2021-01-06T22:03:56.898 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2021-01-06T22:03:56.898 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add name ceph-brx type bridge 2021-01-06T22:03:56.899 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip addr flush dev ceph-brx 2021-01-06T22:03:56.899 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set ceph-brx up 2021-01-06T22:03:56.899 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2021-01-06T22:03:56.900 DEBUG:teuthology.orchestra.run.smithi203:> ') 2021-01-06T22:03:56.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-01-06T22:03:56.986 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2021-01-06T22:03:57.000 INFO:teuthology.orchestra.run.smithi203.stdout:1 2021-01-06T22:03:57.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-01-06T22:03:57.001 DEBUG:teuthology.orchestra.run.smithi203:> route 2021-01-06T22:03:57.068 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2021-01-06T22:03:57.069 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2021-01-06T22:03:57.069 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2021-01-06T22:03:57.069 INFO:teuthology.orchestra.run.smithi203.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2021-01-06T22:03:57.069 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2021-01-06T22:03:57.070 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2021-01-06T22:03:57.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-01-06T22:03:57.071 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec sudo bash -c ' 2021-01-06T22:03:57.071 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2021-01-06T22:03:57.071 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2021-01-06T22:03:57.072 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2021-01-06T22:03:57.072 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2021-01-06T22:03:57.072 DEBUG:teuthology.orchestra.run.smithi203:> ') 2021-01-06T22:03:57.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-01-06T22:03:57.156 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2021-01-06T22:03:57.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-01-06T22:03:57.162 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list-id 2021-01-06T22:03:57.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-01-06T22:03:57.209 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec sudo bash -c ' 2021-01-06T22:03:57.210 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2021-01-06T22:03:57.210 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2021-01-06T22:03:57.210 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2021-01-06T22:03:57.211 DEBUG:teuthology.orchestra.run.smithi203:> ') 2021-01-06T22:03:57.289 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2021-01-06T22:03:57.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-01-06T22:03:57.289 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec sudo bash -c ' 2021-01-06T22:03:57.290 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2021-01-06T22:03:57.291 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2021-01-06T22:03:57.292 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2021-01-06T22:03:57.292 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2021-01-06T22:03:57.292 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2021-01-06T22:03:57.293 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2021-01-06T22:03:57.293 DEBUG:teuthology.orchestra.run.smithi203:> ') 2021-01-06T22:03:57.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-01-06T22:03:57.598 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec sudo bash -c ' 2021-01-06T22:03:57.599 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2021-01-06T22:03:57.599 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2021-01-06T22:03:57.599 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set dev brx.0 master ceph-brx 2021-01-06T22:03:57.600 DEBUG:teuthology.orchestra.run.smithi203:> ') 2021-01-06T22:03:57.638 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-06T22:03:57.809 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.807+0000 7fc3bf59e700 1 Processor -- start 2021-01-06T22:03:57.810 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.807+0000 7fc3bf59e700 1 -- start start 2021-01-06T22:03:57.812 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bf59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3c0064380 0x7fc3c00c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:57.812 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bf59e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc3c014e310 0x7fc3c01471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:57.813 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bf59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c00bb2d0 con 0x7fc3c0064760 2021-01-06T22:03:57.813 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bf59e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c014a560 con 0x7fc3c014e310 2021-01-06T22:03:57.813 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bf59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c014a800 con 0x7fc3c0064380 2021-01-06T22:03:57.813 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3b5d9b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc3c014e310 0x7fc3c01471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:57.813 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3b5d9b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc3c014e310 0x7fc3c01471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49158/0 (socket says 172.21.15.203:49158) 2021-01-06T22:03:57.814 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3be59c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3c0064380 0x7fc3c00c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:57.814 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3b5d9b700 1 -- 172.21.15.203:0/430259297 learned_addr learned my addr 172.21.15.203:0/430259297 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:57.815 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3b5d9b700 1 -- 172.21.15.203:0/430259297 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3c0064380 msgr2=0x7fc3c00c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:57.815 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3b5d9b700 1 --2- 172.21.15.203:0/430259297 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3c0064380 0x7fc3c00c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:57.816 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3b5d9b700 1 -- 172.21.15.203:0/430259297 >> v1:172.21.15.91:6789/0 conn(0x7fc3c0064760 legacy=0x7fc3c00bb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:57.816 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3b5d9b700 1 -- 172.21.15.203:0/430259297 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c014aaa0 con 0x7fc3c014e310 2021-01-06T22:03:57.816 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bdd9b700 1 -- 172.21.15.203:0/430259297 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213968535 0 0) 0x7fc3c00bb2d0 con 0x7fc3c0064760 2021-01-06T22:03:57.816 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3b5d9b700 1 --2- 172.21.15.203:0/430259297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3c014e310 0x7fc3c01471b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fc3a800b9e0 tx=0x7fc3a8006ea0).ready entity=mon.1 client_cookie=c64ce10ea966d596 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:57.816 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bdd9b700 1 -- 172.21.15.203:0/430259297 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a80098a0 con 0x7fc3c014e310 2021-01-06T22:03:57.817 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bdd9b700 1 -- 172.21.15.203:0/430259297 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a8009a00 con 0x7fc3c014e310 2021-01-06T22:03:57.817 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bdd9b700 1 -- 172.21.15.203:0/430259297 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8009d80 con 0x7fc3c014e310 2021-01-06T22:03:57.817 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bf59e700 1 -- 172.21.15.203:0/430259297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3c014e310 msgr2=0x7fc3c01471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:57.817 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bf59e700 1 --2- 172.21.15.203:0/430259297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3c014e310 0x7fc3c01471b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fc3a800b9e0 tx=0x7fc3a8006ea0).stop 2021-01-06T22:03:57.817 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bf59e700 1 -- 172.21.15.203:0/430259297 shutdown_connections 2021-01-06T22:03:57.818 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bf59e700 1 --2- 172.21.15.203:0/430259297 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3c0064380 0x7fc3c00c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:57.818 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bf59e700 1 --2- 172.21.15.203:0/430259297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3c014e310 0x7fc3c01471b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:57.818 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bf59e700 1 -- 172.21.15.203:0/430259297 >> 172.21.15.203:0/430259297 conn(0x7fc3c00667b0 msgr2=0x7fc3c00c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:57.818 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bf59e700 1 -- 172.21.15.203:0/430259297 shutdown_connections 2021-01-06T22:03:57.821 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.811+0000 7fc3bf59e700 1 -- 172.21.15.203:0/430259297 wait complete. 2021-01-06T22:03:57.821 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3bf59e700 1 Processor -- start 2021-01-06T22:03:57.821 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3bf59e700 1 -- start start 2021-01-06T22:03:57.822 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3bf59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3c0064380 0x7fc3c0160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:57.822 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3bf59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3c0064760 0x7fc3c0161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:57.822 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3bf59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3c014e310 0x7fc3c00b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:57.822 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3bf59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c0146060 con 0x7fc3c014e310 2021-01-06T22:03:57.822 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3bf59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c0161710 con 0x7fc3c0064760 2021-01-06T22:03:57.823 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3bf59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c01462d0 con 0x7fc3c0064380 2021-01-06T22:03:57.823 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3b5d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3c0064760 0x7fc3c0161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:57.823 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3b5d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3c0064760 0x7fc3c0161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49162/0 (socket says 172.21.15.203:49162) 2021-01-06T22:03:57.823 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3b5d9b700 1 -- 172.21.15.203:0/2632563378 learned_addr learned my addr 172.21.15.203:0/2632563378 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:57.823 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3be59c700 1 --2- 172.21.15.203:0/2632563378 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3c0064380 0x7fc3c0160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:57.824 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3bed9d700 1 --2- 172.21.15.203:0/2632563378 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3c014e310 0x7fc3c00b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:57.824 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3b5d9b700 1 -- 172.21.15.203:0/2632563378 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3c0064380 msgr2=0x7fc3c0160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:57.824 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3b5d9b700 1 --2- 172.21.15.203:0/2632563378 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3c0064380 0x7fc3c0160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:57.824 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3b5d9b700 1 -- 172.21.15.203:0/2632563378 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3c014e310 msgr2=0x7fc3c00b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:57.824 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3b5d9b700 1 --2- 172.21.15.203:0/2632563378 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3c014e310 0x7fc3c00b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:57.825 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3b5d9b700 1 -- 172.21.15.203:0/2632563378 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c0146570 con 0x7fc3c0064760 2021-01-06T22:03:57.825 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3bed9d700 1 --2- 172.21.15.203:0/2632563378 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3c014e310 0x7fc3c00b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:57.825 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3be59c700 1 --2- 172.21.15.203:0/2632563378 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3c0064380 0x7fc3c0160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:57.825 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3b5d9b700 1 --2- 172.21.15.203:0/2632563378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3c0064760 0x7fc3c0161060 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fc3c00bae20 tx=0x7fc3a8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:57.825 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3b6ffd700 1 -- 172.21.15.203:0/2632563378 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8012e40 con 0x7fc3c0064760 2021-01-06T22:03:57.826 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3b6ffd700 1 -- 172.21.15.203:0/2632563378 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a801c420 con 0x7fc3c0064760 2021-01-06T22:03:57.826 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3b6ffd700 1 -- 172.21.15.203:0/2632563378 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a801c7c0 con 0x7fc3c0064760 2021-01-06T22:03:57.826 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3bf59e700 1 -- 172.21.15.203:0/2632563378 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c0147020 con 0x7fc3c0064760 2021-01-06T22:03:57.826 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.815+0000 7fc3bf59e700 1 -- 172.21.15.203:0/2632563378 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c00be080 con 0x7fc3c0064760 2021-01-06T22:03:57.827 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.819+0000 7fc3bf59e700 1 -- 172.21.15.203:0/2632563378 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c00be210 con 0x7fc3c0064760 2021-01-06T22:03:57.827 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.819+0000 7fc3b6ffd700 1 -- 172.21.15.203:0/2632563378 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fc3a800f070 con 0x7fc3c0064760 2021-01-06T22:03:57.827 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.819+0000 7fc3b6ffd700 1 --2- 172.21.15.203:0/2632563378 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc3a40563d0 0x7fc3a4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:57.827 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.819+0000 7fc3be59c700 1 --2- 172.21.15.203:0/2632563378 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc3a40563d0 0x7fc3a4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:57.827 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.819+0000 7fc3b6ffd700 1 -- 172.21.15.203:0/2632563378 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc3a8070f20 con 0x7fc3c0064760 2021-01-06T22:03:57.828 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.819+0000 7fc3be59c700 1 --2- 172.21.15.203:0/2632563378 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc3a40563d0 0x7fc3a4058890 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fc3b000bb20 tx=0x7fc3b0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:57.828 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.823+0000 7fc3b6ffd700 1 -- 172.21.15.203:0/2632563378 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fc3a8046690 con 0x7fc3c0064760 2021-01-06T22:03:57.978 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.975+0000 7fc3bf59e700 1 -- 172.21.15.203:0/2632563378 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3c00450a0 con 0x7fc3c0064760 2021-01-06T22:03:57.980 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.979+0000 7fc3b6ffd700 1 -- 172.21.15.203:0/2632563378 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16826 (secure 0 0 0) 0x7fc3a8049cb0 con 0x7fc3c0064760 2021-01-06T22:03:57.981 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:57.981 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":23,"fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","created":"2021-01-06T22:03:21.322671+0000","modified":"2021-01-06T22:03:40.744826+0000","last_up_change":"2021-01-06T22:03:31.115611+0000","last_in_change":"2021-01-06T22:03:31.115611+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-06T22:03:25.161465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-06T22:03:35.070364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-01-06T22:03:37.862354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-01-06T22:03:38.738416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e0b35c09-147c-4ccc-b566-c48b81e25412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":18347},{"type":"v1","addr":"172.21.15.203:6815","nonce":18347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":18347},{"type":"v1","addr":"172.21.15.203:6818","nonce":18347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":18347},{"type":"v1","addr":"172.21.15.203:6831","nonce":18347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6821","nonce":18347},{"type":"v1","addr":"172.21.15.203:6827","nonce":18347}]},"public_addr":"172.21.15.203:6815/18347","cluster_addr":"172.21.15.203:6818/18347","heartbeat_back_addr":"172.21.15.203:6831/18347","heartbeat_front_addr":"172.21.15.203:6827/18347","state":["exists","up"]},{"osd":1,"uuid":"8add59a0-2d58-4c1b-a8d1-62049ee9d85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":18337},{"type":"v1","addr":"172.21.15.203:6803","nonce":18337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":18337},{"type":"v1","addr":"172.21.15.203:6805","nonce":18337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":18337},{"type":"v1","addr":"172.21.15.203:6809","nonce":18337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":18337},{"type":"v1","addr":"172.21.15.203:6807","nonce":18337}]},"public_addr":"172.21.15.203:6803/18337","cluster_addr":"172.21.15.203:6805/18337","heartbeat_back_addr":"172.21.15.203:6809/18337","heartbeat_front_addr":"172.21.15.203:6807/18337","state":["exists","up"]},{"osd":2,"uuid":"b294a411-c7c9-477e-8eae-3b7bd0563ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":18354},{"type":"v1","addr":"172.21.15.203:6811","nonce":18354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":18354},{"type":"v1","addr":"172.21.15.203:6819","nonce":18354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6825","nonce":18354},{"type":"v1","addr":"172.21.15.203:6828","nonce":18354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6823","nonce":18354},{"type":"v1","addr":"172.21.15.203:6824","nonce":18354}]},"public_addr":"172.21.15.203:6811/18354","cluster_addr":"172.21.15.203:6819/18354","heartbeat_back_addr":"172.21.15.203:6828/18354","heartbeat_front_addr":"172.21.15.203:6824/18354","state":["exists","up"]},{"osd":3,"uuid":"5878a8f7-1a42-41c3-9810-0ac41a54dd6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":18346},{"type":"v1","addr":"172.21.15.203:6817","nonce":18346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":18346},{"type":"v1","addr":"172.21.15.203:6822","nonce":18346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":18346},{"type":"v1","addr":"172.21.15.203:6833","nonce":18346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":18346},{"type":"v1","addr":"172.21.15.203:6829","nonce":18346}]},"public_addr":"172.21.15.203:6817/18346","cluster_addr":"172.21.15.203:6822/18346","heartbeat_back_addr":"172.21.15.203:6833/18346","heartbeat_front_addr":"172.21.15.203:6829/18346","state":["exists","up"]},{"osd":4,"uuid":"f641a345-81e4-4400-a44c-d2f82fef7f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":18370},{"type":"v1","addr":"172.21.15.91:6809","nonce":18370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":18370},{"type":"v1","addr":"172.21.15.91:6811","nonce":18370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":18370},{"type":"v1","addr":"172.21.15.91:6816","nonce":18370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":18370},{"type":"v1","addr":"172.21.15.91:6813","nonce":18370}]},"public_addr":"172.21.15.91:6809/18370","cluster_addr":"172.21.15.91:6811/18370","heartbeat_back_addr":"172.21.15.91:6816/18370","heartbeat_front_addr":"172.21.15.91:6813/18370","state":["exists","up"]},{"osd":5,"uuid":"c1ff150d-68c6-4e6b-abd4-76af27bb2103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":18372},{"type":"v1","addr":"172.21.15.91:6801","nonce":18372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":18372},{"type":"v1","addr":"172.21.15.91:6803","nonce":18372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":18372},{"type":"v1","addr":"172.21.15.91:6807","nonce":18372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":18372},{"type":"v1","addr":"172.21.15.91:6805","nonce":18372}]},"public_addr":"172.21.15.91:6801/18372","cluster_addr":"172.21.15.91:6803/18372","heartbeat_back_addr":"172.21.15.91:6807/18372","heartbeat_front_addr":"172.21.15.91:6805/18372","state":["exists","up"]},{"osd":6,"uuid":"35a58e1a-0a0a-4f02-a477-adc029038d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6815","nonce":18371},{"type":"v1","addr":"172.21.15.91:6817","nonce":18371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":18371},{"type":"v1","addr":"172.21.15.91:6819","nonce":18371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":18371},{"type":"v1","addr":"172.21.15.91:6823","nonce":18371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":18371},{"type":"v1","addr":"172.21.15.91:6821","nonce":18371}]},"public_addr":"172.21.15.91:6817/18371","cluster_addr":"172.21.15.91:6819/18371","heartbeat_back_addr":"172.21.15.91:6823/18371","heartbeat_front_addr":"172.21.15.91:6821/18371","state":["exists","up"]},{"osd":7,"uuid":"809a2d89-4303-482a-8558-aef907814234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":18373},{"type":"v1","addr":"172.21.15.91:6825","nonce":18373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":18373},{"type":"v1","addr":"172.21.15.91:6827","nonce":18373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":18373},{"type":"v1","addr":"172.21.15.91:6831","nonce":18373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":18373},{"type":"v1","addr":"172.21.15.91:6829","nonce":18373}]},"public_addr":"172.21.15.91:6825/18373","cluster_addr":"172.21.15.91:6827/18373","heartbeat_back_addr":"172.21.15.91:6831/18373","heartbeat_front_addr":"172.21.15.91:6829/18373","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.875052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.103604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.179225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.158512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.206073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.894840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.229708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.248192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-06T22:03:57.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.979+0000 7fc3bf59e700 1 -- 172.21.15.203:0/2632563378 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc3a40563d0 msgr2=0x7fc3a4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:57.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.979+0000 7fc3bf59e700 1 --2- 172.21.15.203:0/2632563378 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc3a40563d0 0x7fc3a4058890 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fc3b000bb20 tx=0x7fc3b0008040).stop 2021-01-06T22:03:57.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.979+0000 7fc3bf59e700 1 -- 172.21.15.203:0/2632563378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3c0064760 msgr2=0x7fc3c0161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:57.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.979+0000 7fc3bf59e700 1 --2- 172.21.15.203:0/2632563378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3c0064760 0x7fc3c0161060 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fc3c00bae20 tx=0x7fc3a8016000).stop 2021-01-06T22:03:57.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.979+0000 7fc3bf59e700 1 -- 172.21.15.203:0/2632563378 shutdown_connections 2021-01-06T22:03:57.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.979+0000 7fc3bf59e700 1 --2- 172.21.15.203:0/2632563378 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc3a40563d0 0x7fc3a4058890 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:57.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.979+0000 7fc3bf59e700 1 --2- 172.21.15.203:0/2632563378 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3c014e310 0x7fc3c00b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:57.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.979+0000 7fc3bf59e700 1 --2- 172.21.15.203:0/2632563378 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3c0064380 0x7fc3c0160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:57.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.979+0000 7fc3bf59e700 1 --2- 172.21.15.203:0/2632563378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3c0064760 0x7fc3c0161060 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:57.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.979+0000 7fc3bf59e700 1 -- 172.21.15.203:0/2632563378 >> 172.21.15.203:0/2632563378 conn(0x7fc3c00667b0 msgr2=0x7fc3c00bb650 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:57.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.979+0000 7fc3bf59e700 1 -- 172.21.15.203:0/2632563378 shutdown_connections 2021-01-06T22:03:57.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:57.979+0000 7fc3bf59e700 1 -- 172.21.15.203:0/2632563378 wait complete. 2021-01-06T22:03:57.996 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-01-06T22:03:25.161465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-01-06T22:03:35.070364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-01-06T22:03:37.862354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-01-06T22:03:38.738416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-01-06T22:03:57.997 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-01-06T22:03:58.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fc493d700 1 Processor -- start 2021-01-06T22:03:58.128 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fc493d700 1 -- start start 2021-01-06T22:03:58.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fc493d700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5fc0066c70 0x7f5fc00bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fc493d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5fc0067050 0x7f5fc0060a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fc493d700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc014eb90 con 0x7f5fc0060f00 2021-01-06T22:03:58.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fc493d700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc00c08b0 con 0x7f5fc0066c70 2021-01-06T22:03:58.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fc493d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc014dc70 con 0x7f5fc0067050 2021-01-06T22:03:58.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fbffff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f5fc0060f00 0x7f5fc0147720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47766/0 (socket says 172.21.15.203:47766) 2021-01-06T22:03:58.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fbffff700 1 -- 172.21.15.203:0/1102391856 learned_addr learned my addr 172.21.15.203:0/1102391856 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:58.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fbf7fe700 1 --2- 172.21.15.203:0/1102391856 >> v2:172.21.15.91:3300/0 conn(0x7f5fc0066c70 0x7f5fc00bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fbeffd700 1 --2- 172.21.15.203:0/1102391856 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5fc0067050 0x7f5fc0060a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fbe7fc700 1 -- 172.21.15.203:0/1102391856 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943145579 0 0) 0x7f5fc014eb90 con 0x7f5fc0060f00 2021-01-06T22:03:58.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fbe7fc700 1 -- 172.21.15.203:0/1102391856 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fac003230 con 0x7f5fc0060f00 2021-01-06T22:03:58.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fbe7fc700 1 -- 172.21.15.203:0/1102391856 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3953702760 0 0) 0x7f5fac003230 con 0x7f5fc0060f00 2021-01-06T22:03:58.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fbf7fe700 1 -- 172.21.15.203:0/1102391856 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5fc0067050 msgr2=0x7f5fc0060a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:58.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fbf7fe700 1 --2- 172.21.15.203:0/1102391856 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5fc0067050 0x7f5fc0060a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fbf7fe700 1 -- 172.21.15.203:0/1102391856 >> v1:172.21.15.203:6789/0 conn(0x7f5fc0060f00 legacy=0x7f5fc0147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.127+0000 7f5fbf7fe700 1 -- 172.21.15.203:0/1102391856 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc014df10 con 0x7f5fc0066c70 2021-01-06T22:03:58.136 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fbf7fe700 1 --2- 172.21.15.203:0/1102391856 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5fc0066c70 0x7f5fc00bffb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f5fb0009c90 tx=0x7f5fb00091d0).ready entity=mon.0 client_cookie=ba406305c94b7df9 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:58.136 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fbe7fc700 1 -- 172.21.15.203:0/1102391856 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb80014e0 con 0x7f5fc0066c70 2021-01-06T22:03:58.136 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fbe7fc700 1 -- 172.21.15.203:0/1102391856 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb000b740 con 0x7f5fc0066c70 2021-01-06T22:03:58.136 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fbe7fc700 1 -- 172.21.15.203:0/1102391856 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb000bac0 con 0x7f5fc0066c70 2021-01-06T22:03:58.136 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fc493d700 1 -- 172.21.15.203:0/1102391856 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5fc0066c70 msgr2=0x7f5fc00bffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.137 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fc493d700 1 --2- 172.21.15.203:0/1102391856 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5fc0066c70 0x7f5fc00bffb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f5fb0009c90 tx=0x7f5fb00091d0).stop 2021-01-06T22:03:58.137 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fc493d700 1 -- 172.21.15.203:0/1102391856 shutdown_connections 2021-01-06T22:03:58.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fc493d700 1 --2- 172.21.15.203:0/1102391856 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5fc0066c70 0x7f5fc00bffb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fc493d700 1 --2- 172.21.15.203:0/1102391856 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5fc0067050 0x7f5fc0060a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fc493d700 1 -- 172.21.15.203:0/1102391856 >> 172.21.15.203:0/1102391856 conn(0x7f5fc00626e0 msgr2=0x7f5fc0062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:58.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fc493d700 1 -- 172.21.15.203:0/1102391856 shutdown_connections 2021-01-06T22:03:58.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fc493d700 1 -- 172.21.15.203:0/1102391856 wait complete. 2021-01-06T22:03:58.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fc493d700 1 Processor -- start 2021-01-06T22:03:58.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fc493d700 1 -- start start 2021-01-06T22:03:58.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fc493d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5fc0060f00 0x7f5fc014c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fc493d700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5fc0066c70 0x7f5fc0060680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fc493d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fc0067050 0x7f5fc01454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fc493d700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc0152160 con 0x7f5fc0066c70 2021-01-06T22:03:58.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fc493d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc014c9c0 con 0x7f5fc0067050 2021-01-06T22:03:58.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fc493d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00b8c80 con 0x7f5fc0060f00 2021-01-06T22:03:58.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fbffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fc0067050 0x7f5fc01454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fbffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fc0067050 0x7f5fc01454d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49178/0 (socket says 172.21.15.203:49178) 2021-01-06T22:03:58.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fbffff700 1 -- 172.21.15.203:0/57660007 learned_addr learned my addr 172.21.15.203:0/57660007 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:58.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fbf7fe700 1 --2- 172.21.15.203:0/57660007 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5fc0060f00 0x7f5fc014c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.131+0000 7f5fbeffd700 1 --2- 172.21.15.203:0/57660007 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5fc0066c70 0x7f5fc0060680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.135+0000 7f5fbffff700 1 -- 172.21.15.203:0/57660007 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5fc0060f00 msgr2=0x7f5fc014c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.135+0000 7f5fbffff700 1 --2- 172.21.15.203:0/57660007 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5fc0060f00 0x7f5fc014c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.135+0000 7f5fbffff700 1 -- 172.21.15.203:0/57660007 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5fc0066c70 msgr2=0x7f5fc0060680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.135+0000 7f5fbffff700 1 --2- 172.21.15.203:0/57660007 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5fc0066c70 0x7f5fc0060680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.135+0000 7f5fbffff700 1 -- 172.21.15.203:0/57660007 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc014f070 con 0x7f5fc0067050 2021-01-06T22:03:58.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.135+0000 7f5fbeffd700 1 --2- 172.21.15.203:0/57660007 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5fc0066c70 0x7f5fc0060680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:58.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.135+0000 7f5fbffff700 1 --2- 172.21.15.203:0/57660007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fc0067050 0x7f5fc01454d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f5fb800c890 tx=0x7f5fb8001280).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:58.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.135+0000 7f5fa77fe700 1 -- 172.21.15.203:0/57660007 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8007170 con 0x7f5fc0067050 2021-01-06T22:03:58.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.135+0000 7f5fc493d700 1 -- 172.21.15.203:0/57660007 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc0153810 con 0x7f5fc0067050 2021-01-06T22:03:58.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.135+0000 7f5fa77fe700 1 -- 172.21.15.203:0/57660007 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb80072d0 con 0x7f5fc0067050 2021-01-06T22:03:58.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.135+0000 7f5fa77fe700 1 -- 172.21.15.203:0/57660007 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8004930 con 0x7f5fc0067050 2021-01-06T22:03:58.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.135+0000 7f5fc493d700 1 -- 172.21.15.203:0/57660007 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc014f570 con 0x7f5fc0067050 2021-01-06T22:03:58.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.135+0000 7f5fc493d700 1 -- 172.21.15.203:0/57660007 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc014f3b0 con 0x7f5fc0067050 2021-01-06T22:03:58.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.139+0000 7f5fa77fe700 1 -- 172.21.15.203:0/57660007 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f5fb8010070 con 0x7f5fc0067050 2021-01-06T22:03:58.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.139+0000 7f5fa77fe700 1 --2- 172.21.15.203:0/57660007 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5fa0056400 0x7f5fa00588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.139+0000 7f5fa77fe700 1 -- 172.21.15.203:0/57660007 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5fb806eea0 con 0x7f5fc0067050 2021-01-06T22:03:58.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.139+0000 7f5fbf7fe700 1 --2- 172.21.15.203:0/57660007 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5fa0056400 0x7f5fa00588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.139+0000 7f5fbf7fe700 1 --2- 172.21.15.203:0/57660007 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5fa0056400 0x7f5fa00588c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f5fb000b650 tx=0x7f5fb0017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:58.148 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.147+0000 7f5fa77fe700 1 -- 172.21.15.203:0/57660007 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f5fb8044720 con 0x7f5fc0067050 2021-01-06T22:03:58.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.303+0000 7f5fc493d700 1 -- 172.21.15.203:0/57660007 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f5fc00450a0 con 0x7f5fc0067050 2021-01-06T22:03:58.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.303+0000 7f5fa77fe700 1 -- 172.21.15.203:0/57660007 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5fb8047d40 con 0x7f5fc0067050 2021-01-06T22:03:58.306 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 1 2021-01-06T22:03:58.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.307+0000 7f5fc493d700 1 -- 172.21.15.203:0/57660007 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5fa0056400 msgr2=0x7f5fa00588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.307+0000 7f5fc493d700 1 --2- 172.21.15.203:0/57660007 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5fa0056400 0x7f5fa00588c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f5fb000b650 tx=0x7f5fb0017000).stop 2021-01-06T22:03:58.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.307+0000 7f5fc493d700 1 -- 172.21.15.203:0/57660007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fc0067050 msgr2=0x7f5fc01454d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.307+0000 7f5fc493d700 1 --2- 172.21.15.203:0/57660007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fc0067050 0x7f5fc01454d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f5fb800c890 tx=0x7f5fb8001280).stop 2021-01-06T22:03:58.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.307+0000 7f5fc493d700 1 -- 172.21.15.203:0/57660007 shutdown_connections 2021-01-06T22:03:58.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.307+0000 7f5fc493d700 1 --2- 172.21.15.203:0/57660007 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5fa0056400 0x7f5fa00588c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.307+0000 7f5fc493d700 1 --2- 172.21.15.203:0/57660007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fc0067050 0x7f5fc01454d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.307+0000 7f5fc493d700 1 --2- 172.21.15.203:0/57660007 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5fc0060f00 0x7f5fc014c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.307+0000 7f5fc493d700 1 --2- 172.21.15.203:0/57660007 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5fc0066c70 0x7f5fc0060680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.307+0000 7f5fc493d700 1 -- 172.21.15.203:0/57660007 >> 172.21.15.203:0/57660007 conn(0x7f5fc00626e0 msgr2=0x7f5fc0153b80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:58.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.307+0000 7f5fc493d700 1 -- 172.21.15.203:0/57660007 shutdown_connections 2021-01-06T22:03:58.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.307+0000 7f5fc493d700 1 -- 172.21.15.203:0/57660007 wait complete. 2021-01-06T22:03:58.321 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-01-06T22:03:58.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.451+0000 7f18fe85e700 1 Processor -- start 2021-01-06T22:03:58.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.451+0000 7f18fe85e700 1 -- start start 2021-01-06T22:03:58.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18fe85e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18f80ba300 0x7f18f80bc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18fe85e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f814ceb0 con 0x7f18f814ca50 2021-01-06T22:03:58.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18fe85e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f814fe30 con 0x7f18f8056d80 2021-01-06T22:03:58.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18fe85e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f814fc40 con 0x7f18f80ba300 2021-01-06T22:03:58.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f77fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f18f8056d80 0x7f18f814f530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47778/0 (socket says 172.21.15.203:47778) 2021-01-06T22:03:58.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f77fe700 1 -- 172.21.15.203:0/1854844001 learned_addr learned my addr 172.21.15.203:0/1854844001 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:58.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f7fff700 1 --2- 172.21.15.203:0/1854844001 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18f80ba300 0x7f18f80bc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 -- 172.21.15.203:0/1854844001 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007708364 0 0) 0x7f18f814fe30 con 0x7f18f8056d80 2021-01-06T22:03:58.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 -- 172.21.15.203:0/1854844001 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18e4003230 con 0x7f18f8056d80 2021-01-06T22:03:58.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 -- 172.21.15.203:0/1854844001 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3234435170 0 0) 0x7f18e4003230 con 0x7f18f8056d80 2021-01-06T22:03:58.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 -- 172.21.15.203:0/1854844001 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18f80ba300 msgr2=0x7f18f80bc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:58.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 --2- 172.21.15.203:0/1854844001 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18f80ba300 0x7f18f80bc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 -- 172.21.15.203:0/1854844001 >> v1:172.21.15.91:6789/0 conn(0x7f18f814ca50 legacy=0x7f18f8149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 -- 172.21.15.203:0/1854844001 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f81500d0 con 0x7f18f8056d80 2021-01-06T22:03:58.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 -- 172.21.15.203:0/1854844001 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f18ec0014e0 con 0x7f18f8056d80 2021-01-06T22:03:58.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 -- 172.21.15.203:0/1854844001 >> v1:172.21.15.203:6789/0 conn(0x7f18f8056d80 legacy=0x7f18f814f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 --2- 172.21.15.203:0/1854844001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e4003720 0x7f18e4003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 --2- 172.21.15.203:0/1854844001 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18f80ba300 0x7f18e40044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 --2- 172.21.15.203:0/1854844001 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18e40049c0 0x7f18e4004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.462 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 -- 172.21.15.203:0/1854844001 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e4004130 con 0x7f18e40049c0 2021-01-06T22:03:58.462 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 -- 172.21.15.203:0/1854844001 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e40042a0 con 0x7f18e4003720 2021-01-06T22:03:58.462 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 -- 172.21.15.203:0/1854844001 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e400b720 con 0x7f18f80ba300 2021-01-06T22:03:58.462 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f7fff700 1 --2- 172.21.15.203:0/1854844001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e4003720 0x7f18e4003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.462 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f77fe700 1 --2- 172.21.15.203:0/1854844001 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18f80ba300 0x7f18e40044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18fcdfb700 1 --2- 172.21.15.203:0/1854844001 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18e40049c0 0x7f18e4004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f7fff700 1 -- 172.21.15.203:0/1854844001 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18f80ba300 msgr2=0x7f18e40044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:58.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f7fff700 1 --2- 172.21.15.203:0/1854844001 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18f80ba300 0x7f18e40044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f7fff700 1 -- 172.21.15.203:0/1854844001 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18e40049c0 msgr2=0x7f18e4004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:58.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f7fff700 1 --2- 172.21.15.203:0/1854844001 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18e40049c0 0x7f18e4004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.464 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f7fff700 1 -- 172.21.15.203:0/1854844001 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e4003230 con 0x7f18e4003720 2021-01-06T22:03:58.464 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18fcdfb700 1 --2- 172.21.15.203:0/1854844001 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18e40049c0 0x7f18e4004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:03:58.464 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f7fff700 1 --2- 172.21.15.203:0/1854844001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e4003720 0x7f18e4003b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f18e800bb90 tx=0x7f18e8008040).ready entity=mon.1 client_cookie=c1d814c29b18df0c server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:58.464 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 -- 172.21.15.203:0/1854844001 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e800b2b0 con 0x7f18e4003720 2021-01-06T22:03:58.464 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 -- 172.21.15.203:0/1854844001 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e800ea30 con 0x7f18e4003720 2021-01-06T22:03:58.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18f6ffd700 1 -- 172.21.15.203:0/1854844001 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e800edd0 con 0x7f18e4003720 2021-01-06T22:03:58.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18fe85e700 1 -- 172.21.15.203:0/1854844001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e4003720 msgr2=0x7f18e4003b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18fe85e700 1 --2- 172.21.15.203:0/1854844001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e4003720 0x7f18e4003b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f18e800bb90 tx=0x7f18e8008040).stop 2021-01-06T22:03:58.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18fcdfb700 1 -- 172.21.15.203:0/1854844001 reap_dead start 2021-01-06T22:03:58.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18fe85e700 1 -- 172.21.15.203:0/1854844001 shutdown_connections 2021-01-06T22:03:58.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18fe85e700 1 -- 172.21.15.203:0/1854844001 >> 172.21.15.203:0/1854844001 conn(0x7f18f80c0cb0 msgr2=0x7f18f80c1090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:58.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18fe85e700 1 -- 172.21.15.203:0/1854844001 shutdown_connections 2021-01-06T22:03:58.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.455+0000 7f18fe85e700 1 -- 172.21.15.203:0/1854844001 wait complete. 2021-01-06T22:03:58.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.459+0000 7f18fe85e700 1 Processor -- start 2021-01-06T22:03:58.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.459+0000 7f18fe85e700 1 -- start start 2021-01-06T22:03:58.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.459+0000 7f18fe85e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18e4003720 0x7f18f80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.459+0000 7f18fe85e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18f8144db0 0x7f18f814ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.459+0000 7f18fe85e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18f814a750 0x7f18f814abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.459+0000 7f18fe85e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f80b9350 con 0x7f18e4003720 2021-01-06T22:03:58.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.459+0000 7f18fe85e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f814f450 con 0x7f18f814a750 2021-01-06T22:03:58.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.459+0000 7f18fe85e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f80b90b0 con 0x7f18f8144db0 2021-01-06T22:03:58.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.459+0000 7f18fcdfb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18f814a750 0x7f18f814abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.459+0000 7f18fcdfb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18f814a750 0x7f18f814abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49198/0 (socket says 172.21.15.203:49198) 2021-01-06T22:03:58.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.459+0000 7f18fcdfb700 1 -- 172.21.15.203:0/1472037753 learned_addr learned my addr 172.21.15.203:0/1472037753 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:58.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.459+0000 7f18f77fe700 1 --2- 172.21.15.203:0/1472037753 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18f8144db0 0x7f18f814ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.459+0000 7f18f7fff700 1 --2- 172.21.15.203:0/1472037753 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18e4003720 0x7f18f80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.463+0000 7f18fcdfb700 1 -- 172.21.15.203:0/1472037753 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18f8144db0 msgr2=0x7f18f814ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.463+0000 7f18fcdfb700 1 --2- 172.21.15.203:0/1472037753 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18f8144db0 0x7f18f814ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.463+0000 7f18fcdfb700 1 -- 172.21.15.203:0/1472037753 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18e4003720 msgr2=0x7f18f80b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.463+0000 7f18fcdfb700 1 --2- 172.21.15.203:0/1472037753 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18e4003720 0x7f18f80b8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.463+0000 7f18fcdfb700 1 -- 172.21.15.203:0/1472037753 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f8149ad0 con 0x7f18f814a750 2021-01-06T22:03:58.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.463+0000 7f18fcdfb700 1 --2- 172.21.15.203:0/1472037753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18f814a750 0x7f18f814abd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f18f80669f0 tx=0x7f18f0009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:58.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.463+0000 7f18dffff700 1 -- 172.21.15.203:0/1472037753 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f000b620 con 0x7f18f814a750 2021-01-06T22:03:58.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.463+0000 7f18dffff700 1 -- 172.21.15.203:0/1472037753 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f000b780 con 0x7f18f814a750 2021-01-06T22:03:58.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.463+0000 7f18fe85e700 1 -- 172.21.15.203:0/1472037753 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f8149d70 con 0x7f18f814a750 2021-01-06T22:03:58.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.463+0000 7f18dffff700 1 -- 172.21.15.203:0/1472037753 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f000bb20 con 0x7f18f814a750 2021-01-06T22:03:58.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.463+0000 7f18fe85e700 1 -- 172.21.15.203:0/1472037753 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f814a2b0 con 0x7f18f814a750 2021-01-06T22:03:58.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.463+0000 7f18fe85e700 1 -- 172.21.15.203:0/1472037753 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f814a470 con 0x7f18f814a750 2021-01-06T22:03:58.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.463+0000 7f18dffff700 1 -- 172.21.15.203:0/1472037753 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f18f001a030 con 0x7f18f814a750 2021-01-06T22:03:58.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.463+0000 7f18dffff700 1 --2- 172.21.15.203:0/1472037753 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18d8056400 0x7f18d80588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.463+0000 7f18dffff700 1 -- 172.21.15.203:0/1472037753 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f18f006f0a0 con 0x7f18f814a750 2021-01-06T22:03:58.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.463+0000 7f18f7fff700 1 --2- 172.21.15.203:0/1472037753 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18d8056400 0x7f18d80588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.467+0000 7f18f7fff700 1 --2- 172.21.15.203:0/1472037753 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18d8056400 0x7f18d80588c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f18e8006cf0 tx=0x7f18e801a000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:58.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.471+0000 7f18dffff700 1 -- 172.21.15.203:0/1472037753 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f18f0044920 con 0x7f18f814a750 2021-01-06T22:03:58.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.623+0000 7f18fe85e700 1 -- 172.21.15.203:0/1472037753 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f18f8144290 con 0x7f18f814a750 2021-01-06T22:03:58.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.623+0000 7f18dffff700 1 -- 172.21.15.203:0/1472037753 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f18f0047f40 con 0x7f18f814a750 2021-01-06T22:03:58.626 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:03:58.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.627+0000 7f18fe85e700 1 -- 172.21.15.203:0/1472037753 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18d8056400 msgr2=0x7f18d80588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.627+0000 7f18fe85e700 1 --2- 172.21.15.203:0/1472037753 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18d8056400 0x7f18d80588c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f18e8006cf0 tx=0x7f18e801a000).stop 2021-01-06T22:03:58.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.627+0000 7f18fe85e700 1 -- 172.21.15.203:0/1472037753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18f814a750 msgr2=0x7f18f814abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.627+0000 7f18fe85e700 1 --2- 172.21.15.203:0/1472037753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18f814a750 0x7f18f814abd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f18f80669f0 tx=0x7f18f0009040).stop 2021-01-06T22:03:58.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.627+0000 7f18fe85e700 1 -- 172.21.15.203:0/1472037753 shutdown_connections 2021-01-06T22:03:58.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.627+0000 7f18fe85e700 1 --2- 172.21.15.203:0/1472037753 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18d8056400 0x7f18d80588c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.627+0000 7f18fe85e700 1 --2- 172.21.15.203:0/1472037753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18f814a750 0x7f18f814abd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.627+0000 7f18fe85e700 1 --2- 172.21.15.203:0/1472037753 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18e4003720 0x7f18f80b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.627+0000 7f18fe85e700 1 --2- 172.21.15.203:0/1472037753 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18f8144db0 0x7f18f814ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.627+0000 7f18fe85e700 1 -- 172.21.15.203:0/1472037753 >> 172.21.15.203:0/1472037753 conn(0x7f18f80c0cb0 msgr2=0x7f18f80c1090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:58.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.627+0000 7f18fe85e700 1 -- 172.21.15.203:0/1472037753 shutdown_connections 2021-01-06T22:03:58.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.627+0000 7f18fe85e700 1 -- 172.21.15.203:0/1472037753 wait complete. 2021-01-06T22:03:58.647 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-01-06T22:03:58.774 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.771+0000 7f4b5508b700 1 Processor -- start 2021-01-06T22:03:58.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.771+0000 7f4b5508b700 1 -- start start 2021-01-06T22:03:58.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b5508b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b50066ad0 0x7f4b50060b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b5508b700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b50067a50 con 0x7f4b500baf40 2021-01-06T22:03:58.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b5508b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b500c02f0 con 0x7f4b500bab60 2021-01-06T22:03:58.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b5508b700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500c0590 con 0x7f4b50066ad0 2021-01-06T22:03:58.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4ffff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f4b500bab60 0x7f4b500b7460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47796/0 (socket says 172.21.15.203:47796) 2021-01-06T22:03:58.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4ffff700 1 -- 172.21.15.203:0/1012575757 learned_addr learned my addr 172.21.15.203:0/1012575757 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:58.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b5488a700 1 --2- 172.21.15.203:0/1012575757 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b50066ad0 0x7f4b50060b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 -- 172.21.15.203:0/1012575757 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044021701 0 0) 0x7f4b500c02f0 con 0x7f4b500bab60 2021-01-06T22:03:58.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 -- 172.21.15.203:0/1012575757 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b34003230 con 0x7f4b500bab60 2021-01-06T22:03:58.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 -- 172.21.15.203:0/1012575757 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2126381238 0 0) 0x7f4b34003230 con 0x7f4b500bab60 2021-01-06T22:03:58.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 -- 172.21.15.203:0/1012575757 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b50066ad0 msgr2=0x7f4b50060b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:58.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 --2- 172.21.15.203:0/1012575757 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b50066ad0 0x7f4b50060b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 -- 172.21.15.203:0/1012575757 >> v1:172.21.15.91:6789/0 conn(0x7f4b500baf40 legacy=0x7f4b500b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 -- 172.21.15.203:0/1012575757 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b500c0830 con 0x7f4b500bab60 2021-01-06T22:03:58.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 -- 172.21.15.203:0/1012575757 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f4b400018a0 con 0x7f4b500bab60 2021-01-06T22:03:58.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 -- 172.21.15.203:0/1012575757 >> v1:172.21.15.203:6789/0 conn(0x7f4b500bab60 legacy=0x7f4b500b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 --2- 172.21.15.203:0/1012575757 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b50066ad0 0x7f4b34003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 --2- 172.21.15.203:0/1012575757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b34003c30 0x7f4b340044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 --2- 172.21.15.203:0/1012575757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b340049c0 0x7f4b34004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 -- 172.21.15.203:0/1012575757 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500c0590 con 0x7f4b340049c0 2021-01-06T22:03:58.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 -- 172.21.15.203:0/1012575757 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b340041c0 con 0x7f4b34003c30 2021-01-06T22:03:58.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 -- 172.21.15.203:0/1012575757 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3400b720 con 0x7f4b50066ad0 2021-01-06T22:03:58.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4f7fe700 1 --2- 172.21.15.203:0/1012575757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b34003c30 0x7f4b340044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4ffff700 1 --2- 172.21.15.203:0/1012575757 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b50066ad0 0x7f4b34003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b5488a700 1 --2- 172.21.15.203:0/1012575757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b340049c0 0x7f4b34004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4f7fe700 1 -- 172.21.15.203:0/1012575757 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b50066ad0 msgr2=0x7f4b34003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:58.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4f7fe700 1 --2- 172.21.15.203:0/1012575757 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b50066ad0 0x7f4b34003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4f7fe700 1 -- 172.21.15.203:0/1012575757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b340049c0 msgr2=0x7f4b34004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:58.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4f7fe700 1 --2- 172.21.15.203:0/1012575757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b340049c0 0x7f4b34004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4f7fe700 1 -- 172.21.15.203:0/1012575757 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b34003230 con 0x7f4b34003c30 2021-01-06T22:03:58.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b5488a700 1 --2- 172.21.15.203:0/1012575757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b340049c0 0x7f4b34004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:03:58.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4f7fe700 1 --2- 172.21.15.203:0/1012575757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b34003c30 0x7f4b340044d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f4b38001920 tx=0x7f4b38007b80).ready entity=mon.1 client_cookie=cf6c71aff3895221 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:58.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 -- 172.21.15.203:0/1012575757 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b38007020 con 0x7f4b34003c30 2021-01-06T22:03:58.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 -- 172.21.15.203:0/1012575757 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b38007180 con 0x7f4b34003c30 2021-01-06T22:03:58.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b4effd700 1 -- 172.21.15.203:0/1012575757 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3800c7d0 con 0x7f4b34003c30 2021-01-06T22:03:58.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b5508b700 1 -- 172.21.15.203:0/1012575757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b34003c30 msgr2=0x7f4b340044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b5508b700 1 --2- 172.21.15.203:0/1012575757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b34003c30 0x7f4b340044d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f4b38001920 tx=0x7f4b38007b80).stop 2021-01-06T22:03:58.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b5488a700 1 -- 172.21.15.203:0/1012575757 reap_dead start 2021-01-06T22:03:58.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b5508b700 1 -- 172.21.15.203:0/1012575757 shutdown_connections 2021-01-06T22:03:58.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b5508b700 1 -- 172.21.15.203:0/1012575757 >> 172.21.15.203:0/1012575757 conn(0x7f4b500626e0 msgr2=0x7f4b50062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:58.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.775+0000 7f4b5508b700 1 -- 172.21.15.203:0/1012575757 shutdown_connections 2021-01-06T22:03:58.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.779+0000 7f4b5508b700 1 -- 172.21.15.203:0/1012575757 wait complete. 2021-01-06T22:03:58.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.779+0000 7f4b5508b700 1 Processor -- start 2021-01-06T22:03:58.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.779+0000 7f4b5508b700 1 -- start start 2021-01-06T22:03:58.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.779+0000 7f4b5508b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b34003c30 0x7f4b50165850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.779+0000 7f4b5508b700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b50152cf0 0x7f4b500b93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.779+0000 7f4b5508b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b50151a60 0x7f4b500b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.779+0000 7f4b5508b700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50152a40 con 0x7f4b50152cf0 2021-01-06T22:03:58.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.779+0000 7f4b5508b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50164f20 con 0x7f4b34003c30 2021-01-06T22:03:58.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.779+0000 7f4b5508b700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500b3360 con 0x7f4b50151a60 2021-01-06T22:03:58.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.779+0000 7f4b4ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b34003c30 0x7f4b50165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.779+0000 7f4b4ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b34003c30 0x7f4b50165850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49214/0 (socket says 172.21.15.203:49214) 2021-01-06T22:03:58.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.779+0000 7f4b4ffff700 1 -- 172.21.15.203:0/2918987235 learned_addr learned my addr 172.21.15.203:0/2918987235 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:58.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.779+0000 7f4b4f7fe700 1 --2- 172.21.15.203:0/2918987235 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b50152cf0 0x7f4b500b93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.779+0000 7f4b5488a700 1 --2- 172.21.15.203:0/2918987235 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b50151a60 0x7f4b500b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.783+0000 7f4b4ffff700 1 -- 172.21.15.203:0/2918987235 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b50151a60 msgr2=0x7f4b500b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.783+0000 7f4b4ffff700 1 --2- 172.21.15.203:0/2918987235 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b50151a60 0x7f4b500b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.783+0000 7f4b4ffff700 1 -- 172.21.15.203:0/2918987235 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b50152cf0 msgr2=0x7f4b500b93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.783+0000 7f4b4ffff700 1 --2- 172.21.15.203:0/2918987235 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b50152cf0 0x7f4b500b93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.783+0000 7f4b4ffff700 1 -- 172.21.15.203:0/2918987235 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b500b3600 con 0x7f4b34003c30 2021-01-06T22:03:58.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.783+0000 7f4b4f7fe700 1 --2- 172.21.15.203:0/2918987235 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b50152cf0 0x7f4b500b93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:58.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.783+0000 7f4b4ffff700 1 --2- 172.21.15.203:0/2918987235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b34003c30 0x7f4b50165850 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f4b400021e0 tx=0x7f4b500bc6d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:58.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.783+0000 7f4b47fff700 1 -- 172.21.15.203:0/2918987235 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4000fb80 con 0x7f4b34003c30 2021-01-06T22:03:58.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.783+0000 7f4b47fff700 1 -- 172.21.15.203:0/2918987235 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4000fce0 con 0x7f4b34003c30 2021-01-06T22:03:58.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.783+0000 7f4b5508b700 1 -- 172.21.15.203:0/2918987235 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b500b38a0 con 0x7f4b34003c30 2021-01-06T22:03:58.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.783+0000 7f4b47fff700 1 -- 172.21.15.203:0/2918987235 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b400125a0 con 0x7f4b34003c30 2021-01-06T22:03:58.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.783+0000 7f4b5508b700 1 -- 172.21.15.203:0/2918987235 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b501653b0 con 0x7f4b34003c30 2021-01-06T22:03:58.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.783+0000 7f4b5508b700 1 -- 172.21.15.203:0/2918987235 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b500b8280 con 0x7f4b34003c30 2021-01-06T22:03:58.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.783+0000 7f4b47fff700 1 -- 172.21.15.203:0/2918987235 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f4b4001b030 con 0x7f4b34003c30 2021-01-06T22:03:58.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.787+0000 7f4b47fff700 1 --2- 172.21.15.203:0/2918987235 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4b280563d0 0x7f4b28058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:58.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.787+0000 7f4b47fff700 1 -- 172.21.15.203:0/2918987235 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4b4006ef40 con 0x7f4b34003c30 2021-01-06T22:03:58.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.787+0000 7f4b4f7fe700 1 --2- 172.21.15.203:0/2918987235 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4b280563d0 0x7f4b28058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:58.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.787+0000 7f4b4f7fe700 1 --2- 172.21.15.203:0/2918987235 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4b280563d0 0x7f4b28058890 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f4b38001de0 tx=0x7f4b38015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:58.796 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.791+0000 7f4b47fff700 1 -- 172.21.15.203:0/2918987235 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f4b500b8280 con 0x7f4b34003c30 2021-01-06T22:03:58.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.947+0000 7f4b5508b700 1 -- 172.21.15.203:0/2918987235 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4b500b8280 con 0x7f4b34003c30 2021-01-06T22:03:58.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.947+0000 7f4b47fff700 1 -- 172.21.15.203:0/2918987235 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4b500b8280 con 0x7f4b34003c30 2021-01-06T22:03:58.949 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:03:58.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.951+0000 7f4b5508b700 1 -- 172.21.15.203:0/2918987235 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4b280563d0 msgr2=0x7f4b28058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.951+0000 7f4b5508b700 1 --2- 172.21.15.203:0/2918987235 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4b280563d0 0x7f4b28058890 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f4b38001de0 tx=0x7f4b38015000).stop 2021-01-06T22:03:58.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.951+0000 7f4b5508b700 1 -- 172.21.15.203:0/2918987235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b34003c30 msgr2=0x7f4b50165850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:58.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.951+0000 7f4b5508b700 1 --2- 172.21.15.203:0/2918987235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b34003c30 0x7f4b50165850 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f4b400021e0 tx=0x7f4b500bc6d0).stop 2021-01-06T22:03:58.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.951+0000 7f4b5508b700 1 -- 172.21.15.203:0/2918987235 shutdown_connections 2021-01-06T22:03:58.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.951+0000 7f4b5508b700 1 --2- 172.21.15.203:0/2918987235 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4b280563d0 0x7f4b28058890 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.951+0000 7f4b5508b700 1 --2- 172.21.15.203:0/2918987235 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b50151a60 0x7f4b500b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.951+0000 7f4b5508b700 1 --2- 172.21.15.203:0/2918987235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b34003c30 0x7f4b50165850 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.951+0000 7f4b5508b700 1 --2- 172.21.15.203:0/2918987235 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b50152cf0 0x7f4b500b93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:58.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.951+0000 7f4b5508b700 1 -- 172.21.15.203:0/2918987235 >> 172.21.15.203:0/2918987235 conn(0x7f4b500626e0 msgr2=0x7f4b500b7390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:58.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.951+0000 7f4b5508b700 1 -- 172.21.15.203:0/2918987235 shutdown_connections 2021-01-06T22:03:58.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:58.951+0000 7f4b5508b700 1 -- 172.21.15.203:0/2918987235 wait complete. 2021-01-06T22:03:58.964 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-01-06T22:03:59.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.083+0000 7f62ba8b0700 1 Processor -- start 2021-01-06T22:03:59.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.083+0000 7f62ba8b0700 1 -- start start 2021-01-06T22:03:59.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62ba8b0700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f62b4147d60 0x7f62b4148140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:59.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62ba8b0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62b40c1430 0x7f62b40bb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:59.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62ba8b0700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b40bb7e0 con 0x7f62b414b470 2021-01-06T22:03:59.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62ba8b0700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b4061b50 con 0x7f62b4147d60 2021-01-06T22:03:59.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62ba8b0700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b40bba80 con 0x7f62b40c1430 2021-01-06T22:03:59.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62ba0af700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f62b414b470 0x7f62b40671a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47820/0 (socket says 172.21.15.203:47820) 2021-01-06T22:03:59.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62ba0af700 1 -- 172.21.15.203:0/1089709413 learned_addr learned my addr 172.21.15.203:0/1089709413 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:59.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b98ae700 1 --2- 172.21.15.203:0/1089709413 >> v2:172.21.15.91:3300/0 conn(0x7f62b4147d60 0x7f62b4148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:59.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b90ad700 1 --2- 172.21.15.203:0/1089709413 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62b40c1430 0x7f62b40bb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:59.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 -- 172.21.15.203:0/1089709413 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106314050 0 0) 0x7f62b40bb7e0 con 0x7f62b414b470 2021-01-06T22:03:59.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 -- 172.21.15.203:0/1089709413 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f629c003230 con 0x7f62b414b470 2021-01-06T22:03:59.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 -- 172.21.15.203:0/1089709413 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1815286716 0 0) 0x7f629c003230 con 0x7f62b414b470 2021-01-06T22:03:59.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 -- 172.21.15.203:0/1089709413 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62b40c1430 msgr2=0x7f62b40bb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:59.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 --2- 172.21.15.203:0/1089709413 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62b40c1430 0x7f62b40bb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 -- 172.21.15.203:0/1089709413 >> v2:172.21.15.91:3300/0 conn(0x7f62b4147d60 msgr2=0x7f62b4148140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:59.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 --2- 172.21.15.203:0/1089709413 >> v2:172.21.15.91:3300/0 conn(0x7f62b4147d60 0x7f62b4148140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 -- 172.21.15.203:0/1089709413 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b40bbd20 con 0x7f62b414b470 2021-01-06T22:03:59.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 -- 172.21.15.203:0/1089709413 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f62ac001860 con 0x7f62b414b470 2021-01-06T22:03:59.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b98ae700 1 --2- 172.21.15.203:0/1089709413 >> v2:172.21.15.91:3300/0 conn(0x7f62b4147d60 0x7f62b4148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:59.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 -- 172.21.15.203:0/1089709413 >> v1:172.21.15.203:6789/0 conn(0x7f62b414b470 legacy=0x7f62b40671a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:59.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 --2- 172.21.15.203:0/1089709413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f629c003720 0x7f629c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:59.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 --2- 172.21.15.203:0/1089709413 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62b40c1430 0x7f629c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:59.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 --2- 172.21.15.203:0/1089709413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f629c0049c0 0x7f629c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:59.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 -- 172.21.15.203:0/1089709413 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b4061b50 con 0x7f629c003720 2021-01-06T22:03:59.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 -- 172.21.15.203:0/1089709413 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b40bba80 con 0x7f629c0049c0 2021-01-06T22:03:59.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 -- 172.21.15.203:0/1089709413 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629c0041c0 con 0x7f62b40c1430 2021-01-06T22:03:59.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62ba0af700 1 --2- 172.21.15.203:0/1089709413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f629c0049c0 0x7f629c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:59.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b98ae700 1 --2- 172.21.15.203:0/1089709413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f629c003720 0x7f629c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:59.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b90ad700 1 --2- 172.21.15.203:0/1089709413 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62b40c1430 0x7f629c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:59.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b98ae700 1 -- 172.21.15.203:0/1089709413 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62b40c1430 msgr2=0x7f629c0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:59.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b98ae700 1 --2- 172.21.15.203:0/1089709413 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62b40c1430 0x7f629c0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b98ae700 1 -- 172.21.15.203:0/1089709413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f629c0049c0 msgr2=0x7f629c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:59.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b98ae700 1 --2- 172.21.15.203:0/1089709413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f629c0049c0 0x7f629c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b98ae700 1 -- 172.21.15.203:0/1089709413 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c003230 con 0x7f629c003720 2021-01-06T22:03:59.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62ba0af700 1 --2- 172.21.15.203:0/1089709413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f629c0049c0 0x7f629c004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:03:59.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b98ae700 1 --2- 172.21.15.203:0/1089709413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f629c003720 0x7f629c003b40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f62b000be10 tx=0x7f62b0008040).ready entity=mon.0 client_cookie=9905157ad5788f0f server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:59.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 -- 172.21.15.203:0/1089709413 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b000b4f0 con 0x7f629c003720 2021-01-06T22:03:59.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 -- 172.21.15.203:0/1089709413 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b000ec40 con 0x7f629c003720 2021-01-06T22:03:59.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.087+0000 7f62b88ac700 1 -- 172.21.15.203:0/1089709413 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b00123f0 con 0x7f629c003720 2021-01-06T22:03:59.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba8b0700 1 -- 172.21.15.203:0/1089709413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f629c003720 msgr2=0x7f629c003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:59.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba8b0700 1 --2- 172.21.15.203:0/1089709413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f629c003720 0x7f629c003b40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f62b000be10 tx=0x7f62b0008040).stop 2021-01-06T22:03:59.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba0af700 1 -- 172.21.15.203:0/1089709413 reap_dead start 2021-01-06T22:03:59.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba8b0700 1 -- 172.21.15.203:0/1089709413 shutdown_connections 2021-01-06T22:03:59.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba8b0700 1 -- 172.21.15.203:0/1089709413 >> 172.21.15.203:0/1089709413 conn(0x7f62b40667b0 msgr2=0x7f62b40c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:59.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba8b0700 1 -- 172.21.15.203:0/1089709413 shutdown_connections 2021-01-06T22:03:59.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba8b0700 1 -- 172.21.15.203:0/1089709413 wait complete. 2021-01-06T22:03:59.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba8b0700 1 Processor -- start 2021-01-06T22:03:59.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba8b0700 1 -- start start 2021-01-06T22:03:59.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba8b0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b4149da0 0x7f62b414e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:59.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba8b0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62b40c01c0 0x7f62b415e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:59.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba8b0700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62b414a180 0x7f62b414bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:59.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba8b0700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b415e8e0 con 0x7f62b414a180 2021-01-06T22:03:59.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba8b0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b414e9e0 con 0x7f62b4149da0 2021-01-06T22:03:59.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba8b0700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b415ec90 con 0x7f62b40c01c0 2021-01-06T22:03:59.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62b98ae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b4149da0 0x7f62b414e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:59.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62b98ae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b4149da0 0x7f62b414e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49234/0 (socket says 172.21.15.203:49234) 2021-01-06T22:03:59.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62b98ae700 1 -- 172.21.15.203:0/1187069501 learned_addr learned my addr 172.21.15.203:0/1187069501 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:59.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62b90ad700 1 --2- 172.21.15.203:0/1187069501 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62b40c01c0 0x7f62b415e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:59.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba0af700 1 --2- 172.21.15.203:0/1187069501 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62b414a180 0x7f62b414bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:59.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62b98ae700 1 -- 172.21.15.203:0/1187069501 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62b40c01c0 msgr2=0x7f62b415e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:59.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62b98ae700 1 --2- 172.21.15.203:0/1187069501 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62b40c01c0 0x7f62b415e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62b98ae700 1 -- 172.21.15.203:0/1187069501 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62b414a180 msgr2=0x7f62b414bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:59.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62b98ae700 1 --2- 172.21.15.203:0/1187069501 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62b414a180 0x7f62b414bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62b98ae700 1 -- 172.21.15.203:0/1187069501 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b415ef30 con 0x7f62b4149da0 2021-01-06T22:03:59.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62b98ae700 1 --2- 172.21.15.203:0/1187069501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b4149da0 0x7f62b414e3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f62b000bfd0 tx=0x7f62b001a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:59.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62a1ffb700 1 -- 172.21.15.203:0/1187069501 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b0025440 con 0x7f62b4149da0 2021-01-06T22:03:59.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62a1ffb700 1 -- 172.21.15.203:0/1187069501 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b00255c0 con 0x7f62b4149da0 2021-01-06T22:03:59.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba8b0700 1 -- 172.21.15.203:0/1187069501 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b415f1d0 con 0x7f62b4149da0 2021-01-06T22:03:59.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62a1ffb700 1 -- 172.21.15.203:0/1187069501 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b0025980 con 0x7f62b4149da0 2021-01-06T22:03:59.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.091+0000 7f62ba8b0700 1 -- 172.21.15.203:0/1187069501 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b415f910 con 0x7f62b4149da0 2021-01-06T22:03:59.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.095+0000 7f62ba8b0700 1 -- 172.21.15.203:0/1187069501 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b40450a0 con 0x7f62b4149da0 2021-01-06T22:03:59.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.095+0000 7f62a1ffb700 1 -- 172.21.15.203:0/1187069501 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f62b0025d70 con 0x7f62b4149da0 2021-01-06T22:03:59.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.095+0000 7f62a1ffb700 1 --2- 172.21.15.203:0/1187069501 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f62940563d0 0x7f6294058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:59.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.095+0000 7f62a1ffb700 1 -- 172.21.15.203:0/1187069501 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f62b0071170 con 0x7f62b4149da0 2021-01-06T22:03:59.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.095+0000 7f62b90ad700 1 --2- 172.21.15.203:0/1187069501 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f62940563d0 0x7f6294058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:59.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.095+0000 7f62b90ad700 1 --2- 172.21.15.203:0/1187069501 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f62940563d0 0x7f6294058890 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f62b4151520 tx=0x7f62a4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:59.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.099+0000 7f62a1ffb700 1 -- 172.21.15.203:0/1187069501 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f62b00469f0 con 0x7f62b4149da0 2021-01-06T22:03:59.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.255+0000 7f62ba8b0700 1 -- 172.21.15.203:0/1187069501 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f62b405ca60 con 0x7f62b4149da0 2021-01-06T22:03:59.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.255+0000 7f62a1ffb700 1 -- 172.21.15.203:0/1187069501 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f62b00160a0 con 0x7f62b4149da0 2021-01-06T22:03:59.257 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:03:59.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.255+0000 7f62ba8b0700 1 -- 172.21.15.203:0/1187069501 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f62940563d0 msgr2=0x7f6294058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:59.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.255+0000 7f62ba8b0700 1 --2- 172.21.15.203:0/1187069501 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f62940563d0 0x7f6294058890 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f62b4151520 tx=0x7f62a4008040).stop 2021-01-06T22:03:59.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.255+0000 7f62ba8b0700 1 -- 172.21.15.203:0/1187069501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b4149da0 msgr2=0x7f62b414e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:59.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.255+0000 7f62ba8b0700 1 --2- 172.21.15.203:0/1187069501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b4149da0 0x7f62b414e3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f62b000bfd0 tx=0x7f62b001a000).stop 2021-01-06T22:03:59.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.259+0000 7f62ba8b0700 1 -- 172.21.15.203:0/1187069501 shutdown_connections 2021-01-06T22:03:59.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.259+0000 7f62ba8b0700 1 --2- 172.21.15.203:0/1187069501 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f62940563d0 0x7f6294058890 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.259+0000 7f62ba8b0700 1 --2- 172.21.15.203:0/1187069501 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62b414a180 0x7f62b414bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.259+0000 7f62ba8b0700 1 --2- 172.21.15.203:0/1187069501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b4149da0 0x7f62b414e3c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.259+0000 7f62ba8b0700 1 --2- 172.21.15.203:0/1187069501 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62b40c01c0 0x7f62b415e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.259+0000 7f62ba8b0700 1 -- 172.21.15.203:0/1187069501 >> 172.21.15.203:0/1187069501 conn(0x7f62b40667b0 msgr2=0x7f62b4151750 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:59.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.259+0000 7f62ba8b0700 1 -- 172.21.15.203:0/1187069501 shutdown_connections 2021-01-06T22:03:59.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.259+0000 7f62ba8b0700 1 -- 172.21.15.203:0/1187069501 wait complete. 2021-01-06T22:03:59.272 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-01-06T22:03:59.272 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:03:59.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.391+0000 7f0e81bad700 1 Processor -- start 2021-01-06T22:03:59.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.391+0000 7f0e81bad700 1 -- start start 2021-01-06T22:03:59.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e81bad700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0e7c148d70 0x7f0e7c0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:59.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e81bad700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0e7c149150 0x7f0e7c14bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:59.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e81bad700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7c14f0a0 con 0x7f0e7c0602b0 2021-01-06T22:03:59.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e81bad700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7c1495a0 con 0x7f0e7c148d70 2021-01-06T22:03:59.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e81bad700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7c14c3c0 con 0x7f0e7c149150 2021-01-06T22:03:59.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e7bfff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f0e7c0602b0 0x7f0e7c14fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47836/0 (socket says 172.21.15.203:47836) 2021-01-06T22:03:59.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e7bfff700 1 -- 172.21.15.203:0/442756585 learned_addr learned my addr 172.21.15.203:0/442756585 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:03:59.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e813ac700 1 --2- 172.21.15.203:0/442756585 >> v2:172.21.15.203:3300/0 conn(0x7f0e7c149150 0x7f0e7c14bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:59.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e80bab700 1 --2- 172.21.15.203:0/442756585 >> v2:172.21.15.91:3300/0 conn(0x7f0e7c148d70 0x7f0e7c0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:59.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e7b7fe700 1 -- 172.21.15.203:0/442756585 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 415040070 0 0) 0x7f0e7c14f0a0 con 0x7f0e7c0602b0 2021-01-06T22:03:59.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e7b7fe700 1 -- 172.21.15.203:0/442756585 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e68003230 con 0x7f0e7c0602b0 2021-01-06T22:03:59.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e813ac700 1 -- 172.21.15.203:0/442756585 >> v2:172.21.15.91:3300/0 conn(0x7f0e7c148d70 msgr2=0x7f0e7c0c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:03:59.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e813ac700 1 --2- 172.21.15.203:0/442756585 >> v2:172.21.15.91:3300/0 conn(0x7f0e7c148d70 0x7f0e7c0c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e813ac700 1 -- 172.21.15.203:0/442756585 >> v1:172.21.15.203:6789/0 conn(0x7f0e7c0602b0 legacy=0x7f0e7c14fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:59.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e813ac700 1 -- 172.21.15.203:0/442756585 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c14c660 con 0x7f0e7c149150 2021-01-06T22:03:59.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e813ac700 1 --2- 172.21.15.203:0/442756585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c149150 0x7f0e7c14bb30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f0e7c0c18e0 tx=0x7f0e740091d0).ready entity=mon.1 client_cookie=6b2fb41be4f61a49 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:59.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e7b7fe700 1 -- 172.21.15.203:0/442756585 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7400ba20 con 0x7f0e7c149150 2021-01-06T22:03:59.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e7b7fe700 1 -- 172.21.15.203:0/442756585 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7400bb80 con 0x7f0e7c149150 2021-01-06T22:03:59.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e7b7fe700 1 -- 172.21.15.203:0/442756585 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74011400 con 0x7f0e7c149150 2021-01-06T22:03:59.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e81bad700 1 -- 172.21.15.203:0/442756585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c149150 msgr2=0x7f0e7c14bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:59.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e81bad700 1 --2- 172.21.15.203:0/442756585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c149150 0x7f0e7c14bb30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f0e7c0c18e0 tx=0x7f0e740091d0).stop 2021-01-06T22:03:59.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e81bad700 1 -- 172.21.15.203:0/442756585 shutdown_connections 2021-01-06T22:03:59.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e81bad700 1 --2- 172.21.15.203:0/442756585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c149150 0x7f0e7c14bb30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e81bad700 1 --2- 172.21.15.203:0/442756585 >> v2:172.21.15.91:3300/0 conn(0x7f0e7c148d70 0x7f0e7c0c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e81bad700 1 -- 172.21.15.203:0/442756585 >> 172.21.15.203:0/442756585 conn(0x7f0e7c0667b0 msgr2=0x7f0e7c0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:59.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e81bad700 1 -- 172.21.15.203:0/442756585 shutdown_connections 2021-01-06T22:03:59.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.395+0000 7f0e81bad700 1 -- 172.21.15.203:0/442756585 wait complete. 2021-01-06T22:03:59.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.399+0000 7f0e81bad700 1 Processor -- start 2021-01-06T22:03:59.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.399+0000 7f0e81bad700 1 -- start start 2021-01-06T22:03:59.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e81bad700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e7c0602b0 0x7f0e7c14aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:59.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e81bad700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e7c148d70 0x7f0e7c15f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:59.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e81bad700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c149150 0x7f0e7c14e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:59.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e81bad700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c15f640 con 0x7f0e7c148d70 2021-01-06T22:03:59.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e81bad700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c1530d0 con 0x7f0e7c149150 2021-01-06T22:03:59.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e81bad700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c1528f0 con 0x7f0e7c0602b0 2021-01-06T22:03:59.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e813ac700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c149150 0x7f0e7c14e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:59.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e813ac700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c149150 0x7f0e7c14e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49252/0 (socket says 172.21.15.203:49252) 2021-01-06T22:03:59.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e813ac700 1 -- 172.21.15.203:0/2059730771 learned_addr learned my addr 172.21.15.203:0/2059730771 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:03:59.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e7bfff700 1 --2- 172.21.15.203:0/2059730771 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e7c148d70 0x7f0e7c15f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:59.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e80bab700 1 --2- 172.21.15.203:0/2059730771 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e7c0602b0 0x7f0e7c14aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:59.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e813ac700 1 -- 172.21.15.203:0/2059730771 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e7c0602b0 msgr2=0x7f0e7c14aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:59.408 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e813ac700 1 --2- 172.21.15.203:0/2059730771 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e7c0602b0 0x7f0e7c14aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.408 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e813ac700 1 -- 172.21.15.203:0/2059730771 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e7c148d70 msgr2=0x7f0e7c15f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:59.408 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e813ac700 1 --2- 172.21.15.203:0/2059730771 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e7c148d70 0x7f0e7c15f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.408 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e813ac700 1 -- 172.21.15.203:0/2059730771 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c152b90 con 0x7f0e7c149150 2021-01-06T22:03:59.408 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e813ac700 1 --2- 172.21.15.203:0/2059730771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c149150 0x7f0e7c14e2c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f0e74011f50 tx=0x7f0e74017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:59.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e78ff9700 1 -- 172.21.15.203:0/2059730771 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7401d640 con 0x7f0e7c149150 2021-01-06T22:03:59.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e78ff9700 1 -- 172.21.15.203:0/2059730771 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7401d7a0 con 0x7f0e7c149150 2021-01-06T22:03:59.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e81bad700 1 -- 172.21.15.203:0/2059730771 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7c152e30 con 0x7f0e7c149150 2021-01-06T22:03:59.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e78ff9700 1 -- 172.21.15.203:0/2059730771 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7401db20 con 0x7f0e7c149150 2021-01-06T22:03:59.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e81bad700 1 -- 172.21.15.203:0/2059730771 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7c149c20 con 0x7f0e7c149150 2021-01-06T22:03:59.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e81bad700 1 -- 172.21.15.203:0/2059730771 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7c149de0 con 0x7f0e7c149150 2021-01-06T22:03:59.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.403+0000 7f0e78ff9700 1 -- 172.21.15.203:0/2059730771 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f0e74025030 con 0x7f0e7c149150 2021-01-06T22:03:59.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.407+0000 7f0e78ff9700 1 --2- 172.21.15.203:0/2059730771 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0e5c0563d0 0x7f0e5c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:03:59.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.407+0000 7f0e78ff9700 1 -- 172.21.15.203:0/2059730771 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0e74070f50 con 0x7f0e7c149150 2021-01-06T22:03:59.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.407+0000 7f0e80bab700 1 --2- 172.21.15.203:0/2059730771 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0e5c0563d0 0x7f0e5c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:03:59.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.407+0000 7f0e80bab700 1 --2- 172.21.15.203:0/2059730771 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0e5c0563d0 0x7f0e5c058890 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f0e6c00bb50 tx=0x7f0e6c006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:03:59.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.411+0000 7f0e78ff9700 1 -- 172.21.15.203:0/2059730771 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f0e74046880 con 0x7f0e7c149150 2021-01-06T22:03:59.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.615+0000 7f0e81bad700 1 -- 172.21.15.203:0/2059730771 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e7c14b010 con 0x7f0e7c149150 2021-01-06T22:03:59.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.615+0000 7f0e78ff9700 1 -- 172.21.15.203:0/2059730771 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+3574 (secure 0 0 0) 0x7f0e74049ea0 con 0x7f0e7c149150 2021-01-06T22:03:59.618 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:03:59.618 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":5}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:03:44.792784+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1116012025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1116012025},{"type":"v1","addr":"172.21.15.203:6835","nonce":1116012025}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:03:59.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.619+0000 7f0e81bad700 1 -- 172.21.15.203:0/2059730771 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0e5c0563d0 msgr2=0x7f0e5c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:59.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.619+0000 7f0e81bad700 1 --2- 172.21.15.203:0/2059730771 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0e5c0563d0 0x7f0e5c058890 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f0e6c00bb50 tx=0x7f0e6c006de0).stop 2021-01-06T22:03:59.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.619+0000 7f0e81bad700 1 -- 172.21.15.203:0/2059730771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c149150 msgr2=0x7f0e7c14e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:03:59.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.619+0000 7f0e81bad700 1 --2- 172.21.15.203:0/2059730771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c149150 0x7f0e7c14e2c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f0e74011f50 tx=0x7f0e74017000).stop 2021-01-06T22:03:59.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.619+0000 7f0e81bad700 1 -- 172.21.15.203:0/2059730771 shutdown_connections 2021-01-06T22:03:59.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.619+0000 7f0e81bad700 1 --2- 172.21.15.203:0/2059730771 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0e5c0563d0 0x7f0e5c058890 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.619+0000 7f0e81bad700 1 --2- 172.21.15.203:0/2059730771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c149150 0x7f0e7c14e2c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.619+0000 7f0e81bad700 1 --2- 172.21.15.203:0/2059730771 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e7c0602b0 0x7f0e7c14aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.619+0000 7f0e81bad700 1 --2- 172.21.15.203:0/2059730771 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e7c148d70 0x7f0e7c15f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:03:59.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.619+0000 7f0e81bad700 1 -- 172.21.15.203:0/2059730771 >> 172.21.15.203:0/2059730771 conn(0x7f0e7c0667b0 msgr2=0x7f0e7c0c0870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:03:59.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.619+0000 7f0e81bad700 1 -- 172.21.15.203:0/2059730771 shutdown_connections 2021-01-06T22:03:59.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:03:59.619+0000 7f0e81bad700 1 -- 172.21.15.203:0/2059730771 wait complete. 2021-01-06T22:03:59.623 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 12 2021-01-06T22:04:00.235 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 12, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-01-06T22:03:39.741327+0000', 'modified': '2021-01-06T22:03:44.792784+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4274, 'mds_1': 4273, 'mds_2': 4270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4270': {'gid': 4270, 'name': 'c', 'rank': 2, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6835/1116012025', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 1116012025}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 1116012025}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4273': {'gid': 4273, 'name': 'a', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6837/3277561222', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 3277561222}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 3277561222}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4274': {'gid': 4274, 'name': 'f', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.91:6837/3888667247', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.91:6835', 'nonce': 3888667247}, {'type': 'v1', 'addr': '172.21.15.91:6837', 'nonce': 3888667247}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-01-06T22:04:00.235 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2021-01-06T22:04:00.236 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-01-06T22:04:00.378 INFO:teuthology.orchestra.run.smithi203.stdout:{"cluster_fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","whoami":2,"id":4270,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":16.577119467999999,"mdsmap_epoch":12,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":24.079035855000001} 2021-01-06T22:04:00.378 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "f96dbde2-6d40-40dc-b34a-9fb254ce805f", "fs_name": "cephfs", "id": 4270, "mdsmap_epoch": 12, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 16.577119468, "state": "up:active", "uptime": 24.079035855, "want_state": "up:active", "whoami": 2 } 2021-01-06T22:04:00.379 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-01-06T22:04:00.512 INFO:teuthology.orchestra.run.smithi203.stdout:{"cluster_fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","whoami":1,"id":4273,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":17.727541670000001,"mdsmap_epoch":12,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":24.211863985000001} 2021-01-06T22:04:00.513 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "f96dbde2-6d40-40dc-b34a-9fb254ce805f", "fs_name": "cephfs", "id": 4273, "mdsmap_epoch": 12, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 17.72754167, "state": "up:active", "uptime": 24.211863985, "want_state": "up:active", "whoami": 1 } 2021-01-06T22:04:00.513 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.f.asok --format=json status 2021-01-06T22:04:00.653 INFO:teuthology.orchestra.run.smithi091.stdout:{"cluster_fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","whoami":0,"id":4274,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":20.901359098,"mdsmap_epoch":12,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":24.329899696999998} 2021-01-06T22:04:00.654 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "f96dbde2-6d40-40dc-b34a-9fb254ce805f", "fs_name": "cephfs", "id": 4274, "mdsmap_epoch": 12, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 20.901359098, "state": "up:active", "uptime": 24.329899697, "want_state": "up:active", "whoami": 0 } 2021-01-06T22:04:00.654 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-01-06T22:04:00.655 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-01-06T22:04:00.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-01-06T22:04:00.655 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p -v /home/ubuntu/cephtest/mnt.0) 2021-01-06T22:04:00.663 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2021-01-06T22:04:00.664 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2021-01-06T22:04:00.720 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-01-06T22:04:00.720 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-01-06T22:04:00.777 INFO:teuthology.orchestra.run.smithi203.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-01-06T22:04:00.778 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-01-06T22:04:00.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-01-06T22:04:00.779 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2021-01-06T22:04:00.828 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-01-06T22:04:00.829 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --id 0 /home/ubuntu/cephtest/mnt.0) 2021-01-06T22:04:00.876 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2021-01-06T22:04:00.894 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-01-06T22:04:00.894 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-01-06T22:04:00.949 INFO:teuthology.orchestra.run.smithi203.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-01-06T22:04:00.950 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-01-06T22:04:00.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-01-06T22:04:00.951 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2021-01-06T22:04:00.966 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi203.stderr:2021-01-06T22:04:00.963+0000 7fa726846100 -1 init, newargv = 0x556100ef1a90 newargc=15 2021-01-06T22:04:00.966 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi203.stderr:ceph-fuse[23137]: starting ceph client 2021-01-06T22:04:01.180 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi203.stderr:ceph-fuse[23137]: starting fuse 2021-01-06T22:04:02.074 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2021-01-06T22:04:02.091 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-01-06T22:04:02.091 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-01-06T22:04:02.149 INFO:teuthology.orchestra.run.smithi203.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-01-06T22:04:02.150 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-01-06T22:04:02.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-01-06T22:04:02.151 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2021-01-06T22:04:02.204 INFO:teuthology.orchestra.run.smithi203.stdout:50 2021-01-06T22:04:02.205 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [50] 2021-01-06T22:04:02.205 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-01-06T22:04:02.205 DEBUG:teuthology.orchestra.run.smithi203:> import glob 2021-01-06T22:04:02.206 DEBUG:teuthology.orchestra.run.smithi203:> import re 2021-01-06T22:04:02.206 DEBUG:teuthology.orchestra.run.smithi203:> import os 2021-01-06T22:04:02.206 DEBUG:teuthology.orchestra.run.smithi203:> import subprocess 2021-01-06T22:04:02.207 DEBUG:teuthology.orchestra.run.smithi203:> 2021-01-06T22:04:02.208 DEBUG:teuthology.orchestra.run.smithi203:> def _find_admin_socket(client_name): 2021-01-06T22:04:02.208 DEBUG:teuthology.orchestra.run.smithi203:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-01-06T22:04:02.208 DEBUG:teuthology.orchestra.run.smithi203:> files = glob.glob(asok_path) 2021-01-06T22:04:02.209 DEBUG:teuthology.orchestra.run.smithi203:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2021-01-06T22:04:02.209 DEBUG:teuthology.orchestra.run.smithi203:> 2021-01-06T22:04:02.209 DEBUG:teuthology.orchestra.run.smithi203:> # Given a non-glob path, it better be there 2021-01-06T22:04:02.210 DEBUG:teuthology.orchestra.run.smithi203:> if "*" not in asok_path: 2021-01-06T22:04:02.210 DEBUG:teuthology.orchestra.run.smithi203:> assert(len(files) == 1) 2021-01-06T22:04:02.210 DEBUG:teuthology.orchestra.run.smithi203:> return files[0] 2021-01-06T22:04:02.211 DEBUG:teuthology.orchestra.run.smithi203:> 2021-01-06T22:04:02.211 DEBUG:teuthology.orchestra.run.smithi203:> for f in files: 2021-01-06T22:04:02.211 DEBUG:teuthology.orchestra.run.smithi203:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-01-06T22:04:02.212 DEBUG:teuthology.orchestra.run.smithi203:> if os.path.exists("/proc/{0}".format(pid)): 2021-01-06T22:04:02.212 DEBUG:teuthology.orchestra.run.smithi203:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2021-01-06T22:04:02.212 DEBUG:teuthology.orchestra.run.smithi203:> contents = proc_f.read() 2021-01-06T22:04:02.213 DEBUG:teuthology.orchestra.run.smithi203:> if mountpoint in contents: 2021-01-06T22:04:02.213 DEBUG:teuthology.orchestra.run.smithi203:> return f 2021-01-06T22:04:02.213 DEBUG:teuthology.orchestra.run.smithi203:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-01-06T22:04:02.214 DEBUG:teuthology.orchestra.run.smithi203:> 2021-01-06T22:04:02.214 DEBUG:teuthology.orchestra.run.smithi203:> print(_find_admin_socket("client.0")) 2021-01-06T22:04:02.214 DEBUG:teuthology.orchestra.run.smithi203:> ' 2021-01-06T22:04:02.337 INFO:teuthology.orchestra.run.smithi203.stdout:/var/run/ceph/ceph-client.0.23137.asok 2021-01-06T22:04:02.518 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23137.asok 2021-01-06T22:04:02.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-01-06T22:04:02.519 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23137.asok status 2021-01-06T22:04:02.674 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2021-01-06T22:04:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: "metadata": { 2021-01-06T22:04:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_sha1": "d0ed162b51928c50f20cee111f8292828eda755e", 2021-01-06T22:04:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 16.0.0-8719-gd0ed162b (d0ed162b51928c50f20cee111f8292828eda755e) pacific (dev)", 2021-01-06T22:04:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: "entity_id": "0", 2021-01-06T22:04:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: "hostname": "smithi203", 2021-01-06T22:04:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-01-06T22:04:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: "pid": "23137", 2021-01-06T22:04:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: "root": "/" 2021-01-06T22:04:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2021-01-06T22:04:02.677 INFO:teuthology.orchestra.run.smithi203.stdout: "dentry_count": 0, 2021-01-06T22:04:02.677 INFO:teuthology.orchestra.run.smithi203.stdout: "dentry_pinned_count": 0, 2021-01-06T22:04:02.677 INFO:teuthology.orchestra.run.smithi203.stdout: "id": 4663, 2021-01-06T22:04:02.677 INFO:teuthology.orchestra.run.smithi203.stdout: "inst": { 2021-01-06T22:04:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: "name": { 2021-01-06T22:04:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "client", 2021-01-06T22:04:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: "num": 4663 2021-01-06T22:04:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2021-01-06T22:04:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": { 2021-01-06T22:04:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "v1", 2021-01-06T22:04:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": "192.168.0.1:0", 2021-01-06T22:04:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: "nonce": 2497282476 2021-01-06T22:04:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: } 2021-01-06T22:04:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2021-01-06T22:04:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": { 2021-01-06T22:04:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "v1", 2021-01-06T22:04:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": "192.168.0.1:0", 2021-01-06T22:04:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: "nonce": 2497282476 2021-01-06T22:04:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2021-01-06T22:04:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: "inst_str": "client.4663 192.168.0.1:0/2497282476", 2021-01-06T22:04:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: "addr_str": "192.168.0.1:0/2497282476", 2021-01-06T22:04:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: "inode_count": 1, 2021-01-06T22:04:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_epoch": 12, 2021-01-06T22:04:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_epoch": 23, 2021-01-06T22:04:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_epoch_barrier": 0, 2021-01-06T22:04:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: "blocklisted": false, 2021-01-06T22:04:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: "fs_name": "cephfs" 2021-01-06T22:04:02.682 INFO:teuthology.orchestra.run.smithi203.stdout:} 2021-01-06T22:04:02.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-01-06T22:04:02.685 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-01-06T22:04:02.685 DEBUG:teuthology.orchestra.run.smithi203:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-01-06T22:04:02.692 INFO:teuthology.orchestra.run.smithi203.stdout:fuseblk 2021-01-06T22:04:02.694 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-01-06T22:04:02.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-01-06T22:04:02.695 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-01-06T22:04:02.764 INFO:teuthology.run_tasks:Running task mds_thrash... 2021-01-06T22:04:02.777 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-06T22:04:02.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f1891c02700 1 Processor -- start 2021-01-06T22:04:02.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f1891c02700 1 -- start start 2021-01-06T22:04:02.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f1891c02700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f188c0676d0 0x7f188c0bb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:02.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f1891c02700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f188c14e1c0 0x7f188c14e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:02.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f1891c02700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c061b00 con 0x7f188c067ab0 2021-01-06T22:04:02.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f1891c02700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f188c14eb60 con 0x7f188c14e1c0 2021-01-06T22:04:02.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f1891c02700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c061080 con 0x7f188c0676d0 2021-01-06T22:04:02.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f188bfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f188c14e1c0 0x7f188c14e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:02.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f188bfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f188c14e1c0 0x7f188c14e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49284/0 (socket says 172.21.15.203:49284) 2021-01-06T22:04:02.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f188bfff700 1 -- 172.21.15.203:0/1253535106 learned_addr learned my addr 172.21.15.203:0/1253535106 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:02.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f188b7fe700 1 --2- 172.21.15.203:0/1253535106 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f188c0676d0 0x7f188c0bb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:02.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f188bfff700 1 -- 172.21.15.203:0/1253535106 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f188c0676d0 msgr2=0x7f188c0bb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:02.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f188bfff700 1 --2- 172.21.15.203:0/1253535106 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f188c0676d0 0x7f188c0bb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:02.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f188bfff700 1 -- 172.21.15.203:0/1253535106 >> v1:172.21.15.91:6789/0 conn(0x7f188c067ab0 legacy=0x7f188c0682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:02.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f188bfff700 1 -- 172.21.15.203:0/1253535106 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c061320 con 0x7f188c14e1c0 2021-01-06T22:04:02.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f188b7fe700 1 --2- 172.21.15.203:0/1253535106 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f188c0676d0 0x7f188c0bb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:04:02.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f188bfff700 1 --2- 172.21.15.203:0/1253535106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f188c14e1c0 0x7f188c14e670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f188400ba10 tx=0x7f1884006ea0).ready entity=mon.1 client_cookie=601788ae51c0371b server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:02.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f188a7fc700 1 -- 172.21.15.203:0/1253535106 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884009890 con 0x7f188c14e1c0 2021-01-06T22:04:02.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f188a7fc700 1 -- 172.21.15.203:0/1253535106 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18840099f0 con 0x7f188c14e1c0 2021-01-06T22:04:02.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f188a7fc700 1 -- 172.21.15.203:0/1253535106 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884009d70 con 0x7f188c14e1c0 2021-01-06T22:04:02.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f1891c02700 1 -- 172.21.15.203:0/1253535106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f188c14e1c0 msgr2=0x7f188c14e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:02.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f1891c02700 1 --2- 172.21.15.203:0/1253535106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f188c14e1c0 0x7f188c14e670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f188400ba10 tx=0x7f1884006ea0).stop 2021-01-06T22:04:02.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f1891c02700 1 -- 172.21.15.203:0/1253535106 shutdown_connections 2021-01-06T22:04:02.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f1891c02700 1 --2- 172.21.15.203:0/1253535106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f188c14e1c0 0x7f188c14e670 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:02.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f1891c02700 1 --2- 172.21.15.203:0/1253535106 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f188c0676d0 0x7f188c0bb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:02.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f1891c02700 1 -- 172.21.15.203:0/1253535106 >> 172.21.15.203:0/1253535106 conn(0x7f188c0c0c50 msgr2=0x7f188c0c1030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:02.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f1891c02700 1 -- 172.21.15.203:0/1253535106 shutdown_connections 2021-01-06T22:04:02.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.935+0000 7f1891c02700 1 -- 172.21.15.203:0/1253535106 wait complete. 2021-01-06T22:04:02.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f1891c02700 1 Processor -- start 2021-01-06T22:04:02.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f1891c02700 1 -- start start 2021-01-06T22:04:02.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f1891c02700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f188c0676d0 0x7f188c14ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:02.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f1891c02700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f188c067ab0 0x7f188c15be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:02.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f1891c02700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f188c14e1c0 0x7f188c13b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:02.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f1891c02700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c146c40 con 0x7f188c14e1c0 2021-01-06T22:04:02.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f1891c02700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c146ee0 con 0x7f188c067ab0 2021-01-06T22:04:02.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f1891c02700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c147180 con 0x7f188c0676d0 2021-01-06T22:04:02.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f188affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f188c067ab0 0x7f188c15be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:02.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f188affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f188c067ab0 0x7f188c15be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49288/0 (socket says 172.21.15.203:49288) 2021-01-06T22:04:02.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f188affd700 1 -- 172.21.15.203:0/4072294308 learned_addr learned my addr 172.21.15.203:0/4072294308 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:02.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f188bfff700 1 --2- 172.21.15.203:0/4072294308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f188c14e1c0 0x7f188c13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:02.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f188b7fe700 1 --2- 172.21.15.203:0/4072294308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f188c0676d0 0x7f188c14ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:02.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f188affd700 1 -- 172.21.15.203:0/4072294308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f188c0676d0 msgr2=0x7f188c14ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:02.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f188affd700 1 --2- 172.21.15.203:0/4072294308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f188c0676d0 0x7f188c14ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:02.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f188affd700 1 -- 172.21.15.203:0/4072294308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f188c14e1c0 msgr2=0x7f188c13b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:02.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f188affd700 1 --2- 172.21.15.203:0/4072294308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f188c14e1c0 0x7f188c13b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:02.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f188affd700 1 -- 172.21.15.203:0/4072294308 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c15b970 con 0x7f188c067ab0 2021-01-06T22:04:02.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f188b7fe700 1 --2- 172.21.15.203:0/4072294308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f188c0676d0 0x7f188c14ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:04:02.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f188bfff700 1 --2- 172.21.15.203:0/4072294308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f188c14e1c0 0x7f188c13b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:04:02.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f188affd700 1 --2- 172.21.15.203:0/4072294308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f188c067ab0 0x7f188c15be10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f187c00ca60 tx=0x7f187c007f60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:02.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f18837fe700 1 -- 172.21.15.203:0/4072294308 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c00a880 con 0x7f188c067ab0 2021-01-06T22:04:02.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f18837fe700 1 -- 172.21.15.203:0/4072294308 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c00a9e0 con 0x7f188c067ab0 2021-01-06T22:04:02.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f1891c02700 1 -- 172.21.15.203:0/4072294308 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f188c15b660 con 0x7f188c067ab0 2021-01-06T22:04:02.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f18837fe700 1 -- 172.21.15.203:0/4072294308 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c00ad80 con 0x7f188c067ab0 2021-01-06T22:04:02.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.939+0000 7f1891c02700 1 -- 172.21.15.203:0/4072294308 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188c0be8f0 con 0x7f188c067ab0 2021-01-06T22:04:02.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.943+0000 7f1891c02700 1 -- 172.21.15.203:0/4072294308 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188c05d600 con 0x7f188c067ab0 2021-01-06T22:04:02.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.943+0000 7f18837fe700 1 -- 172.21.15.203:0/4072294308 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f187c010070 con 0x7f188c067ab0 2021-01-06T22:04:02.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.943+0000 7f18837fe700 1 --2- 172.21.15.203:0/4072294308 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18700563d0 0x7f1870058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:02.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.943+0000 7f18837fe700 1 -- 172.21.15.203:0/4072294308 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f187c06ee80 con 0x7f188c067ab0 2021-01-06T22:04:02.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.943+0000 7f188b7fe700 1 --2- 172.21.15.203:0/4072294308 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18700563d0 0x7f1870058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:02.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.943+0000 7f188b7fe700 1 --2- 172.21.15.203:0/4072294308 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18700563d0 0x7f1870058890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f187400ba10 tx=0x7f1874008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:02.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:02.947+0000 7f18837fe700 1 -- 172.21.15.203:0/4072294308 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f187c044700 con 0x7f188c067ab0 2021-01-06T22:04:03.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.103+0000 7f1891c02700 1 -- 172.21.15.203:0/4072294308 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f188c061cc0 con 0x7f188c067ab0 2021-01-06T22:04:03.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.107+0000 7f18837fe700 1 -- 172.21.15.203:0/4072294308 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f187c047d20 con 0x7f188c067ab0 2021-01-06T22:04:03.108 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:03.108 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":23,"fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","created":"2021-01-06T22:03:21.322671+0000","modified":"2021-01-06T22:03:40.744826+0000","last_up_change":"2021-01-06T22:03:31.115611+0000","last_in_change":"2021-01-06T22:03:31.115611+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-06T22:03:25.161465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-06T22:03:35.070364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-01-06T22:03:37.862354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-01-06T22:03:38.738416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e0b35c09-147c-4ccc-b566-c48b81e25412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":18347},{"type":"v1","addr":"172.21.15.203:6815","nonce":18347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":18347},{"type":"v1","addr":"172.21.15.203:6818","nonce":18347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":18347},{"type":"v1","addr":"172.21.15.203:6831","nonce":18347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6821","nonce":18347},{"type":"v1","addr":"172.21.15.203:6827","nonce":18347}]},"public_addr":"172.21.15.203:6815/18347","cluster_addr":"172.21.15.203:6818/18347","heartbeat_back_addr":"172.21.15.203:6831/18347","heartbeat_front_addr":"172.21.15.203:6827/18347","state":["exists","up"]},{"osd":1,"uuid":"8add59a0-2d58-4c1b-a8d1-62049ee9d85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":18337},{"type":"v1","addr":"172.21.15.203:6803","nonce":18337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":18337},{"type":"v1","addr":"172.21.15.203:6805","nonce":18337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":18337},{"type":"v1","addr":"172.21.15.203:6809","nonce":18337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":18337},{"type":"v1","addr":"172.21.15.203:6807","nonce":18337}]},"public_addr":"172.21.15.203:6803/18337","cluster_addr":"172.21.15.203:6805/18337","heartbeat_back_addr":"172.21.15.203:6809/18337","heartbeat_front_addr":"172.21.15.203:6807/18337","state":["exists","up"]},{"osd":2,"uuid":"b294a411-c7c9-477e-8eae-3b7bd0563ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":18354},{"type":"v1","addr":"172.21.15.203:6811","nonce":18354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":18354},{"type":"v1","addr":"172.21.15.203:6819","nonce":18354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6825","nonce":18354},{"type":"v1","addr":"172.21.15.203:6828","nonce":18354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6823","nonce":18354},{"type":"v1","addr":"172.21.15.203:6824","nonce":18354}]},"public_addr":"172.21.15.203:6811/18354","cluster_addr":"172.21.15.203:6819/18354","heartbeat_back_addr":"172.21.15.203:6828/18354","heartbeat_front_addr":"172.21.15.203:6824/18354","state":["exists","up"]},{"osd":3,"uuid":"5878a8f7-1a42-41c3-9810-0ac41a54dd6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":18346},{"type":"v1","addr":"172.21.15.203:6817","nonce":18346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":18346},{"type":"v1","addr":"172.21.15.203:6822","nonce":18346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":18346},{"type":"v1","addr":"172.21.15.203:6833","nonce":18346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":18346},{"type":"v1","addr":"172.21.15.203:6829","nonce":18346}]},"public_addr":"172.21.15.203:6817/18346","cluster_addr":"172.21.15.203:6822/18346","heartbeat_back_addr":"172.21.15.203:6833/18346","heartbeat_front_addr":"172.21.15.203:6829/18346","state":["exists","up"]},{"osd":4,"uuid":"f641a345-81e4-4400-a44c-d2f82fef7f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":18370},{"type":"v1","addr":"172.21.15.91:6809","nonce":18370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":18370},{"type":"v1","addr":"172.21.15.91:6811","nonce":18370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":18370},{"type":"v1","addr":"172.21.15.91:6816","nonce":18370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":18370},{"type":"v1","addr":"172.21.15.91:6813","nonce":18370}]},"public_addr":"172.21.15.91:6809/18370","cluster_addr":"172.21.15.91:6811/18370","heartbeat_back_addr":"172.21.15.91:6816/18370","heartbeat_front_addr":"172.21.15.91:6813/18370","state":["exists","up"]},{"osd":5,"uuid":"c1ff150d-68c6-4e6b-abd4-76af27bb2103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":18372},{"type":"v1","addr":"172.21.15.91:6801","nonce":18372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":18372},{"type":"v1","addr":"172.21.15.91:6803","nonce":18372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":18372},{"type":"v1","addr":"172.21.15.91:6807","nonce":18372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":18372},{"type":"v1","addr":"172.21.15.91:6805","nonce":18372}]},"public_addr":"172.21.15.91:6801/18372","cluster_addr":"172.21.15.91:6803/18372","heartbeat_back_addr":"172.21.15.91:6807/18372","heartbeat_front_addr":"172.21.15.91:6805/18372","state":["exists","up"]},{"osd":6,"uuid":"35a58e1a-0a0a-4f02-a477-adc029038d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6815","nonce":18371},{"type":"v1","addr":"172.21.15.91:6817","nonce":18371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":18371},{"type":"v1","addr":"172.21.15.91:6819","nonce":18371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":18371},{"type":"v1","addr":"172.21.15.91:6823","nonce":18371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":18371},{"type":"v1","addr":"172.21.15.91:6821","nonce":18371}]},"public_addr":"172.21.15.91:6817/18371","cluster_addr":"172.21.15.91:6819/18371","heartbeat_back_addr":"172.21.15.91:6823/18371","heartbeat_front_addr":"172.21.15.91:6821/18371","state":["exists","up"]},{"osd":7,"uuid":"809a2d89-4303-482a-8558-aef907814234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":18373},{"type":"v1","addr":"172.21.15.91:6825","nonce":18373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":18373},{"type":"v1","addr":"172.21.15.91:6827","nonce":18373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":18373},{"type":"v1","addr":"172.21.15.91:6831","nonce":18373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":18373},{"type":"v1","addr":"172.21.15.91:6829","nonce":18373}]},"public_addr":"172.21.15.91:6825/18373","cluster_addr":"172.21.15.91:6827/18373","heartbeat_back_addr":"172.21.15.91:6831/18373","heartbeat_front_addr":"172.21.15.91:6829/18373","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.875052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.103604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.179225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.158512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.206073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.894840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.229708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.248192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-06T22:04:03.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.107+0000 7f1891c02700 1 -- 172.21.15.203:0/4072294308 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18700563d0 msgr2=0x7f1870058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.107+0000 7f1891c02700 1 --2- 172.21.15.203:0/4072294308 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18700563d0 0x7f1870058890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f187400ba10 tx=0x7f1874008040).stop 2021-01-06T22:04:03.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.107+0000 7f1891c02700 1 -- 172.21.15.203:0/4072294308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f188c067ab0 msgr2=0x7f188c15be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.107+0000 7f1891c02700 1 --2- 172.21.15.203:0/4072294308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f188c067ab0 0x7f188c15be10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f187c00ca60 tx=0x7f187c007f60).stop 2021-01-06T22:04:03.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.107+0000 7f1891c02700 1 -- 172.21.15.203:0/4072294308 shutdown_connections 2021-01-06T22:04:03.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.107+0000 7f1891c02700 1 --2- 172.21.15.203:0/4072294308 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f18700563d0 0x7f1870058890 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.107+0000 7f1891c02700 1 --2- 172.21.15.203:0/4072294308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f188c14e1c0 0x7f188c13b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.107+0000 7f1891c02700 1 --2- 172.21.15.203:0/4072294308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f188c0676d0 0x7f188c14ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.107+0000 7f1891c02700 1 --2- 172.21.15.203:0/4072294308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f188c067ab0 0x7f188c15be10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.107+0000 7f1891c02700 1 -- 172.21.15.203:0/4072294308 >> 172.21.15.203:0/4072294308 conn(0x7f188c0c0c50 msgr2=0x7f188c0bbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:03.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.107+0000 7f1891c02700 1 -- 172.21.15.203:0/4072294308 shutdown_connections 2021-01-06T22:04:03.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.107+0000 7f1891c02700 1 -- 172.21.15.203:0/4072294308 wait complete. 2021-01-06T22:04:03.122 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-01-06T22:03:25.161465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-01-06T22:03:35.070364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-01-06T22:03:37.862354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-01-06T22:03:38.738416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-01-06T22:04:03.123 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-01-06T22:04:03.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.251+0000 7f495680f700 1 Processor -- start 2021-01-06T22:04:03.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.251+0000 7f495680f700 1 -- start start 2021-01-06T22:04:03.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.251+0000 7f495680f700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f49500c0fe0 0x7f49500aa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.251+0000 7f495680f700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49500a9ca0 con 0x7f49500c03f0 2021-01-06T22:04:03.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.251+0000 7f495680f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4950066a50 con 0x7f49500c13c0 2021-01-06T22:04:03.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.251+0000 7f495680f700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4950066cf0 con 0x7f49500c0fe0 2021-01-06T22:04:03.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.251+0000 7f495600e700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f49500c13c0 0x7f4950150110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47890/0 (socket says 172.21.15.203:47890) 2021-01-06T22:04:03.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.251+0000 7f495600e700 1 -- 172.21.15.203:0/4136773277 learned_addr learned my addr 172.21.15.203:0/4136773277 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:04:03.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.251+0000 7f495580d700 1 --2- 172.21.15.203:0/4136773277 >> v2:172.21.15.91:3300/0 conn(0x7f49500c0fe0 0x7f49500aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 -- 172.21.15.203:0/4136773277 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1064764753 0 0) 0x7f4950066a50 con 0x7f49500c13c0 2021-01-06T22:04:03.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 -- 172.21.15.203:0/4136773277 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4938003230 con 0x7f49500c13c0 2021-01-06T22:04:03.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 -- 172.21.15.203:0/4136773277 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 765967094 0 0) 0x7f4938003230 con 0x7f49500c13c0 2021-01-06T22:04:03.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 -- 172.21.15.203:0/4136773277 >> v2:172.21.15.91:3300/0 conn(0x7f49500c0fe0 msgr2=0x7f49500aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:03.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 --2- 172.21.15.203:0/4136773277 >> v2:172.21.15.91:3300/0 conn(0x7f49500c0fe0 0x7f49500aa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 -- 172.21.15.203:0/4136773277 >> v1:172.21.15.91:6789/0 conn(0x7f49500c03f0 legacy=0x7f4950060d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 -- 172.21.15.203:0/4136773277 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4950068460 con 0x7f49500c13c0 2021-01-06T22:04:03.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 -- 172.21.15.203:0/4136773277 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f4948001860 con 0x7f49500c13c0 2021-01-06T22:04:03.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 -- 172.21.15.203:0/4136773277 >> v1:172.21.15.203:6789/0 conn(0x7f49500c13c0 legacy=0x7f4950150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 --2- 172.21.15.203:0/4136773277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4938003720 0x7f4938003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 --2- 172.21.15.203:0/4136773277 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49380044d0 0x7f4938004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 --2- 172.21.15.203:0/4136773277 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4938004e40 0x7f493800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 -- 172.21.15.203:0/4136773277 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4950066cf0 con 0x7f4938004e40 2021-01-06T22:04:03.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 -- 172.21.15.203:0/4136773277 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4938004250 con 0x7f4938003720 2021-01-06T22:04:03.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 -- 172.21.15.203:0/4136773277 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493800bc10 con 0x7f49380044d0 2021-01-06T22:04:03.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495580d700 1 --2- 172.21.15.203:0/4136773277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4938003720 0x7f4938003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495600e700 1 --2- 172.21.15.203:0/4136773277 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4938004e40 0x7f493800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495500c700 1 --2- 172.21.15.203:0/4136773277 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49380044d0 0x7f4938004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495580d700 1 -- 172.21.15.203:0/4136773277 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49380044d0 msgr2=0x7f4938004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:03.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495580d700 1 --2- 172.21.15.203:0/4136773277 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49380044d0 0x7f4938004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495580d700 1 -- 172.21.15.203:0/4136773277 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4938004e40 msgr2=0x7f493800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:03.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495580d700 1 --2- 172.21.15.203:0/4136773277 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4938004e40 0x7f493800b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495580d700 1 -- 172.21.15.203:0/4136773277 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4938003230 con 0x7f4938003720 2021-01-06T22:04:03.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495600e700 1 -- 172.21.15.203:0/4136773277 reap_dead start 2021-01-06T22:04:03.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495580d700 1 --2- 172.21.15.203:0/4136773277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4938003720 0x7f4938003b40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f494c00bb90 tx=0x7f494c008040).ready entity=mon.1 client_cookie=a6f3b5997862c350 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:03.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 -- 172.21.15.203:0/4136773277 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c00b2b0 con 0x7f4938003720 2021-01-06T22:04:03.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 -- 172.21.15.203:0/4136773277 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494c00e960 con 0x7f4938003720 2021-01-06T22:04:03.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495480b700 1 -- 172.21.15.203:0/4136773277 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c00ed00 con 0x7f4938003720 2021-01-06T22:04:03.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495680f700 1 -- 172.21.15.203:0/4136773277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4938003720 msgr2=0x7f4938003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495680f700 1 --2- 172.21.15.203:0/4136773277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4938003720 0x7f4938003b40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f494c00bb90 tx=0x7f494c008040).stop 2021-01-06T22:04:03.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495680f700 1 -- 172.21.15.203:0/4136773277 shutdown_connections 2021-01-06T22:04:03.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495680f700 1 --2- 172.21.15.203:0/4136773277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4938003720 0x7f4938003b40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495680f700 1 -- 172.21.15.203:0/4136773277 >> 172.21.15.203:0/4136773277 conn(0x7f4950061530 msgr2=0x7f4950061910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:03.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495680f700 1 -- 172.21.15.203:0/4136773277 shutdown_connections 2021-01-06T22:04:03.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495680f700 1 -- 172.21.15.203:0/4136773277 wait complete. 2021-01-06T22:04:03.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.255+0000 7f495680f700 1 Processor -- start 2021-01-06T22:04:03.267 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495680f700 1 -- start start 2021-01-06T22:04:03.267 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495680f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4938003720 0x7f4950161770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.267 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495680f700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49500bccf0 0x7f49500be460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.267 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495680f700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49501502d0 0x7f49500b9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.268 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495680f700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4950160e00 con 0x7f49501502d0 2021-01-06T22:04:03.268 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495680f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4950150810 con 0x7f4938003720 2021-01-06T22:04:03.268 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495680f700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4950150ab0 con 0x7f49500bccf0 2021-01-06T22:04:03.268 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495580d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4938003720 0x7f4950161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.268 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495580d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4938003720 0x7f4950161770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49306/0 (socket says 172.21.15.203:49306) 2021-01-06T22:04:03.269 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495600e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49501502d0 0x7f49500b9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.269 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495580d700 1 -- 172.21.15.203:0/1558435939 learned_addr learned my addr 172.21.15.203:0/1558435939 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:03.269 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495500c700 1 --2- 172.21.15.203:0/1558435939 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49500bccf0 0x7f49500be460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.269 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495580d700 1 -- 172.21.15.203:0/1558435939 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49500bccf0 msgr2=0x7f49500be460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.269 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495580d700 1 --2- 172.21.15.203:0/1558435939 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49500bccf0 0x7f49500be460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495580d700 1 -- 172.21.15.203:0/1558435939 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49501502d0 msgr2=0x7f49500b9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495580d700 1 --2- 172.21.15.203:0/1558435939 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49501502d0 0x7f49500b9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495580d700 1 -- 172.21.15.203:0/1558435939 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4950150d50 con 0x7f4938003720 2021-01-06T22:04:03.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495580d700 1 --2- 172.21.15.203:0/1558435939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4938003720 0x7f4950161770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f494c012910 tx=0x7f494c01a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:03.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f4941ffb700 1 -- 172.21.15.203:0/1558435939 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c012da0 con 0x7f4938003720 2021-01-06T22:04:03.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f4941ffb700 1 -- 172.21.15.203:0/1558435939 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494c025440 con 0x7f4938003720 2021-01-06T22:04:03.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f4941ffb700 1 -- 172.21.15.203:0/1558435939 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c025800 con 0x7f4938003720 2021-01-06T22:04:03.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495680f700 1 -- 172.21.15.203:0/1558435939 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4950161030 con 0x7f4938003720 2021-01-06T22:04:03.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495680f700 1 -- 172.21.15.203:0/1558435939 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4950161c60 con 0x7f4938003720 2021-01-06T22:04:03.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.259+0000 7f495680f700 1 -- 172.21.15.203:0/1558435939 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49500c53b0 con 0x7f4938003720 2021-01-06T22:04:03.272 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.263+0000 7f4941ffb700 1 -- 172.21.15.203:0/1558435939 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f494c02b020 con 0x7f4938003720 2021-01-06T22:04:03.272 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.263+0000 7f4941ffb700 1 --2- 172.21.15.203:0/1558435939 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f49300563d0 0x7f4930058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.272 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.263+0000 7f4941ffb700 1 -- 172.21.15.203:0/1558435939 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f494c070e30 con 0x7f4938003720 2021-01-06T22:04:03.272 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.263+0000 7f495500c700 1 --2- 172.21.15.203:0/1558435939 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f49300563d0 0x7f4930058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.272 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.263+0000 7f495500c700 1 --2- 172.21.15.203:0/1558435939 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f49300563d0 0x7f4930058890 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f49440038a0 tx=0x7f4944007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:03.273 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.271+0000 7f4941ffb700 1 -- 172.21.15.203:0/1558435939 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f494c0466b0 con 0x7f4938003720 2021-01-06T22:04:03.426 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.423+0000 7f495680f700 1 -- 172.21.15.203:0/1558435939 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f49500450a0 con 0x7f4938003720 2021-01-06T22:04:03.427 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.427+0000 7f4941ffb700 1 -- 172.21.15.203:0/1558435939 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f494c016070 con 0x7f4938003720 2021-01-06T22:04:03.427 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 1 2021-01-06T22:04:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.427+0000 7f495680f700 1 -- 172.21.15.203:0/1558435939 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f49300563d0 msgr2=0x7f4930058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.427+0000 7f495680f700 1 --2- 172.21.15.203:0/1558435939 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f49300563d0 0x7f4930058890 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f49440038a0 tx=0x7f4944007b80).stop 2021-01-06T22:04:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.427+0000 7f495680f700 1 -- 172.21.15.203:0/1558435939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4938003720 msgr2=0x7f4950161770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.427+0000 7f495680f700 1 --2- 172.21.15.203:0/1558435939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4938003720 0x7f4950161770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f494c012910 tx=0x7f494c01a000).stop 2021-01-06T22:04:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.427+0000 7f495680f700 1 -- 172.21.15.203:0/1558435939 shutdown_connections 2021-01-06T22:04:03.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.427+0000 7f495680f700 1 --2- 172.21.15.203:0/1558435939 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f49300563d0 0x7f4930058890 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.427+0000 7f495680f700 1 --2- 172.21.15.203:0/1558435939 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49501502d0 0x7f49500b9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.427+0000 7f495680f700 1 --2- 172.21.15.203:0/1558435939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4938003720 0x7f4950161770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.427+0000 7f495680f700 1 --2- 172.21.15.203:0/1558435939 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49500bccf0 0x7f49500be460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.427+0000 7f495680f700 1 -- 172.21.15.203:0/1558435939 >> 172.21.15.203:0/1558435939 conn(0x7f4950061530 msgr2=0x7f495014b550 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:03.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.427+0000 7f495680f700 1 -- 172.21.15.203:0/1558435939 shutdown_connections 2021-01-06T22:04:03.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.427+0000 7f495680f700 1 -- 172.21.15.203:0/1558435939 wait complete. 2021-01-06T22:04:03.441 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-01-06T22:04:03.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04bfff700 1 Processor -- start 2021-01-06T22:04:03.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04bfff700 1 -- start start 2021-01-06T22:04:03.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff04c0c45d0 0x7ff04c0c1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04bfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff04c0c49b0 0x7ff04c1511c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04bfff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04c151ac0 con 0x7ff04c066940 2021-01-06T22:04:03.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04bfff700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04c151d60 con 0x7ff04c0c49b0 2021-01-06T22:04:03.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04bfff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c152000 con 0x7ff04c0c45d0 2021-01-06T22:04:03.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04a7fc700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff04c0c49b0 0x7ff04c1511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04a7fc700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff04c0c49b0 0x7ff04c1511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49318/0 (socket says 172.21.15.203:49318) 2021-01-06T22:04:03.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04a7fc700 1 -- 172.21.15.203:0/3242344922 learned_addr learned my addr 172.21.15.203:0/3242344922 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:03.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04affd700 1 --2- 172.21.15.203:0/3242344922 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff04c0c45d0 0x7ff04c0c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04a7fc700 1 -- 172.21.15.203:0/3242344922 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff04c0c45d0 msgr2=0x7ff04c0c1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:03.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04a7fc700 1 --2- 172.21.15.203:0/3242344922 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff04c0c45d0 0x7ff04c0c1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04a7fc700 1 -- 172.21.15.203:0/3242344922 >> v1:172.21.15.91:6789/0 conn(0x7ff04c066940 legacy=0x7ff04c0bdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.576 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04a7fc700 1 -- 172.21.15.203:0/3242344922 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c0c0c60 con 0x7ff04c0c49b0 2021-01-06T22:04:03.576 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04a7fc700 1 --2- 172.21.15.203:0/3242344922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff04c0c49b0 0x7ff04c1511c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7ff03400b9e0 tx=0x7ff034006ea0).ready entity=mon.1 client_cookie=b62880fa61b83ff3 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:03.579 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff049ffb700 1 -- 172.21.15.203:0/3242344922 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034009810 con 0x7ff04c0c49b0 2021-01-06T22:04:03.579 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff049ffb700 1 -- 172.21.15.203:0/3242344922 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff034009970 con 0x7ff04c0c49b0 2021-01-06T22:04:03.579 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff049ffb700 1 -- 172.21.15.203:0/3242344922 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034009cf0 con 0x7ff04c0c49b0 2021-01-06T22:04:03.579 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04bfff700 1 -- 172.21.15.203:0/3242344922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff04c0c49b0 msgr2=0x7ff04c1511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.580 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04bfff700 1 --2- 172.21.15.203:0/3242344922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff04c0c49b0 0x7ff04c1511c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7ff03400b9e0 tx=0x7ff034006ea0).stop 2021-01-06T22:04:03.580 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04bfff700 1 -- 172.21.15.203:0/3242344922 shutdown_connections 2021-01-06T22:04:03.580 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04bfff700 1 --2- 172.21.15.203:0/3242344922 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff04c0c45d0 0x7ff04c0c1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.581 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04bfff700 1 --2- 172.21.15.203:0/3242344922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff04c0c49b0 0x7ff04c1511c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.581 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.571+0000 7ff04bfff700 1 -- 172.21.15.203:0/3242344922 >> 172.21.15.203:0/3242344922 conn(0x7ff04c0626e0 msgr2=0x7ff04c062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:03.581 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04bfff700 1 -- 172.21.15.203:0/3242344922 shutdown_connections 2021-01-06T22:04:03.581 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04bfff700 1 -- 172.21.15.203:0/3242344922 wait complete. 2021-01-06T22:04:03.581 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04bfff700 1 Processor -- start 2021-01-06T22:04:03.582 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04bfff700 1 -- start start 2021-01-06T22:04:03.582 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff04c066940 0x7ff04c0c23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.582 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff04c0c45d0 0x7ff04c0b9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.582 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff04c14c050 0x7ff04c0b9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.582 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04bfff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c0b9e30 con 0x7ff04c066940 2021-01-06T22:04:03.583 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04bfff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c0baef0 con 0x7ff04c0c45d0 2021-01-06T22:04:03.583 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04bfff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c15e5d0 con 0x7ff04c14c050 2021-01-06T22:04:03.583 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04a7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff04c0c45d0 0x7ff04c0b9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.583 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04a7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff04c0c45d0 0x7ff04c0b9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49324/0 (socket says 172.21.15.203:49324) 2021-01-06T22:04:03.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04a7fc700 1 -- 172.21.15.203:0/2647876506 learned_addr learned my addr 172.21.15.203:0/2647876506 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:03.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04affd700 1 --2- 172.21.15.203:0/2647876506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff04c066940 0x7ff04c0c23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04b7fe700 1 --2- 172.21.15.203:0/2647876506 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff04c14c050 0x7ff04c0b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04a7fc700 1 -- 172.21.15.203:0/2647876506 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff04c14c050 msgr2=0x7ff04c0b9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04a7fc700 1 --2- 172.21.15.203:0/2647876506 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff04c14c050 0x7ff04c0b9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04a7fc700 1 -- 172.21.15.203:0/2647876506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff04c066940 msgr2=0x7ff04c0c23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04a7fc700 1 --2- 172.21.15.203:0/2647876506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff04c066940 0x7ff04c0c23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04a7fc700 1 -- 172.21.15.203:0/2647876506 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c15e870 con 0x7ff04c0c45d0 2021-01-06T22:04:03.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04a7fc700 1 --2- 172.21.15.203:0/2647876506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff04c0c45d0 0x7ff04c0b9450 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7ff034007fd0 tx=0x7ff034016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:03.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff042ffd700 1 -- 172.21.15.203:0/2647876506 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034006880 con 0x7ff04c0c45d0 2021-01-06T22:04:03.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04bfff700 1 -- 172.21.15.203:0/2647876506 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04c152580 con 0x7ff04c0c45d0 2021-01-06T22:04:03.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff042ffd700 1 -- 172.21.15.203:0/2647876506 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff034012c90 con 0x7ff04c0c45d0 2021-01-06T22:04:03.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff042ffd700 1 -- 172.21.15.203:0/2647876506 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03401c650 con 0x7ff04c0c45d0 2021-01-06T22:04:03.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.575+0000 7ff04bfff700 1 -- 172.21.15.203:0/2647876506 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04c152ac0 con 0x7ff04c0c45d0 2021-01-06T22:04:03.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.579+0000 7ff04bfff700 1 -- 172.21.15.203:0/2647876506 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04c0450a0 con 0x7ff04c0c45d0 2021-01-06T22:04:03.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.579+0000 7ff042ffd700 1 -- 172.21.15.203:0/2647876506 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff03400f070 con 0x7ff04c0c45d0 2021-01-06T22:04:03.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.579+0000 7ff042ffd700 1 --2- 172.21.15.203:0/2647876506 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0300563d0 0x7ff030058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.579+0000 7ff042ffd700 1 -- 172.21.15.203:0/2647876506 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff034070cb0 con 0x7ff04c0c45d0 2021-01-06T22:04:03.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.579+0000 7ff04affd700 1 --2- 172.21.15.203:0/2647876506 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0300563d0 0x7ff030058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.579+0000 7ff04affd700 1 --2- 172.21.15.203:0/2647876506 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0300563d0 0x7ff030058890 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7ff044009cd0 tx=0x7ff0440091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:03.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.587+0000 7ff042ffd700 1 -- 172.21.15.203:0/2647876506 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff034046420 con 0x7ff04c0c45d0 2021-01-06T22:04:03.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.735+0000 7ff04bfff700 1 -- 172.21.15.203:0/2647876506 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff04c062ac0 con 0x7ff04c0c45d0 2021-01-06T22:04:03.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.739+0000 7ff042ffd700 1 -- 172.21.15.203:0/2647876506 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff034049a40 con 0x7ff04c0c45d0 2021-01-06T22:04:03.739 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:04:03.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.739+0000 7ff04bfff700 1 -- 172.21.15.203:0/2647876506 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0300563d0 msgr2=0x7ff030058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.739+0000 7ff04bfff700 1 --2- 172.21.15.203:0/2647876506 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0300563d0 0x7ff030058890 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7ff044009cd0 tx=0x7ff0440091d0).stop 2021-01-06T22:04:03.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.739+0000 7ff04bfff700 1 -- 172.21.15.203:0/2647876506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff04c0c45d0 msgr2=0x7ff04c0b9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.739+0000 7ff04bfff700 1 --2- 172.21.15.203:0/2647876506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff04c0c45d0 0x7ff04c0b9450 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7ff034007fd0 tx=0x7ff034016000).stop 2021-01-06T22:04:03.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.739+0000 7ff04bfff700 1 -- 172.21.15.203:0/2647876506 shutdown_connections 2021-01-06T22:04:03.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.739+0000 7ff04bfff700 1 --2- 172.21.15.203:0/2647876506 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0300563d0 0x7ff030058890 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.742 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.739+0000 7ff04bfff700 1 --2- 172.21.15.203:0/2647876506 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff04c14c050 0x7ff04c0b9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.742 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.739+0000 7ff04bfff700 1 --2- 172.21.15.203:0/2647876506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff04c066940 0x7ff04c0c23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.742 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.739+0000 7ff04bfff700 1 --2- 172.21.15.203:0/2647876506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff04c0c45d0 0x7ff04c0b9450 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.742 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.739+0000 7ff04bfff700 1 -- 172.21.15.203:0/2647876506 >> 172.21.15.203:0/2647876506 conn(0x7ff04c0626e0 msgr2=0x7ff04c0c36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:03.743 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.739+0000 7ff04bfff700 1 -- 172.21.15.203:0/2647876506 shutdown_connections 2021-01-06T22:04:03.743 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.739+0000 7ff04bfff700 1 -- 172.21.15.203:0/2647876506 wait complete. 2021-01-06T22:04:03.753 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-01-06T22:04:03.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1dbb59e700 1 Processor -- start 2021-01-06T22:04:03.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1dbb59e700 1 -- start start 2021-01-06T22:04:03.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1dbb59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dbc0bab60 0x7f1dbc060b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1dbb59e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1dbc0baf40 0x7f1dbc155640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1dbb59e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dbc067a50 con 0x7f1dbc066ad0 2021-01-06T22:04:03.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1dbb59e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dbc0c02f0 con 0x7f1dbc0baf40 2021-01-06T22:04:03.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1dbb59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc0c0590 con 0x7f1dbc0bab60 2021-01-06T22:04:03.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1dbad9d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f1dbc066ad0 0x7f1dbc0b7460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47924/0 (socket says 172.21.15.203:47924) 2021-01-06T22:04:03.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1dbad9d700 1 -- 172.21.15.203:0/2007036013 learned_addr learned my addr 172.21.15.203:0/2007036013 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:04:03.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db9d9b700 1 --2- 172.21.15.203:0/2007036013 >> v2:172.21.15.91:3300/0 conn(0x7f1dbc0baf40 0x7f1dbc155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1dba59c700 1 --2- 172.21.15.203:0/2007036013 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dbc0bab60 0x7f1dbc060b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db959a700 1 -- 172.21.15.203:0/2007036013 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2581587081 0 0) 0x7f1dbc067a50 con 0x7f1dbc066ad0 2021-01-06T22:04:03.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db959a700 1 -- 172.21.15.203:0/2007036013 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1da0003230 con 0x7f1dbc066ad0 2021-01-06T22:04:03.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db959a700 1 -- 172.21.15.203:0/2007036013 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2044405138 0 0) 0x7f1da0003230 con 0x7f1dbc066ad0 2021-01-06T22:04:03.886 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db959a700 1 -- 172.21.15.203:0/2007036013 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dbc0bab60 msgr2=0x7f1dbc060b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:03.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db959a700 1 --2- 172.21.15.203:0/2007036013 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dbc0bab60 0x7f1dbc060b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db959a700 1 -- 172.21.15.203:0/2007036013 >> v2:172.21.15.91:3300/0 conn(0x7f1dbc0baf40 msgr2=0x7f1dbc155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:03.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db959a700 1 --2- 172.21.15.203:0/2007036013 >> v2:172.21.15.91:3300/0 conn(0x7f1dbc0baf40 0x7f1dbc155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db959a700 1 -- 172.21.15.203:0/2007036013 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dbc0c0830 con 0x7f1dbc066ad0 2021-01-06T22:04:03.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db959a700 1 -- 172.21.15.203:0/2007036013 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f1dac001860 con 0x7f1dbc066ad0 2021-01-06T22:04:03.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db959a700 1 -- 172.21.15.203:0/2007036013 >> v1:172.21.15.203:6789/0 conn(0x7f1dbc066ad0 legacy=0x7f1dbc0b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db959a700 1 --2- 172.21.15.203:0/2007036013 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1da00037a0 0x7f1da0003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db959a700 1 --2- 172.21.15.203:0/2007036013 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dbc0bab60 0x7f1da00044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db959a700 1 --2- 172.21.15.203:0/2007036013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da00049c0 0x7f1da0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db959a700 1 -- 172.21.15.203:0/2007036013 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc0c02f0 con 0x7f1da00037a0 2021-01-06T22:04:03.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db959a700 1 -- 172.21.15.203:0/2007036013 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc0c0590 con 0x7f1da00049c0 2021-01-06T22:04:03.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db959a700 1 -- 172.21.15.203:0/2007036013 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da0004290 con 0x7f1dbc0bab60 2021-01-06T22:04:03.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1dbad9d700 1 --2- 172.21.15.203:0/2007036013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da00049c0 0x7f1da0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1dba59c700 1 --2- 172.21.15.203:0/2007036013 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1da00037a0 0x7f1da0003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1db9d9b700 1 --2- 172.21.15.203:0/2007036013 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dbc0bab60 0x7f1da00044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1dbad9d700 1 -- 172.21.15.203:0/2007036013 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dbc0bab60 msgr2=0x7f1da00044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:03.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1dbad9d700 1 --2- 172.21.15.203:0/2007036013 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dbc0bab60 0x7f1da00044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1dbad9d700 1 -- 172.21.15.203:0/2007036013 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1da00037a0 msgr2=0x7f1da0003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:03.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.879+0000 7f1dbad9d700 1 --2- 172.21.15.203:0/2007036013 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1da00037a0 0x7f1da0003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.883+0000 7f1dbad9d700 1 -- 172.21.15.203:0/2007036013 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da0003230 con 0x7f1da00049c0 2021-01-06T22:04:03.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.883+0000 7f1dbad9d700 1 --2- 172.21.15.203:0/2007036013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da00049c0 0x7f1da0004e40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f1dac00c550 tx=0x7f1dbc0bc6d0).ready entity=mon.1 client_cookie=ca6c338d61abaf87 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:03.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.883+0000 7f1db959a700 1 -- 172.21.15.203:0/2007036013 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac005b70 con 0x7f1da00049c0 2021-01-06T22:04:03.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.883+0000 7f1db959a700 1 -- 172.21.15.203:0/2007036013 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dac005cd0 con 0x7f1da00049c0 2021-01-06T22:04:03.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.883+0000 7f1db959a700 1 -- 172.21.15.203:0/2007036013 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0072e0 con 0x7f1da00049c0 2021-01-06T22:04:03.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.883+0000 7f1dbb59e700 1 -- 172.21.15.203:0/2007036013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da00049c0 msgr2=0x7f1da0004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.883+0000 7f1dbb59e700 1 --2- 172.21.15.203:0/2007036013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da00049c0 0x7f1da0004e40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f1dac00c550 tx=0x7f1dbc0bc6d0).stop 2021-01-06T22:04:03.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.883+0000 7f1dbad9d700 1 -- 172.21.15.203:0/2007036013 reap_dead start 2021-01-06T22:04:03.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.883+0000 7f1dbb59e700 1 -- 172.21.15.203:0/2007036013 shutdown_connections 2021-01-06T22:04:03.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.883+0000 7f1dbb59e700 1 -- 172.21.15.203:0/2007036013 >> 172.21.15.203:0/2007036013 conn(0x7f1dbc0626e0 msgr2=0x7f1dbc062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:03.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.883+0000 7f1dbb59e700 1 -- 172.21.15.203:0/2007036013 shutdown_connections 2021-01-06T22:04:03.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.883+0000 7f1dbb59e700 1 -- 172.21.15.203:0/2007036013 wait complete. 2021-01-06T22:04:03.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.883+0000 7f1dbb59e700 1 Processor -- start 2021-01-06T22:04:03.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.883+0000 7f1dbb59e700 1 -- start start 2021-01-06T22:04:03.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1dbb59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dbc0bece0 0x7f1dbc14aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1dbb59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dbc14bfb0 0x7f1dbc14b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1dbb59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dbc1509a0 0x7f1dbc13f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1dbb59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc157c80 con 0x7f1dbc0bece0 2021-01-06T22:04:03.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1dbb59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc152b20 con 0x7f1dbc14bfb0 2021-01-06T22:04:03.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1dbb59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc1581c0 con 0x7f1dbc1509a0 2021-01-06T22:04:03.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1db9d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dbc14bfb0 0x7f1dbc14b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1db9d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dbc14bfb0 0x7f1dbc14b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49342/0 (socket says 172.21.15.203:49342) 2021-01-06T22:04:03.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1db9d9b700 1 -- 172.21.15.203:0/2132631506 learned_addr learned my addr 172.21.15.203:0/2132631506 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:03.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1dba59c700 1 --2- 172.21.15.203:0/2132631506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dbc0bece0 0x7f1dbc14aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1dbad9d700 1 --2- 172.21.15.203:0/2132631506 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dbc1509a0 0x7f1dbc13f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1db9d9b700 1 -- 172.21.15.203:0/2132631506 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dbc1509a0 msgr2=0x7f1dbc13f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1db9d9b700 1 --2- 172.21.15.203:0/2132631506 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dbc1509a0 0x7f1dbc13f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1db9d9b700 1 -- 172.21.15.203:0/2132631506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dbc0bece0 msgr2=0x7f1dbc14aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:03.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1db9d9b700 1 --2- 172.21.15.203:0/2132631506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dbc0bece0 0x7f1dbc14aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:03.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1db9d9b700 1 -- 172.21.15.203:0/2132631506 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dbc152d80 con 0x7f1dbc14bfb0 2021-01-06T22:04:03.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1dba59c700 1 --2- 172.21.15.203:0/2132631506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dbc0bece0 0x7f1dbc14aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:04:03.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1db9d9b700 1 --2- 172.21.15.203:0/2132631506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dbc14bfb0 0x7f1dbc14b3b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f1da4011a90 tx=0x7f1da4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:03.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1db27fc700 1 -- 172.21.15.203:0/2132631506 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da400f910 con 0x7f1dbc14bfb0 2021-01-06T22:04:03.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1db27fc700 1 -- 172.21.15.203:0/2132631506 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da400fa70 con 0x7f1dbc14bfb0 2021-01-06T22:04:03.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1dbb59e700 1 -- 172.21.15.203:0/2132631506 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dbc152210 con 0x7f1dbc14bfb0 2021-01-06T22:04:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1db27fc700 1 -- 172.21.15.203:0/2132631506 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da400fdf0 con 0x7f1dbc14bfb0 2021-01-06T22:04:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.887+0000 7f1dbb59e700 1 -- 172.21.15.203:0/2132631506 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dbc157e80 con 0x7f1dbc14bfb0 2021-01-06T22:04:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.891+0000 7f1dbb59e700 1 -- 172.21.15.203:0/2132631506 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dbc0450a0 con 0x7f1dbc14bfb0 2021-01-06T22:04:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.891+0000 7f1db27fc700 1 -- 172.21.15.203:0/2132631506 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f1da4024070 con 0x7f1dbc14bfb0 2021-01-06T22:04:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.891+0000 7f1db27fc700 1 --2- 172.21.15.203:0/2132631506 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d940563d0 0x7f1d94058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:03.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.891+0000 7f1db27fc700 1 -- 172.21.15.203:0/2132631506 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1da406ff80 con 0x7f1dbc14bfb0 2021-01-06T22:04:03.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.891+0000 7f1dba59c700 1 --2- 172.21.15.203:0/2132631506 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d940563d0 0x7f1d94058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:03.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.891+0000 7f1dba59c700 1 --2- 172.21.15.203:0/2132631506 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d940563d0 0x7f1d94058890 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f1db40143b0 tx=0x7f1db4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:03.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:03.899+0000 7f1db27fc700 1 -- 172.21.15.203:0/2132631506 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f1da40456f0 con 0x7f1dbc14bfb0 2021-01-06T22:04:04.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.047+0000 7f1dbb59e700 1 -- 172.21.15.203:0/2132631506 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1dbc0be920 con 0x7f1dbc14bfb0 2021-01-06T22:04:04.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.047+0000 7f1db27fc700 1 -- 172.21.15.203:0/2132631506 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1da4018070 con 0x7f1dbc14bfb0 2021-01-06T22:04:04.049 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:04:04.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.047+0000 7f1dbb59e700 1 -- 172.21.15.203:0/2132631506 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d940563d0 msgr2=0x7f1d94058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.047+0000 7f1dbb59e700 1 --2- 172.21.15.203:0/2132631506 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d940563d0 0x7f1d94058890 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f1db40143b0 tx=0x7f1db4008040).stop 2021-01-06T22:04:04.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.047+0000 7f1dbb59e700 1 -- 172.21.15.203:0/2132631506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dbc14bfb0 msgr2=0x7f1dbc14b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.047+0000 7f1dbb59e700 1 --2- 172.21.15.203:0/2132631506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dbc14bfb0 0x7f1dbc14b3b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f1da4011a90 tx=0x7f1da4008040).stop 2021-01-06T22:04:04.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.051+0000 7f1dbb59e700 1 -- 172.21.15.203:0/2132631506 shutdown_connections 2021-01-06T22:04:04.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.051+0000 7f1dbb59e700 1 --2- 172.21.15.203:0/2132631506 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d940563d0 0x7f1d94058890 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.051+0000 7f1dbb59e700 1 --2- 172.21.15.203:0/2132631506 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dbc1509a0 0x7f1dbc13f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.051+0000 7f1dbb59e700 1 --2- 172.21.15.203:0/2132631506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dbc0bece0 0x7f1dbc14aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.051+0000 7f1dbb59e700 1 --2- 172.21.15.203:0/2132631506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dbc14bfb0 0x7f1dbc14b3b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.051+0000 7f1dbb59e700 1 -- 172.21.15.203:0/2132631506 >> 172.21.15.203:0/2132631506 conn(0x7f1dbc0626e0 msgr2=0x7f1dbc062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:04.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.051+0000 7f1dbb59e700 1 -- 172.21.15.203:0/2132631506 shutdown_connections 2021-01-06T22:04:04.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.051+0000 7f1dbb59e700 1 -- 172.21.15.203:0/2132631506 wait complete. 2021-01-06T22:04:04.063 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-01-06T22:04:04.186 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.183+0000 7f029ffff700 1 Processor -- start 2021-01-06T22:04:04.187 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.183+0000 7f029ffff700 1 -- start start 2021-01-06T22:04:04.189 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029ffff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f02a00c49b0 0x7f02a01511c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.190 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029ffff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a0151ac0 con 0x7f02a0066940 2021-01-06T22:04:04.190 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029ffff700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a0151d60 con 0x7f02a00c45d0 2021-01-06T22:04:04.190 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029ffff700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02a00c1d50 con 0x7f02a00c49b0 2021-01-06T22:04:04.190 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029effd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f02a00c45d0 0x7f02a00c1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47942/0 (socket says 172.21.15.203:47942) 2021-01-06T22:04:04.191 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029effd700 1 -- 172.21.15.203:0/2378349388 learned_addr learned my addr 172.21.15.203:0/2378349388 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:04:04.191 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029e7fc700 1 --2- 172.21.15.203:0/2378349388 >> v2:172.21.15.91:3300/0 conn(0x7f02a00c49b0 0x7f02a01511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.191 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 -- 172.21.15.203:0/2378349388 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511421917 0 0) 0x7f02a0151d60 con 0x7f02a00c45d0 2021-01-06T22:04:04.191 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 -- 172.21.15.203:0/2378349388 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0280003230 con 0x7f02a00c45d0 2021-01-06T22:04:04.191 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 -- 172.21.15.203:0/2378349388 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1982049468 0 0) 0x7f0280003230 con 0x7f02a00c45d0 2021-01-06T22:04:04.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 -- 172.21.15.203:0/2378349388 >> v2:172.21.15.91:3300/0 conn(0x7f02a00c49b0 msgr2=0x7f02a01511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:04.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 --2- 172.21.15.203:0/2378349388 >> v2:172.21.15.91:3300/0 conn(0x7f02a00c49b0 0x7f02a01511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 -- 172.21.15.203:0/2378349388 >> v1:172.21.15.91:6789/0 conn(0x7f02a0066940 legacy=0x7f02a00bdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 -- 172.21.15.203:0/2378349388 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a0152000 con 0x7f02a00c45d0 2021-01-06T22:04:04.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 -- 172.21.15.203:0/2378349388 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f02980014e0 con 0x7f02a00c45d0 2021-01-06T22:04:04.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 -- 172.21.15.203:0/2378349388 >> v1:172.21.15.203:6789/0 conn(0x7f02a00c45d0 legacy=0x7f02a00c1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 --2- 172.21.15.203:0/2378349388 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0280003720 0x7f0280003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 --2- 172.21.15.203:0/2378349388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02800044d0 0x7f0280004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 --2- 172.21.15.203:0/2378349388 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0280004e40 0x7f028000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 -- 172.21.15.203:0/2378349388 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a00c1d50 con 0x7f0280003720 2021-01-06T22:04:04.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 -- 172.21.15.203:0/2378349388 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0280004250 con 0x7f02800044d0 2021-01-06T22:04:04.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 -- 172.21.15.203:0/2378349388 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028000bc10 con 0x7f0280004e40 2021-01-06T22:04:04.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029e7fc700 1 --2- 172.21.15.203:0/2378349388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02800044d0 0x7f0280004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029effd700 1 --2- 172.21.15.203:0/2378349388 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0280003720 0x7f0280003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029e7fc700 1 -- 172.21.15.203:0/2378349388 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0280004e40 msgr2=0x7f028000b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:04.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029e7fc700 1 --2- 172.21.15.203:0/2378349388 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0280004e40 0x7f028000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:04:04.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029e7fc700 1 -- 172.21.15.203:0/2378349388 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0280003720 msgr2=0x7f0280003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:04.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029e7fc700 1 --2- 172.21.15.203:0/2378349388 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0280003720 0x7f0280003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029e7fc700 1 -- 172.21.15.203:0/2378349388 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0280003230 con 0x7f02800044d0 2021-01-06T22:04:04.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029f7fe700 1 -- 172.21.15.203:0/2378349388 reap_dead start 2021-01-06T22:04:04.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029e7fc700 1 --2- 172.21.15.203:0/2378349388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02800044d0 0x7f0280004950 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f028800bb60 tx=0x7f0288008040).ready entity=mon.1 client_cookie=58b324954f82754c server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:04.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 -- 172.21.15.203:0/2378349388 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028800b2b0 con 0x7f02800044d0 2021-01-06T22:04:04.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 -- 172.21.15.203:0/2378349388 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028800ea90 con 0x7f02800044d0 2021-01-06T22:04:04.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029dffb700 1 -- 172.21.15.203:0/2378349388 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028800ee30 con 0x7f02800044d0 2021-01-06T22:04:04.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029ffff700 1 -- 172.21.15.203:0/2378349388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02800044d0 msgr2=0x7f0280004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029ffff700 1 --2- 172.21.15.203:0/2378349388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02800044d0 0x7f0280004950 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f028800bb60 tx=0x7f0288008040).stop 2021-01-06T22:04:04.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029ffff700 1 -- 172.21.15.203:0/2378349388 shutdown_connections 2021-01-06T22:04:04.198 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029ffff700 1 --2- 172.21.15.203:0/2378349388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02800044d0 0x7f0280004950 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.198 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029ffff700 1 -- 172.21.15.203:0/2378349388 >> 172.21.15.203:0/2378349388 conn(0x7f02a00626e0 msgr2=0x7f02a0062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:04.198 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029ffff700 1 -- 172.21.15.203:0/2378349388 shutdown_connections 2021-01-06T22:04:04.198 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.187+0000 7f029ffff700 1 -- 172.21.15.203:0/2378349388 wait complete. 2021-01-06T22:04:04.201 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029ffff700 1 Processor -- start 2021-01-06T22:04:04.201 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029ffff700 1 -- start start 2021-01-06T22:04:04.201 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029ffff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02800044d0 0x7f02a01462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.201 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029ffff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02a00b96c0 0x7f02a0147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.202 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02a014e650 0x7f02a0148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.202 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029ffff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a014ba80 con 0x7f02a00b96c0 2021-01-06T22:04:04.202 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029ffff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a00b86c0 con 0x7f02a014e650 2021-01-06T22:04:04.202 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029ffff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a00b8960 con 0x7f02800044d0 2021-01-06T22:04:04.202 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02a014e650 0x7f02a0148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.203 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02a014e650 0x7f02a0148450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49364/0 (socket says 172.21.15.203:49364) 2021-01-06T22:04:04.203 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029f7fe700 1 -- 172.21.15.203:0/216658454 learned_addr learned my addr 172.21.15.203:0/216658454 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:04.203 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029effd700 1 --2- 172.21.15.203:0/216658454 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02800044d0 0x7f02a01462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.203 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029e7fc700 1 --2- 172.21.15.203:0/216658454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02a00b96c0 0x7f02a0147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029f7fe700 1 -- 172.21.15.203:0/216658454 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02800044d0 msgr2=0x7f02a01462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029f7fe700 1 --2- 172.21.15.203:0/216658454 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02800044d0 0x7f02a01462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029f7fe700 1 -- 172.21.15.203:0/216658454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02a00b96c0 msgr2=0x7f02a0147f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029f7fe700 1 --2- 172.21.15.203:0/216658454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02a00b96c0 0x7f02a0147f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029f7fe700 1 -- 172.21.15.203:0/216658454 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a014d0f0 con 0x7f02a014e650 2021-01-06T22:04:04.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029e7fc700 1 --2- 172.21.15.203:0/216658454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02a00b96c0 0x7f02a0147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:04:04.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029f7fe700 1 --2- 172.21.15.203:0/216658454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02a014e650 0x7f02a0148450 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f02a0065f00 tx=0x7f0290007ce0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:04.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f0296ffd700 1 -- 172.21.15.203:0/216658454 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290005a60 con 0x7f02a014e650 2021-01-06T22:04:04.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f0296ffd700 1 -- 172.21.15.203:0/216658454 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0290005be0 con 0x7f02a014e650 2021-01-06T22:04:04.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029ffff700 1 -- 172.21.15.203:0/216658454 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a014d390 con 0x7f02a014e650 2021-01-06T22:04:04.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f0296ffd700 1 -- 172.21.15.203:0/216658454 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02900070d0 con 0x7f02a014e650 2021-01-06T22:04:04.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.191+0000 7f029ffff700 1 -- 172.21.15.203:0/216658454 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a01467c0 con 0x7f02a014e650 2021-01-06T22:04:04.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.195+0000 7f029ffff700 1 -- 172.21.15.203:0/216658454 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a00c1380 con 0x7f02a014e650 2021-01-06T22:04:04.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.195+0000 7f0296ffd700 1 -- 172.21.15.203:0/216658454 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f0290010070 con 0x7f02a014e650 2021-01-06T22:04:04.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.195+0000 7f0296ffd700 1 --2- 172.21.15.203:0/216658454 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f02840563d0 0x7f0284058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.195+0000 7f0296ffd700 1 -- 172.21.15.203:0/216658454 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f029006ed20 con 0x7f02a014e650 2021-01-06T22:04:04.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.195+0000 7f029effd700 1 --2- 172.21.15.203:0/216658454 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f02840563d0 0x7f0284058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.195+0000 7f029effd700 1 --2- 172.21.15.203:0/216658454 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f02840563d0 0x7f0284058890 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f0298001010 tx=0x7f0298007040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:04.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.199+0000 7f0296ffd700 1 -- 172.21.15.203:0/216658454 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f02a00c1380 con 0x7f02a014e650 2021-01-06T22:04:04.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.351+0000 7f029ffff700 1 -- 172.21.15.203:0/216658454 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02a00c1380 con 0x7f02a014e650 2021-01-06T22:04:04.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.351+0000 7f0296ffd700 1 -- 172.21.15.203:0/216658454 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f02a00c1380 con 0x7f02a014e650 2021-01-06T22:04:04.354 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:04:04.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.355+0000 7f029ffff700 1 -- 172.21.15.203:0/216658454 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f02840563d0 msgr2=0x7f0284058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.355+0000 7f029ffff700 1 --2- 172.21.15.203:0/216658454 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f02840563d0 0x7f0284058890 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f0298001010 tx=0x7f0298007040).stop 2021-01-06T22:04:04.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.355+0000 7f029ffff700 1 -- 172.21.15.203:0/216658454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02a014e650 msgr2=0x7f02a0148450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.355+0000 7f029ffff700 1 --2- 172.21.15.203:0/216658454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02a014e650 0x7f02a0148450 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f02a0065f00 tx=0x7f0290007ce0).stop 2021-01-06T22:04:04.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.355+0000 7f029ffff700 1 -- 172.21.15.203:0/216658454 shutdown_connections 2021-01-06T22:04:04.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.355+0000 7f029ffff700 1 --2- 172.21.15.203:0/216658454 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f02840563d0 0x7f0284058890 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.355+0000 7f029ffff700 1 --2- 172.21.15.203:0/216658454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02a014e650 0x7f02a0148450 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.355+0000 7f029ffff700 1 --2- 172.21.15.203:0/216658454 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02800044d0 0x7f02a01462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.355+0000 7f029ffff700 1 --2- 172.21.15.203:0/216658454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02a00b96c0 0x7f02a0147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.355+0000 7f029ffff700 1 -- 172.21.15.203:0/216658454 >> 172.21.15.203:0/216658454 conn(0x7f02a00626e0 msgr2=0x7f02a0062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:04.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.355+0000 7f029ffff700 1 -- 172.21.15.203:0/216658454 shutdown_connections 2021-01-06T22:04:04.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.355+0000 7f029ffff700 1 -- 172.21.15.203:0/216658454 wait complete. 2021-01-06T22:04:04.369 INFO:tasks.mds_thrash:mds thrasher using random seed: 1609970644 2021-01-06T22:04:04.369 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-06T22:04:04.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.491+0000 7f9fdb59e700 1 Processor -- start 2021-01-06T22:04:04.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.491+0000 7f9fdb59e700 1 -- start start 2021-01-06T22:04:04.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.491+0000 7f9fdb59e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9fdc0bb3f0 0x7f9fdc0ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.491+0000 7f9fdb59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9fdc0a5e50 0x7f9fdc0669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.491+0000 7f9fdb59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fdc0646b0 con 0x7f9fdc0bb7d0 2021-01-06T22:04:04.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.491+0000 7f9fdb59e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fdc064450 con 0x7f9fdc0bb3f0 2021-01-06T22:04:04.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.491+0000 7f9fdb59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc068530 con 0x7f9fdc0a5e50 2021-01-06T22:04:04.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.491+0000 7f9fd9d9b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9fdc0a5e50 0x7f9fdc0669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.491+0000 7f9fda59c700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9fdc0bb3f0 0x7f9fdc0ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.491+0000 7f9fd9d9b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9fdc0a5e50 0x7f9fdc0669b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:54450/0 (socket says 172.21.15.203:54450) 2021-01-06T22:04:04.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.491+0000 7f9fd9d9b700 1 -- 172.21.15.203:0/3541144507 learned_addr learned my addr 172.21.15.203:0/3541144507 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:04.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.491+0000 7f9fdad9d700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f9fdc0bb7d0 0x7f9fdc0602b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:42620/0 (socket says 172.21.15.203:42620) 2021-01-06T22:04:04.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.495+0000 7f9fd9d9b700 1 -- 172.21.15.203:0/3541144507 >> v2:172.21.15.91:3300/0 conn(0x7f9fdc0bb3f0 msgr2=0x7f9fdc0ba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:04.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.495+0000 7f9fd9d9b700 1 --2- 172.21.15.203:0/3541144507 >> v2:172.21.15.91:3300/0 conn(0x7f9fdc0bb3f0 0x7f9fdc0ba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.495+0000 7f9fd9d9b700 1 -- 172.21.15.203:0/3541144507 >> v1:172.21.15.91:6789/0 conn(0x7f9fdc0bb7d0 legacy=0x7f9fdc0602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.495+0000 7f9fd9d9b700 1 -- 172.21.15.203:0/3541144507 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc152cc0 con 0x7f9fdc0a5e50 2021-01-06T22:04:04.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.495+0000 7f9fd9d9b700 1 --2- 172.21.15.203:0/3541144507 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9fdc0a5e50 0x7f9fdc0669b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9fc400b9e0 tx=0x7f9fc4006ea0).ready entity=mon.2 client_cookie=293ad31d8509cc9e server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:04.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.495+0000 7f9fd959a700 1 -- 172.21.15.203:0/3541144507 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4009810 con 0x7f9fdc0a5e50 2021-01-06T22:04:04.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.495+0000 7f9fd959a700 1 -- 172.21.15.203:0/3541144507 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc4009970 con 0x7f9fdc0a5e50 2021-01-06T22:04:04.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.495+0000 7f9fd959a700 1 -- 172.21.15.203:0/3541144507 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4009cf0 con 0x7f9fdc0a5e50 2021-01-06T22:04:04.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.495+0000 7f9fdb59e700 1 -- 172.21.15.203:0/3541144507 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9fdc0a5e50 msgr2=0x7f9fdc0669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.495+0000 7f9fdb59e700 1 --2- 172.21.15.203:0/3541144507 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9fdc0a5e50 0x7f9fdc0669b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9fc400b9e0 tx=0x7f9fc4006ea0).stop 2021-01-06T22:04:04.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.495+0000 7f9fdb59e700 1 -- 172.21.15.203:0/3541144507 shutdown_connections 2021-01-06T22:04:04.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.495+0000 7f9fdb59e700 1 --2- 172.21.15.203:0/3541144507 >> v2:172.21.15.91:3300/0 conn(0x7f9fdc0bb3f0 0x7f9fdc0ba9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.495+0000 7f9fdb59e700 1 --2- 172.21.15.203:0/3541144507 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9fdc0a5e50 0x7f9fdc0669b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.495+0000 7f9fdb59e700 1 -- 172.21.15.203:0/3541144507 >> 172.21.15.203:0/3541144507 conn(0x7f9fdc061530 msgr2=0x7f9fdc063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:04.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.495+0000 7f9fdb59e700 1 -- 172.21.15.203:0/3541144507 shutdown_connections 2021-01-06T22:04:04.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.495+0000 7f9fdb59e700 1 -- 172.21.15.203:0/3541144507 wait complete. 2021-01-06T22:04:04.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fdb59e700 1 Processor -- start 2021-01-06T22:04:04.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fdb59e700 1 -- start start 2021-01-06T22:04:04.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fdb59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9fdc0a5e50 0x7f9fdc0b9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fdb59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fdc0bb3f0 0x7f9fdc164b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fdb59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9fdc0bb7d0 0x7f9fdc0c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fdb59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0b9c30 con 0x7f9fdc0a5e50 2021-01-06T22:04:04.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fdb59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0b9580 con 0x7f9fdc0bb3f0 2021-01-06T22:04:04.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fdb59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc165270 con 0x7f9fdc0bb7d0 2021-01-06T22:04:04.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fd9d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fdc0bb3f0 0x7f9fdc164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fd9d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fdc0bb3f0 0x7f9fdc164b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49376/0 (socket says 172.21.15.203:49376) 2021-01-06T22:04:04.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fd9d9b700 1 -- 172.21.15.203:0/2423404830 learned_addr learned my addr 172.21.15.203:0/2423404830 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:04.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fda59c700 1 --2- 172.21.15.203:0/2423404830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9fdc0a5e50 0x7f9fdc0b9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fdad9d700 1 --2- 172.21.15.203:0/2423404830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9fdc0bb7d0 0x7f9fdc0c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fd9d9b700 1 -- 172.21.15.203:0/2423404830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9fdc0bb7d0 msgr2=0x7f9fdc0c01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fd9d9b700 1 --2- 172.21.15.203:0/2423404830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9fdc0bb7d0 0x7f9fdc0c01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fd9d9b700 1 -- 172.21.15.203:0/2423404830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9fdc0a5e50 msgr2=0x7f9fdc0b9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fd9d9b700 1 --2- 172.21.15.203:0/2423404830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9fdc0a5e50 0x7f9fdc0b9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.499+0000 7f9fd9d9b700 1 -- 172.21.15.203:0/2423404830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc165050 con 0x7f9fdc0bb3f0 2021-01-06T22:04:04.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.503+0000 7f9fd9d9b700 1 --2- 172.21.15.203:0/2423404830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fdc0bb3f0 0x7f9fdc164b60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f9fc400b190 tx=0x7f9fc4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:04.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.503+0000 7f9fd27fc700 1 -- 172.21.15.203:0/2423404830 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4012c90 con 0x7f9fdc0bb3f0 2021-01-06T22:04:04.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.503+0000 7f9fd27fc700 1 -- 172.21.15.203:0/2423404830 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc4012df0 con 0x7f9fdc0bb3f0 2021-01-06T22:04:04.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.503+0000 7f9fdb59e700 1 -- 172.21.15.203:0/2423404830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fdc0b85a0 con 0x7f9fdc0bb3f0 2021-01-06T22:04:04.508 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.503+0000 7f9fd27fc700 1 -- 172.21.15.203:0/2423404830 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc401c670 con 0x7f9fdc0bb3f0 2021-01-06T22:04:04.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.503+0000 7f9fdb59e700 1 -- 172.21.15.203:0/2423404830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fdc0a1da0 con 0x7f9fdc0bb3f0 2021-01-06T22:04:04.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.503+0000 7f9fdb59e700 1 -- 172.21.15.203:0/2423404830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fdc0450a0 con 0x7f9fdc0bb3f0 2021-01-06T22:04:04.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.503+0000 7f9fd27fc700 1 -- 172.21.15.203:0/2423404830 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f9fc400f070 con 0x7f9fdc0bb3f0 2021-01-06T22:04:04.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.503+0000 7f9fd27fc700 1 --2- 172.21.15.203:0/2423404830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9fb40563d0 0x7f9fb4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.510 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.503+0000 7f9fd27fc700 1 -- 172.21.15.203:0/2423404830 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9fc4070b90 con 0x7f9fdc0bb3f0 2021-01-06T22:04:04.510 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.503+0000 7f9fda59c700 1 --2- 172.21.15.203:0/2423404830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9fb40563d0 0x7f9fb4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.510 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.503+0000 7f9fda59c700 1 --2- 172.21.15.203:0/2423404830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9fb40563d0 0x7f9fb4058890 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f9fcc009cd0 tx=0x7f9fcc0091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:04.513 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.511+0000 7f9fd27fc700 1 -- 172.21.15.203:0/2423404830 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f9fc4046410 con 0x7f9fdc0bb3f0 2021-01-06T22:04:04.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.663+0000 7f9fdb59e700 1 -- 172.21.15.203:0/2423404830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fdc0bc140 con 0x7f9fdc0bb3f0 2021-01-06T22:04:04.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.667+0000 7f9fd27fc700 1 -- 172.21.15.203:0/2423404830 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f9fc4049a30 con 0x7f9fdc0bb3f0 2021-01-06T22:04:04.667 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:04.668 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":23,"fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","created":"2021-01-06T22:03:21.322671+0000","modified":"2021-01-06T22:03:40.744826+0000","last_up_change":"2021-01-06T22:03:31.115611+0000","last_in_change":"2021-01-06T22:03:31.115611+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-06T22:03:25.161465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-06T22:03:35.070364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-01-06T22:03:37.862354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-01-06T22:03:38.738416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e0b35c09-147c-4ccc-b566-c48b81e25412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":18347},{"type":"v1","addr":"172.21.15.203:6815","nonce":18347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":18347},{"type":"v1","addr":"172.21.15.203:6818","nonce":18347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":18347},{"type":"v1","addr":"172.21.15.203:6831","nonce":18347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6821","nonce":18347},{"type":"v1","addr":"172.21.15.203:6827","nonce":18347}]},"public_addr":"172.21.15.203:6815/18347","cluster_addr":"172.21.15.203:6818/18347","heartbeat_back_addr":"172.21.15.203:6831/18347","heartbeat_front_addr":"172.21.15.203:6827/18347","state":["exists","up"]},{"osd":1,"uuid":"8add59a0-2d58-4c1b-a8d1-62049ee9d85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":18337},{"type":"v1","addr":"172.21.15.203:6803","nonce":18337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":18337},{"type":"v1","addr":"172.21.15.203:6805","nonce":18337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":18337},{"type":"v1","addr":"172.21.15.203:6809","nonce":18337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":18337},{"type":"v1","addr":"172.21.15.203:6807","nonce":18337}]},"public_addr":"172.21.15.203:6803/18337","cluster_addr":"172.21.15.203:6805/18337","heartbeat_back_addr":"172.21.15.203:6809/18337","heartbeat_front_addr":"172.21.15.203:6807/18337","state":["exists","up"]},{"osd":2,"uuid":"b294a411-c7c9-477e-8eae-3b7bd0563ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":18354},{"type":"v1","addr":"172.21.15.203:6811","nonce":18354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":18354},{"type":"v1","addr":"172.21.15.203:6819","nonce":18354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6825","nonce":18354},{"type":"v1","addr":"172.21.15.203:6828","nonce":18354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6823","nonce":18354},{"type":"v1","addr":"172.21.15.203:6824","nonce":18354}]},"public_addr":"172.21.15.203:6811/18354","cluster_addr":"172.21.15.203:6819/18354","heartbeat_back_addr":"172.21.15.203:6828/18354","heartbeat_front_addr":"172.21.15.203:6824/18354","state":["exists","up"]},{"osd":3,"uuid":"5878a8f7-1a42-41c3-9810-0ac41a54dd6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":18346},{"type":"v1","addr":"172.21.15.203:6817","nonce":18346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":18346},{"type":"v1","addr":"172.21.15.203:6822","nonce":18346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":18346},{"type":"v1","addr":"172.21.15.203:6833","nonce":18346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":18346},{"type":"v1","addr":"172.21.15.203:6829","nonce":18346}]},"public_addr":"172.21.15.203:6817/18346","cluster_addr":"172.21.15.203:6822/18346","heartbeat_back_addr":"172.21.15.203:6833/18346","heartbeat_front_addr":"172.21.15.203:6829/18346","state":["exists","up"]},{"osd":4,"uuid":"f641a345-81e4-4400-a44c-d2f82fef7f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":18370},{"type":"v1","addr":"172.21.15.91:6809","nonce":18370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":18370},{"type":"v1","addr":"172.21.15.91:6811","nonce":18370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":18370},{"type":"v1","addr":"172.21.15.91:6816","nonce":18370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":18370},{"type":"v1","addr":"172.21.15.91:6813","nonce":18370}]},"public_addr":"172.21.15.91:6809/18370","cluster_addr":"172.21.15.91:6811/18370","heartbeat_back_addr":"172.21.15.91:6816/18370","heartbeat_front_addr":"172.21.15.91:6813/18370","state":["exists","up"]},{"osd":5,"uuid":"c1ff150d-68c6-4e6b-abd4-76af27bb2103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":18372},{"type":"v1","addr":"172.21.15.91:6801","nonce":18372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":18372},{"type":"v1","addr":"172.21.15.91:6803","nonce":18372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":18372},{"type":"v1","addr":"172.21.15.91:6807","nonce":18372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":18372},{"type":"v1","addr":"172.21.15.91:6805","nonce":18372}]},"public_addr":"172.21.15.91:6801/18372","cluster_addr":"172.21.15.91:6803/18372","heartbeat_back_addr":"172.21.15.91:6807/18372","heartbeat_front_addr":"172.21.15.91:6805/18372","state":["exists","up"]},{"osd":6,"uuid":"35a58e1a-0a0a-4f02-a477-adc029038d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6815","nonce":18371},{"type":"v1","addr":"172.21.15.91:6817","nonce":18371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":18371},{"type":"v1","addr":"172.21.15.91:6819","nonce":18371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":18371},{"type":"v1","addr":"172.21.15.91:6823","nonce":18371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":18371},{"type":"v1","addr":"172.21.15.91:6821","nonce":18371}]},"public_addr":"172.21.15.91:6817/18371","cluster_addr":"172.21.15.91:6819/18371","heartbeat_back_addr":"172.21.15.91:6823/18371","heartbeat_front_addr":"172.21.15.91:6821/18371","state":["exists","up"]},{"osd":7,"uuid":"809a2d89-4303-482a-8558-aef907814234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":18373},{"type":"v1","addr":"172.21.15.91:6825","nonce":18373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":18373},{"type":"v1","addr":"172.21.15.91:6827","nonce":18373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":18373},{"type":"v1","addr":"172.21.15.91:6831","nonce":18373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":18373},{"type":"v1","addr":"172.21.15.91:6829","nonce":18373}]},"public_addr":"172.21.15.91:6825/18373","cluster_addr":"172.21.15.91:6827/18373","heartbeat_back_addr":"172.21.15.91:6831/18373","heartbeat_front_addr":"172.21.15.91:6829/18373","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.875052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.103604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.179225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.158512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.206073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.894840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.229708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.248192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-06T22:04:04.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.667+0000 7f9fdb59e700 1 -- 172.21.15.203:0/2423404830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9fb40563d0 msgr2=0x7f9fb4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.667+0000 7f9fdb59e700 1 --2- 172.21.15.203:0/2423404830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9fb40563d0 0x7f9fb4058890 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f9fcc009cd0 tx=0x7f9fcc0091d0).stop 2021-01-06T22:04:04.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.667+0000 7f9fdb59e700 1 -- 172.21.15.203:0/2423404830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fdc0bb3f0 msgr2=0x7f9fdc164b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.667+0000 7f9fdb59e700 1 --2- 172.21.15.203:0/2423404830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fdc0bb3f0 0x7f9fdc164b60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f9fc400b190 tx=0x7f9fc4016000).stop 2021-01-06T22:04:04.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.667+0000 7f9fdb59e700 1 -- 172.21.15.203:0/2423404830 shutdown_connections 2021-01-06T22:04:04.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.667+0000 7f9fdb59e700 1 --2- 172.21.15.203:0/2423404830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9fb40563d0 0x7f9fb4058890 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.667+0000 7f9fdb59e700 1 --2- 172.21.15.203:0/2423404830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9fdc0bb7d0 0x7f9fdc0c01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.667+0000 7f9fdb59e700 1 --2- 172.21.15.203:0/2423404830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9fdc0a5e50 0x7f9fdc0b9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.667+0000 7f9fdb59e700 1 --2- 172.21.15.203:0/2423404830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fdc0bb3f0 0x7f9fdc164b60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.667+0000 7f9fdb59e700 1 -- 172.21.15.203:0/2423404830 >> 172.21.15.203:0/2423404830 conn(0x7f9fdc061530 msgr2=0x7f9fdc0bf200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:04.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.667+0000 7f9fdb59e700 1 -- 172.21.15.203:0/2423404830 shutdown_connections 2021-01-06T22:04:04.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.667+0000 7f9fdb59e700 1 -- 172.21.15.203:0/2423404830 wait complete. 2021-01-06T22:04:04.682 INFO:tasks.mds_thrash.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-01-06T22:03:25.161465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-01-06T22:03:35.070364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-01-06T22:03:37.862354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-01-06T22:03:38.738416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-01-06T22:04:04.682 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-01-06T22:04:04.812 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.811+0000 7ff431a4b700 1 Processor -- start 2021-01-06T22:04:04.813 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.811+0000 7ff431a4b700 1 -- start start 2021-01-06T22:04:04.814 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.811+0000 7ff431a4b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff42c066ad0 0x7ff42c060b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.814 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.811+0000 7ff431a4b700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42c067a50 con 0x7ff42c0baf40 2021-01-06T22:04:04.814 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.811+0000 7ff431a4b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42c0c02f0 con 0x7ff42c0bab60 2021-01-06T22:04:04.814 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.811+0000 7ff431a4b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff42c0c0590 con 0x7ff42c066ad0 2021-01-06T22:04:04.815 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.811+0000 7ff43124a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff42c066ad0 0x7ff42c060b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.815 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.811+0000 7ff430a49700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7ff42c0bab60 0x7ff42c0b7460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47976/0 (socket says 172.21.15.203:47976) 2021-01-06T22:04:04.815 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.811+0000 7ff430a49700 1 -- 172.21.15.203:0/2369541951 learned_addr learned my addr 172.21.15.203:0/2369541951 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:04:04.816 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.815+0000 7ff43124a700 1 -- 172.21.15.203:0/2369541951 >> v1:172.21.15.203:6789/0 conn(0x7ff42c0bab60 legacy=0x7ff42c0b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.817 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.815+0000 7ff43124a700 1 -- 172.21.15.203:0/2369541951 >> v1:172.21.15.91:6789/0 conn(0x7ff42c0baf40 legacy=0x7ff42c0b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.817 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.815+0000 7ff43124a700 1 -- 172.21.15.203:0/2369541951 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42c0c0830 con 0x7ff42c066ad0 2021-01-06T22:04:04.817 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.815+0000 7ff43124a700 1 --2- 172.21.15.203:0/2369541951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff42c066ad0 0x7ff42c060b70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7ff41400b980 tx=0x7ff414006de0).ready entity=mon.1 client_cookie=c594f62c570a84a server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:04.817 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.815+0000 7ff42b7fe700 1 -- 172.21.15.203:0/2369541951 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414009840 con 0x7ff42c066ad0 2021-01-06T22:04:04.817 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.815+0000 7ff42b7fe700 1 -- 172.21.15.203:0/2369541951 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4140099a0 con 0x7ff42c066ad0 2021-01-06T22:04:04.818 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.815+0000 7ff42b7fe700 1 -- 172.21.15.203:0/2369541951 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414009d20 con 0x7ff42c066ad0 2021-01-06T22:04:04.818 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.815+0000 7ff431a4b700 1 -- 172.21.15.203:0/2369541951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff42c066ad0 msgr2=0x7ff42c060b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.818 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.815+0000 7ff431a4b700 1 --2- 172.21.15.203:0/2369541951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff42c066ad0 0x7ff42c060b70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7ff41400b980 tx=0x7ff414006de0).stop 2021-01-06T22:04:04.818 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.815+0000 7ff431a4b700 1 -- 172.21.15.203:0/2369541951 shutdown_connections 2021-01-06T22:04:04.819 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.815+0000 7ff431a4b700 1 --2- 172.21.15.203:0/2369541951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff42c066ad0 0x7ff42c060b70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.819 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.815+0000 7ff431a4b700 1 -- 172.21.15.203:0/2369541951 >> 172.21.15.203:0/2369541951 conn(0x7ff42c0626e0 msgr2=0x7ff42c062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:04.819 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.815+0000 7ff431a4b700 1 -- 172.21.15.203:0/2369541951 shutdown_connections 2021-01-06T22:04:04.819 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.815+0000 7ff431a4b700 1 -- 172.21.15.203:0/2369541951 wait complete. 2021-01-06T22:04:04.821 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.815+0000 7ff431a4b700 1 Processor -- start 2021-01-06T22:04:04.821 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.815+0000 7ff431a4b700 1 -- start start 2021-01-06T22:04:04.822 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff431a4b700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff42c066ad0 0x7ff42c165a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.822 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff431a4b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff42c0bab60 0x7ff42c0b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.822 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff431a4b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff42c0baf40 0x7ff42c161980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.822 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff431a4b700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c161e70 con 0x7ff42c066ad0 2021-01-06T22:04:04.822 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff431a4b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0b5580 con 0x7ff42c0bab60 2021-01-06T22:04:04.823 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff431a4b700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0b5820 con 0x7ff42c0baf40 2021-01-06T22:04:04.823 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff42bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff42c0bab60 0x7ff42c0b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.823 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff42bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff42c0bab60 0x7ff42c0b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49390/0 (socket says 172.21.15.203:49390) 2021-01-06T22:04:04.823 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff42bfff700 1 -- 172.21.15.203:0/3782852853 learned_addr learned my addr 172.21.15.203:0/3782852853 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:04.823 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff430a49700 1 --2- 172.21.15.203:0/3782852853 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff42c066ad0 0x7ff42c165a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.824 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff43124a700 1 --2- 172.21.15.203:0/3782852853 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff42c0baf40 0x7ff42c161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.824 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff42bfff700 1 -- 172.21.15.203:0/3782852853 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff42c0baf40 msgr2=0x7ff42c161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.824 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff42bfff700 1 --2- 172.21.15.203:0/3782852853 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff42c0baf40 0x7ff42c161980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.824 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff42bfff700 1 -- 172.21.15.203:0/3782852853 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff42c066ad0 msgr2=0x7ff42c165a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.824 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff42bfff700 1 --2- 172.21.15.203:0/3782852853 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff42c066ad0 0x7ff42c165a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.825 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff42bfff700 1 -- 172.21.15.203:0/3782852853 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42c165050 con 0x7ff42c0bab60 2021-01-06T22:04:04.825 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff42bfff700 1 --2- 172.21.15.203:0/3782852853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff42c0bab60 0x7ff42c0b3a30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7ff41c00ca90 tx=0x7ff41c0013c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:04.825 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff428ff9700 1 -- 172.21.15.203:0/3782852853 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c006860 con 0x7ff42c0bab60 2021-01-06T22:04:04.825 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff428ff9700 1 -- 172.21.15.203:0/3782852853 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41c0069c0 con 0x7ff42c0bab60 2021-01-06T22:04:04.825 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff431a4b700 1 -- 172.21.15.203:0/3782852853 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42c1652f0 con 0x7ff42c0bab60 2021-01-06T22:04:04.826 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff428ff9700 1 -- 172.21.15.203:0/3782852853 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c006d40 con 0x7ff42c0bab60 2021-01-06T22:04:04.826 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff431a4b700 1 -- 172.21.15.203:0/3782852853 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42c0b5240 con 0x7ff42c0bab60 2021-01-06T22:04:04.827 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.819+0000 7ff431a4b700 1 -- 172.21.15.203:0/3782852853 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42c0450a0 con 0x7ff42c0bab60 2021-01-06T22:04:04.827 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.823+0000 7ff428ff9700 1 -- 172.21.15.203:0/3782852853 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff41c010070 con 0x7ff42c0bab60 2021-01-06T22:04:04.827 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.823+0000 7ff428ff9700 1 --2- 172.21.15.203:0/3782852853 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff4040563d0 0x7ff404058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:04.827 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.823+0000 7ff428ff9700 1 -- 172.21.15.203:0/3782852853 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff41c06ec30 con 0x7ff42c0bab60 2021-01-06T22:04:04.828 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.823+0000 7ff430a49700 1 --2- 172.21.15.203:0/3782852853 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff4040563d0 0x7ff404058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:04.828 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.823+0000 7ff430a49700 1 --2- 172.21.15.203:0/3782852853 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff4040563d0 0x7ff404058890 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7ff4240045d0 tx=0x7ff424007b70).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:04.830 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.827+0000 7ff428ff9700 1 -- 172.21.15.203:0/3782852853 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff41c0443a0 con 0x7ff42c0bab60 2021-01-06T22:04:04.983 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.983+0000 7ff431a4b700 1 -- 172.21.15.203:0/3782852853 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff42c05ca20 con 0x7ff42c0bab60 2021-01-06T22:04:04.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.983+0000 7ff428ff9700 1 -- 172.21.15.203:0/3782852853 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff41c0479c0 con 0x7ff42c0bab60 2021-01-06T22:04:04.985 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 1 2021-01-06T22:04:04.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.983+0000 7ff431a4b700 1 -- 172.21.15.203:0/3782852853 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff4040563d0 msgr2=0x7ff404058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.983+0000 7ff431a4b700 1 --2- 172.21.15.203:0/3782852853 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff4040563d0 0x7ff404058890 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7ff4240045d0 tx=0x7ff424007b70).stop 2021-01-06T22:04:04.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.983+0000 7ff431a4b700 1 -- 172.21.15.203:0/3782852853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff42c0bab60 msgr2=0x7ff42c0b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:04.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.983+0000 7ff431a4b700 1 --2- 172.21.15.203:0/3782852853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff42c0bab60 0x7ff42c0b3a30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7ff41c00ca90 tx=0x7ff41c0013c0).stop 2021-01-06T22:04:04.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.983+0000 7ff431a4b700 1 -- 172.21.15.203:0/3782852853 shutdown_connections 2021-01-06T22:04:04.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.983+0000 7ff431a4b700 1 --2- 172.21.15.203:0/3782852853 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff4040563d0 0x7ff404058890 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.983+0000 7ff431a4b700 1 --2- 172.21.15.203:0/3782852853 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff42c0baf40 0x7ff42c161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.983+0000 7ff431a4b700 1 --2- 172.21.15.203:0/3782852853 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff42c066ad0 0x7ff42c165a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.983+0000 7ff431a4b700 1 --2- 172.21.15.203:0/3782852853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff42c0bab60 0x7ff42c0b3a30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:04.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.983+0000 7ff431a4b700 1 -- 172.21.15.203:0/3782852853 >> 172.21.15.203:0/3782852853 conn(0x7ff42c0626e0 msgr2=0x7ff42c0b8310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:04.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.983+0000 7ff431a4b700 1 -- 172.21.15.203:0/3782852853 shutdown_connections 2021-01-06T22:04:04.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:04.983+0000 7ff431a4b700 1 -- 172.21.15.203:0/3782852853 wait complete. 2021-01-06T22:04:04.999 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-01-06T22:04:05.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c9524d700 1 Processor -- start 2021-01-06T22:04:05.128 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c9524d700 1 -- start start 2021-01-06T22:04:05.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c9524d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c900b97a0 0x7f8c900b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:05.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c9524d700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c900be410 con 0x7f8c900bd420 2021-01-06T22:04:05.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c9524d700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c90066510 con 0x7f8c900bed80 2021-01-06T22:04:05.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c9524d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c900667b0 con 0x7f8c900b97a0 2021-01-06T22:04:05.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c94a4c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f8c900bed80 0x7f8c9014c030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:47992/0 (socket says 172.21.15.203:47992) 2021-01-06T22:04:05.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c94a4c700 1 -- 172.21.15.203:0/3480773845 learned_addr learned my addr 172.21.15.203:0/3480773845 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:04:05.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c8ffff700 1 --2- 172.21.15.203:0/3480773845 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c900b97a0 0x7f8c900b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:05.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c8effd700 1 -- 172.21.15.203:0/3480773845 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724825462 0 0) 0x7f8c90066510 con 0x7f8c900bed80 2021-01-06T22:04:05.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c8effd700 1 -- 172.21.15.203:0/3480773845 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c70003230 con 0x7f8c900bed80 2021-01-06T22:04:05.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c8effd700 1 -- 172.21.15.203:0/3480773845 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228887706 0 0) 0x7f8c900be410 con 0x7f8c900bd420 2021-01-06T22:04:05.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c8effd700 1 -- 172.21.15.203:0/3480773845 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c90066510 con 0x7f8c900bd420 2021-01-06T22:04:05.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c8ffff700 1 -- 172.21.15.203:0/3480773845 >> v1:172.21.15.203:6789/0 conn(0x7f8c900bed80 legacy=0x7f8c9014c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c8ffff700 1 -- 172.21.15.203:0/3480773845 >> v1:172.21.15.91:6789/0 conn(0x7f8c900bd420 legacy=0x7f8c90065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c8ffff700 1 -- 172.21.15.203:0/3480773845 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c90066a50 con 0x7f8c900b97a0 2021-01-06T22:04:05.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c8ffff700 1 --2- 172.21.15.203:0/3480773845 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c900b97a0 0x7f8c900b9b80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f8c88009cc0 tx=0x7f8c880091d0).ready entity=mon.2 client_cookie=125870918f2b5307 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:05.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c8effd700 1 -- 172.21.15.203:0/3480773845 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8800bab0 con 0x7f8c900b97a0 2021-01-06T22:04:05.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.127+0000 7f8c8effd700 1 -- 172.21.15.203:0/3480773845 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8800bc10 con 0x7f8c900b97a0 2021-01-06T22:04:05.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.131+0000 7f8c8effd700 1 -- 172.21.15.203:0/3480773845 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88011400 con 0x7f8c900b97a0 2021-01-06T22:04:05.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.131+0000 7f8c9524d700 1 -- 172.21.15.203:0/3480773845 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c900b97a0 msgr2=0x7f8c900b9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.131+0000 7f8c9524d700 1 --2- 172.21.15.203:0/3480773845 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c900b97a0 0x7f8c900b9b80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f8c88009cc0 tx=0x7f8c880091d0).stop 2021-01-06T22:04:05.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.131+0000 7f8c9524d700 1 -- 172.21.15.203:0/3480773845 shutdown_connections 2021-01-06T22:04:05.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.131+0000 7f8c9524d700 1 --2- 172.21.15.203:0/3480773845 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c900b97a0 0x7f8c900b9b80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.131+0000 7f8c9524d700 1 -- 172.21.15.203:0/3480773845 >> 172.21.15.203:0/3480773845 conn(0x7f8c900648f0 msgr2=0x7f8c90064cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:05.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.131+0000 7f8c9524d700 1 -- 172.21.15.203:0/3480773845 shutdown_connections 2021-01-06T22:04:05.136 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.131+0000 7f8c9524d700 1 -- 172.21.15.203:0/3480773845 wait complete. 2021-01-06T22:04:05.137 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.131+0000 7f8c9524d700 1 Processor -- start 2021-01-06T22:04:05.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.131+0000 7f8c9524d700 1 -- start start 2021-01-06T22:04:05.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.131+0000 7f8c9524d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c900b97a0 0x7f8c9015e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:05.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.131+0000 7f8c9524d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c900bd420 0x7f8c900a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:05.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c9524d700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c900bed80 0x7f8c9014b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:05.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c9524d700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c900b7e90 con 0x7f8c900bed80 2021-01-06T22:04:05.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c9524d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c900b8130 con 0x7f8c900bd420 2021-01-06T22:04:05.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c9524d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c900b83d0 con 0x7f8c900b97a0 2021-01-06T22:04:05.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c8f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c900bd420 0x7f8c900a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:05.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c8f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c900bd420 0x7f8c900a3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49404/0 (socket says 172.21.15.203:49404) 2021-01-06T22:04:05.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c8f7fe700 1 -- 172.21.15.203:0/3472155414 learned_addr learned my addr 172.21.15.203:0/3472155414 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:05.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c8ffff700 1 --2- 172.21.15.203:0/3472155414 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c900b97a0 0x7f8c9015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:05.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c94a4c700 1 --2- 172.21.15.203:0/3472155414 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c900bed80 0x7f8c9014b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:05.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c8f7fe700 1 -- 172.21.15.203:0/3472155414 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c900b97a0 msgr2=0x7f8c9015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c8f7fe700 1 --2- 172.21.15.203:0/3472155414 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c900b97a0 0x7f8c9015e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c8f7fe700 1 -- 172.21.15.203:0/3472155414 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c900bed80 msgr2=0x7f8c9014b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c8f7fe700 1 --2- 172.21.15.203:0/3472155414 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c900bed80 0x7f8c9014b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c8f7fe700 1 -- 172.21.15.203:0/3472155414 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c900acee0 con 0x7f8c900bd420 2021-01-06T22:04:05.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c8ffff700 1 --2- 172.21.15.203:0/3472155414 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c900b97a0 0x7f8c9015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:04:05.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c8f7fe700 1 --2- 172.21.15.203:0/3472155414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c900bd420 0x7f8c900a3970 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f8c7800c6e0 tx=0x7f8c78007aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c87fff700 1 -- 172.21.15.203:0/3472155414 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800e560 con 0x7f8c900bd420 2021-01-06T22:04:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c87fff700 1 -- 172.21.15.203:0/3472155414 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7800e6c0 con 0x7f8c900bd420 2021-01-06T22:04:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c9524d700 1 -- 172.21.15.203:0/3472155414 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c900ad180 con 0x7f8c900bd420 2021-01-06T22:04:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c87fff700 1 -- 172.21.15.203:0/3472155414 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800ea60 con 0x7f8c900bd420 2021-01-06T22:04:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c9524d700 1 -- 172.21.15.203:0/3472155414 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c900a7440 con 0x7f8c900bd420 2021-01-06T22:04:05.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c87fff700 1 -- 172.21.15.203:0/3472155414 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f8c78010070 con 0x7f8c900bd420 2021-01-06T22:04:05.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.135+0000 7f8c9524d700 1 -- 172.21.15.203:0/3472155414 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c900450a0 con 0x7f8c900bd420 2021-01-06T22:04:05.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.139+0000 7f8c87fff700 1 --2- 172.21.15.203:0/3472155414 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8c740563d0 0x7f8c74058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:05.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.139+0000 7f8c8ffff700 1 --2- 172.21.15.203:0/3472155414 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8c740563d0 0x7f8c74058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:05.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.139+0000 7f8c87fff700 1 -- 172.21.15.203:0/3472155414 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8c7806f110 con 0x7f8c900bd420 2021-01-06T22:04:05.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.139+0000 7f8c8ffff700 1 --2- 172.21.15.203:0/3472155414 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8c740563d0 0x7f8c74058890 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f8c88009fd0 tx=0x7f8c88017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:05.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.143+0000 7f8c87fff700 1 -- 172.21.15.203:0/3472155414 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f8c78044990 con 0x7f8c900bd420 2021-01-06T22:04:05.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.295+0000 7f8c9524d700 1 -- 172.21.15.203:0/3472155414 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8c9005ca60 con 0x7f8c900bd420 2021-01-06T22:04:05.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.295+0000 7f8c87fff700 1 -- 172.21.15.203:0/3472155414 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8c78047fb0 con 0x7f8c900bd420 2021-01-06T22:04:05.297 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:04:05.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.299+0000 7f8c9524d700 1 -- 172.21.15.203:0/3472155414 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8c740563d0 msgr2=0x7f8c74058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.299+0000 7f8c9524d700 1 --2- 172.21.15.203:0/3472155414 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8c740563d0 0x7f8c74058890 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f8c88009fd0 tx=0x7f8c88017000).stop 2021-01-06T22:04:05.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.299+0000 7f8c9524d700 1 -- 172.21.15.203:0/3472155414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c900bd420 msgr2=0x7f8c900a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.299+0000 7f8c9524d700 1 --2- 172.21.15.203:0/3472155414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c900bd420 0x7f8c900a3970 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f8c7800c6e0 tx=0x7f8c78007aa0).stop 2021-01-06T22:04:05.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.299+0000 7f8c9524d700 1 -- 172.21.15.203:0/3472155414 shutdown_connections 2021-01-06T22:04:05.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.299+0000 7f8c9524d700 1 --2- 172.21.15.203:0/3472155414 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8c740563d0 0x7f8c74058890 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.299+0000 7f8c9524d700 1 --2- 172.21.15.203:0/3472155414 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c900bed80 0x7f8c9014b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.299+0000 7f8c9524d700 1 --2- 172.21.15.203:0/3472155414 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c900b97a0 0x7f8c9015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.299+0000 7f8c9524d700 1 --2- 172.21.15.203:0/3472155414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c900bd420 0x7f8c900a3970 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.299+0000 7f8c9524d700 1 -- 172.21.15.203:0/3472155414 >> 172.21.15.203:0/3472155414 conn(0x7f8c900648f0 msgr2=0x7f8c900bfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:05.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.299+0000 7f8c9524d700 1 -- 172.21.15.203:0/3472155414 shutdown_connections 2021-01-06T22:04:05.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.299+0000 7f8c9524d700 1 -- 172.21.15.203:0/3472155414 wait complete. 2021-01-06T22:04:05.312 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-01-06T22:04:05.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.439+0000 7f3c161a1700 1 Processor -- start 2021-01-06T22:04:05.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.439+0000 7f3c161a1700 1 -- start start 2021-01-06T22:04:05.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.439+0000 7f3c161a1700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c100baf40 0x7f3c10060b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:05.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.439+0000 7f3c161a1700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c10067a50 con 0x7f3c10066ad0 2021-01-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.439+0000 7f3c161a1700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c100c02f0 con 0x7f3c100bab60 2021-01-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.439+0000 7f3c161a1700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c100c0590 con 0x7f3c100baf40 2021-01-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.439+0000 7f3c1499e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c100baf40 0x7f3c10060b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c1499e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c100baf40 0x7f3c10060b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49414/0 (socket says 172.21.15.203:49414) 2021-01-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c1519f700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f3c100bab60 0x7f3c100b7460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48004/0 (socket says 172.21.15.203:48004) 2021-01-06T22:04:05.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c1499e700 1 -- 172.21.15.203:0/2073391818 learned_addr learned my addr 172.21.15.203:0/2073391818 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:05.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c1499e700 1 -- 172.21.15.203:0/2073391818 >> v1:172.21.15.203:6789/0 conn(0x7f3c100bab60 legacy=0x7f3c100b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c1499e700 1 -- 172.21.15.203:0/2073391818 >> v1:172.21.15.91:6789/0 conn(0x7f3c10066ad0 legacy=0x7f3c100b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:05.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c1499e700 1 -- 172.21.15.203:0/2073391818 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c100c0830 con 0x7f3c100baf40 2021-01-06T22:04:05.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c1499e700 1 --2- 172.21.15.203:0/2073391818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c100baf40 0x7f3c10060b70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f3bf800b9e0 tx=0x7f3bf8006ea0).ready entity=mon.1 client_cookie=8ca234f1a7fd3e1b server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:05.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c0ffff700 1 -- 172.21.15.203:0/2073391818 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8009920 con 0x7f3c100baf40 2021-01-06T22:04:05.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c0ffff700 1 -- 172.21.15.203:0/2073391818 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf8009a80 con 0x7f3c100baf40 2021-01-06T22:04:05.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c0ffff700 1 -- 172.21.15.203:0/2073391818 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8009e00 con 0x7f3c100baf40 2021-01-06T22:04:05.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c161a1700 1 -- 172.21.15.203:0/2073391818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c100baf40 msgr2=0x7f3c10060b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c161a1700 1 --2- 172.21.15.203:0/2073391818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c100baf40 0x7f3c10060b70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f3bf800b9e0 tx=0x7f3bf8006ea0).stop 2021-01-06T22:04:05.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c161a1700 1 -- 172.21.15.203:0/2073391818 shutdown_connections 2021-01-06T22:04:05.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c161a1700 1 --2- 172.21.15.203:0/2073391818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c100baf40 0x7f3c10060b70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c161a1700 1 -- 172.21.15.203:0/2073391818 >> 172.21.15.203:0/2073391818 conn(0x7f3c100626e0 msgr2=0x7f3c10062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:05.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c161a1700 1 -- 172.21.15.203:0/2073391818 shutdown_connections 2021-01-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c161a1700 1 -- 172.21.15.203:0/2073391818 wait complete. 2021-01-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c161a1700 1 Processor -- start 2021-01-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c161a1700 1 -- start start 2021-01-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c161a1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c10066ad0 0x7f3c10161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c161a1700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c100bab60 0x7f3c1014ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:05.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.443+0000 7f3c161a1700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c100baf40 0x7f3c100bf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:05.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c161a1700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1015fad0 con 0x7f3c100baf40 2021-01-06T22:04:05.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c161a1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c10165040 con 0x7f3c10066ad0 2021-01-06T22:04:05.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c161a1700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c101620d0 con 0x7f3c100bab60 2021-01-06T22:04:05.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c1519f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c10066ad0 0x7f3c10161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:05.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c1519f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c10066ad0 0x7f3c10161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49416/0 (socket says 172.21.15.203:49416) 2021-01-06T22:04:05.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c1519f700 1 -- 172.21.15.203:0/4044400376 learned_addr learned my addr 172.21.15.203:0/4044400376 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:05.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c1499e700 1 --2- 172.21.15.203:0/4044400376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c100bab60 0x7f3c1014ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:05.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c159a0700 1 --2- 172.21.15.203:0/4044400376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c100baf40 0x7f3c100bf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:05.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c1519f700 1 -- 172.21.15.203:0/4044400376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c100bab60 msgr2=0x7f3c1014ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c1519f700 1 --2- 172.21.15.203:0/4044400376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c100bab60 0x7f3c1014ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c1519f700 1 -- 172.21.15.203:0/4044400376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c100baf40 msgr2=0x7f3c100bf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c1519f700 1 --2- 172.21.15.203:0/4044400376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c100baf40 0x7f3c100bf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c1519f700 1 -- 172.21.15.203:0/4044400376 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c10162370 con 0x7f3c10066ad0 2021-01-06T22:04:05.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c1519f700 1 --2- 172.21.15.203:0/4044400376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c10066ad0 0x7f3c10161aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f3c0800ca60 tx=0x7f3c08001460).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:05.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c0d7fa700 1 -- 172.21.15.203:0/4044400376 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08005ab0 con 0x7f3c10066ad0 2021-01-06T22:04:05.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c0d7fa700 1 -- 172.21.15.203:0/4044400376 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c08005c10 con 0x7f3c10066ad0 2021-01-06T22:04:05.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c161a1700 1 -- 172.21.15.203:0/4044400376 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c10162610 con 0x7f3c10066ad0 2021-01-06T22:04:05.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c0d7fa700 1 -- 172.21.15.203:0/4044400376 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08007110 con 0x7f3c10066ad0 2021-01-06T22:04:05.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c161a1700 1 -- 172.21.15.203:0/4044400376 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c10165270 con 0x7f3c10066ad0 2021-01-06T22:04:05.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c161a1700 1 -- 172.21.15.203:0/4044400376 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c10060f00 con 0x7f3c10066ad0 2021-01-06T22:04:05.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c0d7fa700 1 -- 172.21.15.203:0/4044400376 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f3c08010030 con 0x7f3c10066ad0 2021-01-06T22:04:05.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c0d7fa700 1 --2- 172.21.15.203:0/4044400376 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3bf40563d0 0x7f3bf4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:05.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c0d7fa700 1 -- 172.21.15.203:0/4044400376 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3c0806f4f0 con 0x7f3c10066ad0 2021-01-06T22:04:05.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.447+0000 7f3c1499e700 1 --2- 172.21.15.203:0/4044400376 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3bf40563d0 0x7f3bf4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:05.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.451+0000 7f3c1499e700 1 --2- 172.21.15.203:0/4044400376 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3bf40563d0 0x7f3bf4058890 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f3bf8006ee0 tx=0x7f3bf8016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:05.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.455+0000 7f3c0d7fa700 1 -- 172.21.15.203:0/4044400376 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f3c10060f00 con 0x7f3c10066ad0 2021-01-06T22:04:05.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.603+0000 7f3c161a1700 1 -- 172.21.15.203:0/4044400376 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3c10060f00 con 0x7f3c10066ad0 2021-01-06T22:04:05.607 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.607+0000 7f3c0d7fa700 1 -- 172.21.15.203:0/4044400376 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3c10060f00 con 0x7f3c10066ad0 2021-01-06T22:04:05.607 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:04:05.609 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.607+0000 7f3c161a1700 1 -- 172.21.15.203:0/4044400376 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3bf40563d0 msgr2=0x7f3bf4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.610 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.607+0000 7f3c161a1700 1 --2- 172.21.15.203:0/4044400376 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3bf40563d0 0x7f3bf4058890 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f3bf8006ee0 tx=0x7f3bf8016000).stop 2021-01-06T22:04:05.610 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.607+0000 7f3c161a1700 1 -- 172.21.15.203:0/4044400376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c10066ad0 msgr2=0x7f3c10161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.610 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.607+0000 7f3c161a1700 1 --2- 172.21.15.203:0/4044400376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c10066ad0 0x7f3c10161aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f3c0800ca60 tx=0x7f3c08001460).stop 2021-01-06T22:04:05.610 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.607+0000 7f3c161a1700 1 -- 172.21.15.203:0/4044400376 shutdown_connections 2021-01-06T22:04:05.610 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.607+0000 7f3c161a1700 1 --2- 172.21.15.203:0/4044400376 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3bf40563d0 0x7f3bf4058890 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.611 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.607+0000 7f3c161a1700 1 --2- 172.21.15.203:0/4044400376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c100baf40 0x7f3c100bf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.611 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.607+0000 7f3c161a1700 1 --2- 172.21.15.203:0/4044400376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c10066ad0 0x7f3c10161aa0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.611 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.607+0000 7f3c161a1700 1 --2- 172.21.15.203:0/4044400376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c100bab60 0x7f3c1014ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.611 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.607+0000 7f3c161a1700 1 -- 172.21.15.203:0/4044400376 >> 172.21.15.203:0/4044400376 conn(0x7f3c100626e0 msgr2=0x7f3c100b7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:05.611 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.607+0000 7f3c161a1700 1 -- 172.21.15.203:0/4044400376 shutdown_connections 2021-01-06T22:04:05.612 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.607+0000 7f3c161a1700 1 -- 172.21.15.203:0/4044400376 wait complete. 2021-01-06T22:04:05.623 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-01-06T22:04:05.748 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.747+0000 7f59e5768700 1 Processor -- start 2021-01-06T22:04:05.749 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.747+0000 7f59e5768700 1 -- start start 2021-01-06T22:04:05.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.747+0000 7f59e5768700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f59e0063010 0x7f59e0151160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:05.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.747+0000 7f59e5768700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f59e00be1b0 0x7f59e00be620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:05.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.747+0000 7f59e5768700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e00602b0 con 0x7f59e00633f0 2021-01-06T22:04:05.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.747+0000 7f59e5768700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e014b760 con 0x7f59e00be1b0 2021-01-06T22:04:05.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.747+0000 7f59e5768700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e00bdb20 con 0x7f59e0063010 2021-01-06T22:04:05.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59dffff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f59e0063010 0x7f59e0151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:05.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59dffff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f59e0063010 0x7f59e0151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49424/0 (socket says 172.21.15.203:49424) 2021-01-06T22:04:05.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59dffff700 1 -- 172.21.15.203:0/2099381558 learned_addr learned my addr 172.21.15.203:0/2099381558 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:05.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59e4f67700 1 --2- 172.21.15.203:0/2099381558 >> v2:172.21.15.91:3300/0 conn(0x7f59e00be1b0 0x7f59e00be620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:05.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59dffff700 1 -- 172.21.15.203:0/2099381558 >> v2:172.21.15.91:3300/0 conn(0x7f59e00be1b0 msgr2=0x7f59e00be620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:05.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59dffff700 1 --2- 172.21.15.203:0/2099381558 >> v2:172.21.15.91:3300/0 conn(0x7f59e00be1b0 0x7f59e00be620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59dffff700 1 -- 172.21.15.203:0/2099381558 >> v1:172.21.15.91:6789/0 conn(0x7f59e00633f0 legacy=0x7f59e014e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:05.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59dffff700 1 -- 172.21.15.203:0/2099381558 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e0151710 con 0x7f59e0063010 2021-01-06T22:04:05.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59dffff700 1 --2- 172.21.15.203:0/2099381558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59e0063010 0x7f59e0151160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f59e00667e0 tx=0x7f59d0006ea0).ready entity=mon.1 client_cookie=f86715658dc2f0c6 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:05.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59deffd700 1 -- 172.21.15.203:0/2099381558 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d0009960 con 0x7f59e0063010 2021-01-06T22:04:05.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59deffd700 1 -- 172.21.15.203:0/2099381558 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d0009ac0 con 0x7f59e0063010 2021-01-06T22:04:05.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59deffd700 1 -- 172.21.15.203:0/2099381558 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d0009e40 con 0x7f59e0063010 2021-01-06T22:04:05.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59e5768700 1 -- 172.21.15.203:0/2099381558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59e0063010 msgr2=0x7f59e0151160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59e5768700 1 --2- 172.21.15.203:0/2099381558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59e0063010 0x7f59e0151160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f59e00667e0 tx=0x7f59d0006ea0).stop 2021-01-06T22:04:05.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59e5768700 1 -- 172.21.15.203:0/2099381558 shutdown_connections 2021-01-06T22:04:05.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59e5768700 1 --2- 172.21.15.203:0/2099381558 >> v2:172.21.15.91:3300/0 conn(0x7f59e00be1b0 0x7f59e00be620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59e5768700 1 --2- 172.21.15.203:0/2099381558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59e0063010 0x7f59e0151160 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59e5768700 1 -- 172.21.15.203:0/2099381558 >> 172.21.15.203:0/2099381558 conn(0x7f59e0067e20 msgr2=0x7f59e0068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:05.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59e5768700 1 -- 172.21.15.203:0/2099381558 shutdown_connections 2021-01-06T22:04:05.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59e5768700 1 -- 172.21.15.203:0/2099381558 wait complete. 2021-01-06T22:04:05.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59e5768700 1 Processor -- start 2021-01-06T22:04:05.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.751+0000 7f59e5768700 1 -- start start 2021-01-06T22:04:05.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e5768700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59e0063010 0x7f59e015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:05.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e5768700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59e00633f0 0x7f59e0153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:05.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e5768700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59e00be1b0 0x7f59e0145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:05.760 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e5768700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e00c1f60 con 0x7f59e0063010 2021-01-06T22:04:05.760 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e5768700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e015e180 con 0x7f59e00be1b0 2021-01-06T22:04:05.760 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e5768700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e0152e70 con 0x7f59e00633f0 2021-01-06T22:04:05.761 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e4f67700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59e00be1b0 0x7f59e0145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:05.761 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e4f67700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59e00be1b0 0x7f59e0145540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49434/0 (socket says 172.21.15.203:49434) 2021-01-06T22:04:05.761 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59dffff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59e0063010 0x7f59e015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:05.761 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e4f67700 1 -- 172.21.15.203:0/2167388471 learned_addr learned my addr 172.21.15.203:0/2167388471 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:05.761 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59df7fe700 1 --2- 172.21.15.203:0/2167388471 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59e00633f0 0x7f59e0153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:05.762 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e4f67700 1 -- 172.21.15.203:0/2167388471 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59e00633f0 msgr2=0x7f59e0153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.762 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e4f67700 1 --2- 172.21.15.203:0/2167388471 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59e00633f0 0x7f59e0153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.762 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e4f67700 1 -- 172.21.15.203:0/2167388471 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59e0063010 msgr2=0x7f59e015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.762 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e4f67700 1 --2- 172.21.15.203:0/2167388471 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59e0063010 0x7f59e015dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.763 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e4f67700 1 -- 172.21.15.203:0/2167388471 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e0152690 con 0x7f59e00be1b0 2021-01-06T22:04:05.763 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e4f67700 1 --2- 172.21.15.203:0/2167388471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59e00be1b0 0x7f59e0145540 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f59d800db30 tx=0x7f59d8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:05.763 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59c7fff700 1 -- 172.21.15.203:0/2167388471 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d80098e0 con 0x7f59e00be1b0 2021-01-06T22:04:05.763 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59c7fff700 1 -- 172.21.15.203:0/2167388471 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d8009a40 con 0x7f59e00be1b0 2021-01-06T22:04:05.764 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e5768700 1 -- 172.21.15.203:0/2167388471 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e0152930 con 0x7f59e00be1b0 2021-01-06T22:04:05.764 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59c7fff700 1 -- 172.21.15.203:0/2167388471 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8009de0 con 0x7f59e00be1b0 2021-01-06T22:04:05.764 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e5768700 1 -- 172.21.15.203:0/2167388471 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e0153340 con 0x7f59e00be1b0 2021-01-06T22:04:05.765 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.755+0000 7f59e5768700 1 -- 172.21.15.203:0/2167388471 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e00450a0 con 0x7f59e00be1b0 2021-01-06T22:04:05.765 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.759+0000 7f59c7fff700 1 -- 172.21.15.203:0/2167388471 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f59d8013690 con 0x7f59e00be1b0 2021-01-06T22:04:05.766 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.759+0000 7f59c7fff700 1 --2- 172.21.15.203:0/2167388471 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f59c00563d0 0x7f59c0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:05.766 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.759+0000 7f59c7fff700 1 -- 172.21.15.203:0/2167388471 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f59d806fa70 con 0x7f59e00be1b0 2021-01-06T22:04:05.766 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.759+0000 7f59dffff700 1 --2- 172.21.15.203:0/2167388471 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f59c00563d0 0x7f59c0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:05.766 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.759+0000 7f59dffff700 1 --2- 172.21.15.203:0/2167388471 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f59c00563d0 0x7f59c0058890 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f59d000b190 tx=0x7f59d0016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:05.768 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.767+0000 7f59c7fff700 1 -- 172.21.15.203:0/2167388471 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f59d8045290 con 0x7f59e00be1b0 2021-01-06T22:04:05.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.919+0000 7f59e5768700 1 -- 172.21.15.203:0/2167388471 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f59e0061e70 con 0x7f59e00be1b0 2021-01-06T22:04:05.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.923+0000 7f59c7fff700 1 -- 172.21.15.203:0/2167388471 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f59d806fe60 con 0x7f59e00be1b0 2021-01-06T22:04:05.923 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:04:05.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.923+0000 7f59e5768700 1 -- 172.21.15.203:0/2167388471 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f59c00563d0 msgr2=0x7f59c0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.923+0000 7f59e5768700 1 --2- 172.21.15.203:0/2167388471 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f59c00563d0 0x7f59c0058890 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f59d000b190 tx=0x7f59d0016000).stop 2021-01-06T22:04:05.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.923+0000 7f59e5768700 1 -- 172.21.15.203:0/2167388471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59e00be1b0 msgr2=0x7f59e0145540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:05.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.923+0000 7f59e5768700 1 --2- 172.21.15.203:0/2167388471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59e00be1b0 0x7f59e0145540 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f59d800db30 tx=0x7f59d8008040).stop 2021-01-06T22:04:05.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.923+0000 7f59e5768700 1 -- 172.21.15.203:0/2167388471 shutdown_connections 2021-01-06T22:04:05.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.923+0000 7f59e5768700 1 --2- 172.21.15.203:0/2167388471 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f59c00563d0 0x7f59c0058890 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.923+0000 7f59e5768700 1 --2- 172.21.15.203:0/2167388471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59e00be1b0 0x7f59e0145540 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.923+0000 7f59e5768700 1 --2- 172.21.15.203:0/2167388471 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59e0063010 0x7f59e015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.923+0000 7f59e5768700 1 --2- 172.21.15.203:0/2167388471 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59e00633f0 0x7f59e0153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:05.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.923+0000 7f59e5768700 1 -- 172.21.15.203:0/2167388471 >> 172.21.15.203:0/2167388471 conn(0x7f59e0067e20 msgr2=0x7f59e00ba3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:05.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.923+0000 7f59e5768700 1 -- 172.21.15.203:0/2167388471 shutdown_connections 2021-01-06T22:04:05.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:05.923+0000 7f59e5768700 1 -- 172.21.15.203:0/2167388471 wait complete. 2021-01-06T22:04:05.938 INFO:tasks.mds_thrash:Wait for all MDSs to reach steady state... 2021-01-06T22:04:05.938 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:04:06.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.059+0000 7fc756b89700 1 Processor -- start 2021-01-06T22:04:06.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.059+0000 7fc756b89700 1 -- start start 2021-01-06T22:04:06.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.059+0000 7fc756b89700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7500636c0 0x7fc75014ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.059+0000 7fc756b89700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc750063aa0 0x7fc750147f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.059+0000 7fc756b89700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc750060780 con 0x7fc7500602b0 2021-01-06T22:04:06.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc756b89700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc75014f340 con 0x7fc750063aa0 2021-01-06T22:04:06.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc756b89700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc750060a20 con 0x7fc7500636c0 2021-01-06T22:04:06.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc754925700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7500636c0 0x7fc75014ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc754925700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7500636c0 0x7fc75014ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:54520/0 (socket says 172.21.15.203:54520) 2021-01-06T22:04:06.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc74ffff700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc7500602b0 0x7fc7500bb650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:42688/0 (socket says 172.21.15.203:42688) 2021-01-06T22:04:06.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc754925700 1 -- 172.21.15.203:0/672942041 learned_addr learned my addr 172.21.15.203:0/672942041 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:06.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc755126700 1 --2- 172.21.15.203:0/672942041 >> v2:172.21.15.91:3300/0 conn(0x7fc750063aa0 0x7fc750147f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc74f7fe700 1 -- 172.21.15.203:0/672942041 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780527091 0 0) 0x7fc750060780 con 0x7fc7500602b0 2021-01-06T22:04:06.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc754925700 1 -- 172.21.15.203:0/672942041 >> v2:172.21.15.91:3300/0 conn(0x7fc750063aa0 msgr2=0x7fc750147f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:06.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc754925700 1 --2- 172.21.15.203:0/672942041 >> v2:172.21.15.91:3300/0 conn(0x7fc750063aa0 0x7fc750147f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc754925700 1 -- 172.21.15.203:0/672942041 >> v1:172.21.15.91:6789/0 conn(0x7fc7500602b0 legacy=0x7fc7500bb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc754925700 1 -- 172.21.15.203:0/672942041 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc750060cc0 con 0x7fc7500636c0 2021-01-06T22:04:06.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc754925700 1 --2- 172.21.15.203:0/672942041 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7500636c0 0x7fc75014ee50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fc74000d2f0 tx=0x7fc74000b530).ready entity=mon.2 client_cookie=1f7504e3f95477d3 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:06.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc74f7fe700 1 -- 172.21.15.203:0/672942041 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74000de70 con 0x7fc7500636c0 2021-01-06T22:04:06.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc74f7fe700 1 -- 172.21.15.203:0/672942041 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc740010680 con 0x7fc7500636c0 2021-01-06T22:04:06.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc74f7fe700 1 -- 172.21.15.203:0/672942041 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc740010a00 con 0x7fc7500636c0 2021-01-06T22:04:06.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc756b89700 1 -- 172.21.15.203:0/672942041 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7500636c0 msgr2=0x7fc75014ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc756b89700 1 --2- 172.21.15.203:0/672942041 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7500636c0 0x7fc75014ee50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fc74000d2f0 tx=0x7fc74000b530).stop 2021-01-06T22:04:06.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc756b89700 1 -- 172.21.15.203:0/672942041 shutdown_connections 2021-01-06T22:04:06.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc756b89700 1 --2- 172.21.15.203:0/672942041 >> v2:172.21.15.91:3300/0 conn(0x7fc750063aa0 0x7fc750147f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc756b89700 1 --2- 172.21.15.203:0/672942041 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7500636c0 0x7fc75014ee50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc756b89700 1 -- 172.21.15.203:0/672942041 >> 172.21.15.203:0/672942041 conn(0x7fc7500c0cb0 msgr2=0x7fc7500c1090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:06.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc756b89700 1 -- 172.21.15.203:0/672942041 shutdown_connections 2021-01-06T22:04:06.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.063+0000 7fc756b89700 1 -- 172.21.15.203:0/672942041 wait complete. 2021-01-06T22:04:06.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc756b89700 1 Processor -- start 2021-01-06T22:04:06.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc756b89700 1 -- start start 2021-01-06T22:04:06.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc756b89700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7500602b0 0x7fc750141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc756b89700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7500636c0 0x7fc750141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc756b89700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc750063aa0 0x7fc75013f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc756b89700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7500b4d50 con 0x7fc7500636c0 2021-01-06T22:04:06.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc756b89700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc750141310 con 0x7fc750063aa0 2021-01-06T22:04:06.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc756b89700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7501420e0 con 0x7fc7500602b0 2021-01-06T22:04:06.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc755126700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc750063aa0 0x7fc75013f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc755126700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc750063aa0 0x7fc75013f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49448/0 (socket says 172.21.15.203:49448) 2021-01-06T22:04:06.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc755126700 1 -- 172.21.15.203:0/327137004 learned_addr learned my addr 172.21.15.203:0/327137004 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:06.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc74ffff700 1 --2- 172.21.15.203:0/327137004 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7500636c0 0x7fc750141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc754925700 1 --2- 172.21.15.203:0/327137004 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7500602b0 0x7fc750141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc755126700 1 -- 172.21.15.203:0/327137004 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7500602b0 msgr2=0x7fc750141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc755126700 1 --2- 172.21.15.203:0/327137004 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7500602b0 0x7fc750141700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc755126700 1 -- 172.21.15.203:0/327137004 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7500636c0 msgr2=0x7fc750141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc755126700 1 --2- 172.21.15.203:0/327137004 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7500636c0 0x7fc750141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc755126700 1 -- 172.21.15.203:0/327137004 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7500b4f40 con 0x7fc750063aa0 2021-01-06T22:04:06.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc755126700 1 --2- 172.21.15.203:0/327137004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc750063aa0 0x7fc75013f3c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fc74800ebd0 tx=0x7fc748006de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:06.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc74cff9700 1 -- 172.21.15.203:0/327137004 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74800b8a0 con 0x7fc750063aa0 2021-01-06T22:04:06.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc74cff9700 1 -- 172.21.15.203:0/327137004 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74800ba00 con 0x7fc750063aa0 2021-01-06T22:04:06.083 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc756b89700 1 -- 172.21.15.203:0/327137004 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7500b9c60 con 0x7fc750063aa0 2021-01-06T22:04:06.083 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc74cff9700 1 -- 172.21.15.203:0/327137004 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74800bda0 con 0x7fc750063aa0 2021-01-06T22:04:06.084 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.067+0000 7fc756b89700 1 -- 172.21.15.203:0/327137004 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7500b51a0 con 0x7fc750063aa0 2021-01-06T22:04:06.084 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.071+0000 7fc756b89700 1 -- 172.21.15.203:0/327137004 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc75005d600 con 0x7fc750063aa0 2021-01-06T22:04:06.084 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.071+0000 7fc74cff9700 1 -- 172.21.15.203:0/327137004 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fc748012600 con 0x7fc750063aa0 2021-01-06T22:04:06.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.071+0000 7fc74cff9700 1 --2- 172.21.15.203:0/327137004 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc7300563d0 0x7fc730058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.071+0000 7fc74cff9700 1 -- 172.21.15.203:0/327137004 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc74806f840 con 0x7fc750063aa0 2021-01-06T22:04:06.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.071+0000 7fc754925700 1 --2- 172.21.15.203:0/327137004 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc7300563d0 0x7fc730058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.071+0000 7fc754925700 1 --2- 172.21.15.203:0/327137004 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc7300563d0 0x7fc730058890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fc74000b570 tx=0x7fc750057180).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:06.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.079+0000 7fc74cff9700 1 -- 172.21.15.203:0/327137004 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fc7480450c0 con 0x7fc750063aa0 2021-01-06T22:04:06.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.275+0000 7fc756b89700 1 -- 172.21.15.203:0/327137004 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc750147920 con 0x7fc750063aa0 2021-01-06T22:04:06.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.279+0000 7fc74cff9700 1 -- 172.21.15.203:0/327137004 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+3574 (secure 0 0 0) 0x7fc7480486e0 con 0x7fc750063aa0 2021-01-06T22:04:06.281 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:06.281 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":5}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:03:44.792784+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1116012025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1116012025},{"type":"v1","addr":"172.21.15.203:6835","nonce":1116012025}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:04:06.283 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.279+0000 7fc756b89700 1 -- 172.21.15.203:0/327137004 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc7300563d0 msgr2=0x7fc730058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.283 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.279+0000 7fc756b89700 1 --2- 172.21.15.203:0/327137004 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc7300563d0 0x7fc730058890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fc74000b570 tx=0x7fc750057180).stop 2021-01-06T22:04:06.283 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.279+0000 7fc756b89700 1 -- 172.21.15.203:0/327137004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc750063aa0 msgr2=0x7fc75013f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.284 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.279+0000 7fc756b89700 1 --2- 172.21.15.203:0/327137004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc750063aa0 0x7fc75013f3c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fc74800ebd0 tx=0x7fc748006de0).stop 2021-01-06T22:04:06.284 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.279+0000 7fc756b89700 1 -- 172.21.15.203:0/327137004 shutdown_connections 2021-01-06T22:04:06.284 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.279+0000 7fc756b89700 1 --2- 172.21.15.203:0/327137004 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc7300563d0 0x7fc730058890 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.284 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.279+0000 7fc756b89700 1 --2- 172.21.15.203:0/327137004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc750063aa0 0x7fc75013f3c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.285 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.279+0000 7fc756b89700 1 --2- 172.21.15.203:0/327137004 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7500602b0 0x7fc750141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.285 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.279+0000 7fc756b89700 1 --2- 172.21.15.203:0/327137004 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7500636c0 0x7fc750141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.285 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.279+0000 7fc756b89700 1 -- 172.21.15.203:0/327137004 >> 172.21.15.203:0/327137004 conn(0x7fc7500c0cb0 msgr2=0x7fc7500626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:06.286 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.279+0000 7fc756b89700 1 -- 172.21.15.203:0/327137004 shutdown_connections 2021-01-06T22:04:06.286 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.279+0000 7fc756b89700 1 -- 172.21.15.203:0/327137004 wait complete. 2021-01-06T22:04:06.286 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 12 2021-01-06T22:04:06.295 INFO:tasks.mds_thrash:Ready to start thrashing 2021-01-06T22:04:06.295 INFO:tasks.mds_thrash.ceph_manager:waiting for clean 2021-01-06T22:04:06.296 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-06T22:04:06.428 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.427+0000 7f4fc4a6b700 1 Processor -- start 2021-01-06T22:04:06.429 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.427+0000 7f4fc4a6b700 1 -- start start 2021-01-06T22:04:06.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.427+0000 7f4fc4a6b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fc00bbb20 0x7f4fc00670d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.427+0000 7f4fc4a6b700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc00bf010 con 0x7f4fc00c4760 2021-01-06T22:04:06.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.427+0000 7f4fc4a6b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc0067680 con 0x7f4fc00c1860 2021-01-06T22:04:06.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.427+0000 7f4fc4a6b700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc00bf240 con 0x7f4fc00bbb20 2021-01-06T22:04:06.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.427+0000 7f4fb5d9b700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f4fc00c1860 0x7f4fc014ae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48046/0 (socket says 172.21.15.203:48046) 2021-01-06T22:04:06.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.427+0000 7f4fb5d9b700 1 -- 172.21.15.203:0/2455196553 learned_addr learned my addr 172.21.15.203:0/2455196553 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:04:06.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.427+0000 7f4fbed9d700 1 --2- 172.21.15.203:0/2455196553 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fc00bbb20 0x7f4fc00670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.427+0000 7f4fbdd9b700 1 -- 172.21.15.203:0/2455196553 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175585431 0 0) 0x7f4fc0067680 con 0x7f4fc00c1860 2021-01-06T22:04:06.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.427+0000 7f4fbdd9b700 1 -- 172.21.15.203:0/2455196553 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa0003230 con 0x7f4fc00c1860 2021-01-06T22:04:06.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 -- 172.21.15.203:0/2455196553 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075245311 0 0) 0x7f4fc00bf010 con 0x7f4fc00c4760 2021-01-06T22:04:06.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 -- 172.21.15.203:0/2455196553 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fc0067680 con 0x7f4fc00c4760 2021-01-06T22:04:06.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 -- 172.21.15.203:0/2455196553 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4147683287 0 0) 0x7f4fa0003230 con 0x7f4fc00c1860 2021-01-06T22:04:06.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 -- 172.21.15.203:0/2455196553 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fc00bbb20 msgr2=0x7f4fc00670d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:06.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 --2- 172.21.15.203:0/2455196553 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fc00bbb20 0x7f4fc00670d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 -- 172.21.15.203:0/2455196553 >> v1:172.21.15.91:6789/0 conn(0x7f4fc00c4760 legacy=0x7f4fc00c1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 -- 172.21.15.203:0/2455196553 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc00bf4e0 con 0x7f4fc00c1860 2021-01-06T22:04:06.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 -- 172.21.15.203:0/2455196553 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f4fa80014e0 con 0x7f4fc00c1860 2021-01-06T22:04:06.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 -- 172.21.15.203:0/2455196553 >> v1:172.21.15.203:6789/0 conn(0x7f4fc00c1860 legacy=0x7f4fc014ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 --2- 172.21.15.203:0/2455196553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fa00038b0 0x7f4fa0003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 --2- 172.21.15.203:0/2455196553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fa00044d0 0x7f4fa0004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 --2- 172.21.15.203:0/2455196553 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fc00bbb20 0x7f4fa0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 -- 172.21.15.203:0/2455196553 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc00bf240 con 0x7f4fa00044d0 2021-01-06T22:04:06.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 -- 172.21.15.203:0/2455196553 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa0005330 con 0x7f4fa00038b0 2021-01-06T22:04:06.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 -- 172.21.15.203:0/2455196553 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa000b720 con 0x7f4fc00bbb20 2021-01-06T22:04:06.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbe59c700 1 --2- 172.21.15.203:0/2455196553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fa00038b0 0x7f4fa0003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbed9d700 1 --2- 172.21.15.203:0/2455196553 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fc00bbb20 0x7f4fa0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fb5d9b700 1 --2- 172.21.15.203:0/2455196553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fa00044d0 0x7f4fa0004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbe59c700 1 -- 172.21.15.203:0/2455196553 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fc00bbb20 msgr2=0x7f4fa0004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:06.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbe59c700 1 --2- 172.21.15.203:0/2455196553 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fc00bbb20 0x7f4fa0004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbe59c700 1 -- 172.21.15.203:0/2455196553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fa00044d0 msgr2=0x7f4fa0004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:06.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbe59c700 1 --2- 172.21.15.203:0/2455196553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fa00044d0 0x7f4fa0004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbe59c700 1 -- 172.21.15.203:0/2455196553 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa0003230 con 0x7f4fa00038b0 2021-01-06T22:04:06.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbe59c700 1 --2- 172.21.15.203:0/2455196553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fa00038b0 0x7f4fa0003cb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f4fb0002000 tx=0x7f4fb0007f00).ready entity=mon.1 client_cookie=22b38432e48ab4fd server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:06.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 -- 172.21.15.203:0/2455196553 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb00016a0 con 0x7f4fa00038b0 2021-01-06T22:04:06.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 -- 172.21.15.203:0/2455196553 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb000cad0 con 0x7f4fa00038b0 2021-01-06T22:04:06.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbdd9b700 1 -- 172.21.15.203:0/2455196553 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb000ce70 con 0x7f4fa00038b0 2021-01-06T22:04:06.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fc4a6b700 1 -- 172.21.15.203:0/2455196553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fa00038b0 msgr2=0x7f4fa0003cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fc4a6b700 1 --2- 172.21.15.203:0/2455196553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fa00038b0 0x7f4fa0003cb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f4fb0002000 tx=0x7f4fb0007f00).stop 2021-01-06T22:04:06.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fbed9d700 1 -- 172.21.15.203:0/2455196553 reap_dead start 2021-01-06T22:04:06.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fc4a6b700 1 -- 172.21.15.203:0/2455196553 shutdown_connections 2021-01-06T22:04:06.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fc4a6b700 1 -- 172.21.15.203:0/2455196553 >> 172.21.15.203:0/2455196553 conn(0x7f4fc0066200 msgr2=0x7f4fc0056cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:06.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fc4a6b700 1 -- 172.21.15.203:0/2455196553 shutdown_connections 2021-01-06T22:04:06.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.431+0000 7f4fc4a6b700 1 -- 172.21.15.203:0/2455196553 wait complete. 2021-01-06T22:04:06.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fc4a6b700 1 Processor -- start 2021-01-06T22:04:06.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fc4a6b700 1 -- start start 2021-01-06T22:04:06.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fc4a6b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fa00038b0 0x7f4fc00644f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fc4a6b700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fc014bca0 0x7f4fc00bdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fc4a6b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fc0161190 0x7f4fc01615f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fc4a6b700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0160a90 con 0x7f4fc014bca0 2021-01-06T22:04:06.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fc4a6b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0151500 con 0x7f4fa00038b0 2021-01-06T22:04:06.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fc4a6b700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc00bc110 con 0x7f4fc0161190 2021-01-06T22:04:06.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fbe59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fa00038b0 0x7f4fc00644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fbe59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fa00038b0 0x7f4fc00644f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49466/0 (socket says 172.21.15.203:49466) 2021-01-06T22:04:06.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fbe59c700 1 -- 172.21.15.203:0/3981825430 learned_addr learned my addr 172.21.15.203:0/3981825430 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:06.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fb5d9b700 1 --2- 172.21.15.203:0/3981825430 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fc014bca0 0x7f4fc00bdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fbed9d700 1 --2- 172.21.15.203:0/3981825430 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fc0161190 0x7f4fc01615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fbe59c700 1 -- 172.21.15.203:0/3981825430 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fc0161190 msgr2=0x7f4fc01615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fbe59c700 1 --2- 172.21.15.203:0/3981825430 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fc0161190 0x7f4fc01615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fbe59c700 1 -- 172.21.15.203:0/3981825430 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fc014bca0 msgr2=0x7f4fc00bdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fbe59c700 1 --2- 172.21.15.203:0/3981825430 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fc014bca0 0x7f4fc00bdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fbe59c700 1 -- 172.21.15.203:0/3981825430 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc00bc3b0 con 0x7f4fa00038b0 2021-01-06T22:04:06.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fb5d9b700 1 --2- 172.21.15.203:0/3981825430 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fc014bca0 0x7f4fc00bdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:04:06.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fbe59c700 1 --2- 172.21.15.203:0/3981825430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fa00038b0 0x7f4fc00644f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f4fb0007f60 tx=0x7f4fb0014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:06.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fb6ffd700 1 -- 172.21.15.203:0/3981825430 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb001d5c0 con 0x7f4fa00038b0 2021-01-06T22:04:06.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fb6ffd700 1 -- 172.21.15.203:0/3981825430 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb001d740 con 0x7f4fa00038b0 2021-01-06T22:04:06.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fc4a6b700 1 -- 172.21.15.203:0/3981825430 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc00bc650 con 0x7f4fa00038b0 2021-01-06T22:04:06.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fb6ffd700 1 -- 172.21.15.203:0/3981825430 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb001db00 con 0x7f4fa00038b0 2021-01-06T22:04:06.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.435+0000 7f4fc4a6b700 1 -- 172.21.15.203:0/3981825430 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc0160cf0 con 0x7f4fa00038b0 2021-01-06T22:04:06.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.439+0000 7f4fc4a6b700 1 -- 172.21.15.203:0/3981825430 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc006a480 con 0x7f4fa00038b0 2021-01-06T22:04:06.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.439+0000 7f4fb6ffd700 1 -- 172.21.15.203:0/3981825430 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f4fb0025030 con 0x7f4fa00038b0 2021-01-06T22:04:06.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.439+0000 7f4fb6ffd700 1 --2- 172.21.15.203:0/3981825430 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4fa4056400 0x7f4fa40588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.439+0000 7f4fb6ffd700 1 -- 172.21.15.203:0/3981825430 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4fb0070d40 con 0x7f4fa00038b0 2021-01-06T22:04:06.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.439+0000 7f4fb5d9b700 1 --2- 172.21.15.203:0/3981825430 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4fa4056400 0x7f4fa40588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.439+0000 7f4fb5d9b700 1 --2- 172.21.15.203:0/3981825430 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4fa4056400 0x7f4fa40588c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f4fa8001610 tx=0x7f4fa8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:06.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.443+0000 7f4fb6ffd700 1 -- 172.21.15.203:0/3981825430 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f4fb0046670 con 0x7f4fa00038b0 2021-01-06T22:04:06.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.591+0000 7f4fc4a6b700 1 -- 172.21.15.203:0/3981825430 --> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4fc0064b70 con 0x7f4fa4056400 2021-01-06T22:04:06.601 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.599+0000 7f4fb6ffd700 1 -- 172.21.15.203:0/3981825430 <== mgr.4102 v2:172.21.15.203:6800/18146 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82257 (secure 0 0 0) 0x7f4fc0064b70 con 0x7f4fa4056400 2021-01-06T22:04:06.603 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:06.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.603+0000 7f4fc4a6b700 1 -- 172.21.15.203:0/3981825430 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4fa4056400 msgr2=0x7f4fa40588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.603+0000 7f4fc4a6b700 1 --2- 172.21.15.203:0/3981825430 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4fa4056400 0x7f4fa40588c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f4fa8001610 tx=0x7f4fa8009040).stop 2021-01-06T22:04:06.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.603+0000 7f4fc4a6b700 1 -- 172.21.15.203:0/3981825430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fa00038b0 msgr2=0x7f4fc00644f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.603+0000 7f4fc4a6b700 1 --2- 172.21.15.203:0/3981825430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fa00038b0 0x7f4fc00644f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f4fb0007f60 tx=0x7f4fb0014000).stop 2021-01-06T22:04:06.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.603+0000 7f4fc4a6b700 1 -- 172.21.15.203:0/3981825430 shutdown_connections 2021-01-06T22:04:06.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.603+0000 7f4fc4a6b700 1 --2- 172.21.15.203:0/3981825430 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4fa4056400 0x7f4fa40588c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.603+0000 7f4fc4a6b700 1 --2- 172.21.15.203:0/3981825430 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fc0161190 0x7f4fc01615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.603+0000 7f4fc4a6b700 1 --2- 172.21.15.203:0/3981825430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fa00038b0 0x7f4fc00644f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.607 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.603+0000 7f4fc4a6b700 1 --2- 172.21.15.203:0/3981825430 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fc014bca0 0x7f4fc00bdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.607 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.603+0000 7f4fc4a6b700 1 -- 172.21.15.203:0/3981825430 >> 172.21.15.203:0/3981825430 conn(0x7f4fc0066200 msgr2=0x7f4fc0056cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:06.607 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.603+0000 7f4fc4a6b700 1 -- 172.21.15.203:0/3981825430 shutdown_connections 2021-01-06T22:04:06.607 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.603+0000 7f4fc4a6b700 1 -- 172.21.15.203:0/3981825430 wait complete. 2021-01-06T22:04:06.608 INFO:teuthology.orchestra.run.smithi203.stderr:dumped all 2021-01-06T22:04:06.619 INFO:teuthology.orchestra.run.smithi203.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2021-01-06T22:04:05.098797+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4962,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":40988,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":38912,"kb_avail":754933732,"statfs":{"total":773094113280,"available":773052141568,"internally_reserved":0,"allocated":2060288,"data_stored":741074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39845888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004021"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.758566+0000","last_change":"2021-01-06T22:03:39.757200+0000","last_active":"2021-01-06T22:03:39.758566+0000","last_peered":"2021-01-06T22:03:39.758566+0000","last_clean":"2021-01-06T22:03:39.758566+0000","last_became_active":"2021-01-06T22:03:39.756706+0000","last_became_peered":"2021-01-06T22:03:39.756706+0000","last_unstale":"2021-01-06T22:03:39.758566+0000","last_undegraded":"2021-01-06T22:03:39.758566+0000","last_fullsized":"2021-01-06T22:03:39.758566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.755461+0000","last_change":"2021-01-06T22:03:36.733731+0000","last_active":"2021-01-06T22:03:39.755461+0000","last_peered":"2021-01-06T22:03:39.755461+0000","last_clean":"2021-01-06T22:03:39.755461+0000","last_became_active":"2021-01-06T22:03:36.733026+0000","last_became_peered":"2021-01-06T22:03:36.733026+0000","last_unstale":"2021-01-06T22:03:39.755461+0000","last_undegraded":"2021-01-06T22:03:39.755461+0000","last_fullsized":"2021-01-06T22:03:39.755461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"23'10","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.806682+0000","last_change":"2021-01-06T22:03:39.372339+0000","last_active":"2021-01-06T22:03:43.806682+0000","last_peered":"2021-01-06T22:03:43.806682+0000","last_clean":"2021-01-06T22:03:43.806682+0000","last_became_active":"2021-01-06T22:03:39.372000+0000","last_became_peered":"2021-01-06T22:03:39.372000+0000","last_unstale":"2021-01-06T22:03:43.806682+0000","last_undegraded":"2021-01-06T22:03:43.806682+0000","last_fullsized":"2021-01-06T22:03:43.806682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":735,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.758556+0000","last_change":"2021-01-06T22:03:39.757152+0000","last_active":"2021-01-06T22:03:39.758556+0000","last_peered":"2021-01-06T22:03:39.758556+0000","last_clean":"2021-01-06T22:03:39.758556+0000","last_became_active":"2021-01-06T22:03:39.756673+0000","last_became_peered":"2021-01-06T22:03:39.756673+0000","last_unstale":"2021-01-06T22:03:39.758556+0000","last_undegraded":"2021-01-06T22:03:39.758556+0000","last_fullsized":"2021-01-06T22:03:39.758556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747142+0000","last_change":"2021-01-06T22:03:36.731052+0000","last_active":"2021-01-06T22:03:39.747142+0000","last_peered":"2021-01-06T22:03:39.747142+0000","last_clean":"2021-01-06T22:03:39.747142+0000","last_became_active":"2021-01-06T22:03:36.730252+0000","last_became_peered":"2021-01-06T22:03:36.730252+0000","last_unstale":"2021-01-06T22:03:39.747142+0000","last_undegraded":"2021-01-06T22:03:39.747142+0000","last_fullsized":"2021-01-06T22:03:39.747142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"23'18","reported_seq":"28","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.807276+0000","last_change":"2021-01-06T22:03:39.371415+0000","last_active":"2021-01-06T22:03:43.807276+0000","last_peered":"2021-01-06T22:03:43.807276+0000","last_clean":"2021-01-06T22:03:43.807276+0000","last_became_active":"2021-01-06T22:03:39.370981+0000","last_became_peered":"2021-01-06T22:03:39.370981+0000","last_unstale":"2021-01-06T22:03:43.807276+0000","last_undegraded":"2021-01-06T22:03:43.807276+0000","last_fullsized":"2021-01-06T22:03:43.807276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":642,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.776586+0000","last_change":"2021-01-06T22:03:39.775845+0000","last_active":"2021-01-06T22:03:39.776586+0000","last_peered":"2021-01-06T22:03:39.776586+0000","last_clean":"2021-01-06T22:03:39.776586+0000","last_became_active":"2021-01-06T22:03:39.775329+0000","last_became_peered":"2021-01-06T22:03:39.775329+0000","last_unstale":"2021-01-06T22:03:39.776586+0000","last_undegraded":"2021-01-06T22:03:39.776586+0000","last_fullsized":"2021-01-06T22:03:39.776586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.751998+0000","last_change":"2021-01-06T22:03:36.209909+0000","last_active":"2021-01-06T22:03:39.751998+0000","last_peered":"2021-01-06T22:03:39.751998+0000","last_clean":"2021-01-06T22:03:39.751998+0000","last_became_active":"2021-01-06T22:03:36.209405+0000","last_became_peered":"2021-01-06T22:03:36.209405+0000","last_unstale":"2021-01-06T22:03:39.751998+0000","last_undegraded":"2021-01-06T22:03:39.751998+0000","last_fullsized":"2021-01-06T22:03:39.751998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"23'21","reported_seq":"31","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.819440+0000","last_change":"2021-01-06T22:03:39.374989+0000","last_active":"2021-01-06T22:03:43.819440+0000","last_peered":"2021-01-06T22:03:43.819440+0000","last_clean":"2021-01-06T22:03:43.819440+0000","last_became_active":"2021-01-06T22:03:39.374506+0000","last_became_peered":"2021-01-06T22:03:39.374506+0000","last_unstale":"2021-01-06T22:03:43.819440+0000","last_undegraded":"2021-01-06T22:03:43.819440+0000","last_fullsized":"2021-01-06T22:03:43.819440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":112,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:40.227173+0000","last_change":"2021-01-06T22:03:40.226400+0000","last_active":"2021-01-06T22:03:40.227173+0000","last_peered":"2021-01-06T22:03:40.227173+0000","last_clean":"2021-01-06T22:03:40.227173+0000","last_became_active":"2021-01-06T22:03:40.225932+0000","last_became_peered":"2021-01-06T22:03:40.225932+0000","last_unstale":"2021-01-06T22:03:40.227173+0000","last_undegraded":"2021-01-06T22:03:40.227173+0000","last_fullsized":"2021-01-06T22:03:40.227173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747380+0000","last_change":"2021-01-06T22:03:36.243205+0000","last_active":"2021-01-06T22:03:39.747380+0000","last_peered":"2021-01-06T22:03:39.747380+0000","last_clean":"2021-01-06T22:03:39.747380+0000","last_became_active":"2021-01-06T22:03:36.242610+0000","last_became_peered":"2021-01-06T22:03:36.242610+0000","last_unstale":"2021-01-06T22:03:39.747380+0000","last_undegraded":"2021-01-06T22:03:39.747380+0000","last_fullsized":"2021-01-06T22:03:39.747380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"17","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:40.222155+0000","last_change":"2021-01-06T22:03:32.172575+0000","last_active":"2021-01-06T22:03:40.222155+0000","last_peered":"2021-01-06T22:03:40.222155+0000","last_clean":"2021-01-06T22:03:40.222155+0000","last_became_active":"2021-01-06T22:03:32.171025+0000","last_became_peered":"2021-01-06T22:03:32.171025+0000","last_unstale":"2021-01-06T22:03:40.222155+0000","last_undegraded":"2021-01-06T22:03:40.222155+0000","last_fullsized":"2021-01-06T22:03:40.222155+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":8,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:25.162227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:25.162227+0000","last_clean_scrub_stamp":"2021-01-06T22:03:25.162227+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"23'8","reported_seq":"18","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.815252+0000","last_change":"2021-01-06T22:03:39.374385+0000","last_active":"2021-01-06T22:03:43.815252+0000","last_peered":"2021-01-06T22:03:43.815252+0000","last_clean":"2021-01-06T22:03:43.815252+0000","last_became_active":"2021-01-06T22:03:39.374029+0000","last_became_peered":"2021-01-06T22:03:39.374029+0000","last_unstale":"2021-01-06T22:03:43.815252+0000","last_undegraded":"2021-01-06T22:03:43.815252+0000","last_fullsized":"2021-01-06T22:03:43.815252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.765901+0000","last_change":"2021-01-06T22:03:39.762496+0000","last_active":"2021-01-06T22:03:39.765901+0000","last_peered":"2021-01-06T22:03:39.765901+0000","last_clean":"2021-01-06T22:03:39.765901+0000","last_became_active":"2021-01-06T22:03:39.761903+0000","last_became_peered":"2021-01-06T22:03:39.761903+0000","last_unstale":"2021-01-06T22:03:39.765901+0000","last_undegraded":"2021-01-06T22:03:39.765901+0000","last_fullsized":"2021-01-06T22:03:39.765901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747295+0000","last_change":"2021-01-06T22:03:36.206609+0000","last_active":"2021-01-06T22:03:39.747295+0000","last_peered":"2021-01-06T22:03:39.747295+0000","last_clean":"2021-01-06T22:03:39.747295+0000","last_became_active":"2021-01-06T22:03:36.206123+0000","last_became_peered":"2021-01-06T22:03:36.206123+0000","last_unstale":"2021-01-06T22:03:39.747295+0000","last_undegraded":"2021-01-06T22:03:39.747295+0000","last_fullsized":"2021-01-06T22:03:39.747295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"23'12","reported_seq":"22","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.821871+0000","last_change":"2021-01-06T22:03:39.377897+0000","last_active":"2021-01-06T22:03:43.821871+0000","last_peered":"2021-01-06T22:03:43.821871+0000","last_clean":"2021-01-06T22:03:43.821871+0000","last_became_active":"2021-01-06T22:03:39.376836+0000","last_became_peered":"2021-01-06T22:03:39.376836+0000","last_unstale":"2021-01-06T22:03:43.821871+0000","last_undegraded":"2021-01-06T22:03:43.821871+0000","last_fullsized":"2021-01-06T22:03:43.821871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":720,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.778763+0000","last_change":"2021-01-06T22:03:39.777501+0000","last_active":"2021-01-06T22:03:39.778763+0000","last_peered":"2021-01-06T22:03:39.778763+0000","last_clean":"2021-01-06T22:03:39.778763+0000","last_became_active":"2021-01-06T22:03:39.777158+0000","last_became_peered":"2021-01-06T22:03:39.777158+0000","last_unstale":"2021-01-06T22:03:39.778763+0000","last_undegraded":"2021-01-06T22:03:39.778763+0000","last_fullsized":"2021-01-06T22:03:39.778763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747656+0000","last_change":"2021-01-06T22:03:36.208974+0000","last_active":"2021-01-06T22:03:39.747656+0000","last_peered":"2021-01-06T22:03:39.747656+0000","last_clean":"2021-01-06T22:03:39.747656+0000","last_became_active":"2021-01-06T22:03:36.208576+0000","last_became_peered":"2021-01-06T22:03:36.208576+0000","last_unstale":"2021-01-06T22:03:39.747656+0000","last_undegraded":"2021-01-06T22:03:39.747656+0000","last_fullsized":"2021-01-06T22:03:39.747656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"23'13","reported_seq":"23","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.816002+0000","last_change":"2021-01-06T22:03:39.375127+0000","last_active":"2021-01-06T22:03:43.816002+0000","last_peered":"2021-01-06T22:03:43.816002+0000","last_clean":"2021-01-06T22:03:43.816002+0000","last_became_active":"2021-01-06T22:03:39.374751+0000","last_became_peered":"2021-01-06T22:03:39.374751+0000","last_unstale":"2021-01-06T22:03:43.816002+0000","last_undegraded":"2021-01-06T22:03:43.816002+0000","last_fullsized":"2021-01-06T22:03:43.816002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2088,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:40.232168+0000","last_change":"2021-01-06T22:03:40.230887+0000","last_active":"2021-01-06T22:03:40.232168+0000","last_peered":"2021-01-06T22:03:40.232168+0000","last_clean":"2021-01-06T22:03:40.232168+0000","last_became_active":"2021-01-06T22:03:40.230100+0000","last_became_peered":"2021-01-06T22:03:40.230100+0000","last_unstale":"2021-01-06T22:03:40.232168+0000","last_undegraded":"2021-01-06T22:03:40.232168+0000","last_fullsized":"2021-01-06T22:03:40.232168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"23'18","reported_seq":"28","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:46.596414+0000","last_change":"2021-01-06T22:03:39.377489+0000","last_active":"2021-01-06T22:03:46.596414+0000","last_peered":"2021-01-06T22:03:46.596414+0000","last_clean":"2021-01-06T22:03:46.596414+0000","last_became_active":"2021-01-06T22:03:39.377092+0000","last_became_peered":"2021-01-06T22:03:39.377092+0000","last_unstale":"2021-01-06T22:03:46.596414+0000","last_undegraded":"2021-01-06T22:03:46.596414+0000","last_fullsized":"2021-01-06T22:03:46.596414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":665,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.759458+0000","last_change":"2021-01-06T22:03:36.213113+0000","last_active":"2021-01-06T22:03:39.759458+0000","last_peered":"2021-01-06T22:03:39.759458+0000","last_clean":"2021-01-06T22:03:39.759458+0000","last_became_active":"2021-01-06T22:03:36.212790+0000","last_became_peered":"2021-01-06T22:03:36.212790+0000","last_unstale":"2021-01-06T22:03:39.759458+0000","last_undegraded":"2021-01-06T22:03:39.759458+0000","last_fullsized":"2021-01-06T22:03:39.759458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.764030+0000","last_change":"2021-01-06T22:03:39.759582+0000","last_active":"2021-01-06T22:03:39.764030+0000","last_peered":"2021-01-06T22:03:39.764030+0000","last_clean":"2021-01-06T22:03:39.764030+0000","last_became_active":"2021-01-06T22:03:39.759212+0000","last_became_peered":"2021-01-06T22:03:39.759212+0000","last_unstale":"2021-01-06T22:03:39.764030+0000","last_undegraded":"2021-01-06T22:03:39.764030+0000","last_fullsized":"2021-01-06T22:03:39.764030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747367+0000","last_change":"2021-01-06T22:03:36.207481+0000","last_active":"2021-01-06T22:03:39.747367+0000","last_peered":"2021-01-06T22:03:39.747367+0000","last_clean":"2021-01-06T22:03:39.747367+0000","last_became_active":"2021-01-06T22:03:36.206991+0000","last_became_peered":"2021-01-06T22:03:36.206991+0000","last_unstale":"2021-01-06T22:03:39.747367+0000","last_undegraded":"2021-01-06T22:03:39.747367+0000","last_fullsized":"2021-01-06T22:03:39.747367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'9","reported_seq":"19","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.813735+0000","last_change":"2021-01-06T22:03:39.370350+0000","last_active":"2021-01-06T22:03:43.813735+0000","last_peered":"2021-01-06T22:03:43.813735+0000","last_clean":"2021-01-06T22:03:43.813735+0000","last_became_active":"2021-01-06T22:03:39.369929+0000","last_became_peered":"2021-01-06T22:03:39.369929+0000","last_unstale":"2021-01-06T22:03:43.813735+0000","last_undegraded":"2021-01-06T22:03:43.813735+0000","last_fullsized":"2021-01-06T22:03:43.813735+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":4962,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":10367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5112,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366728,"statfs":{"total":96636764160,"available":96631529472,"internally_reserved":0,"allocated":245760,"data_stored":91578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5470000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8879999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3029999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3090000000000002}]}]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5224,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366616,"statfs":{"total":96636764160,"available":96631414784,"internally_reserved":0,"allocated":294912,"data_stored":95674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3580000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1520000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4060000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2149999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.097}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0430000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5128,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366712,"statfs":{"total":96636764160,"available":96631513088,"internally_reserved":0,"allocated":262144,"data_stored":93666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4220000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9550000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.524}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9420000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":512.53099999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8079999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5116,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366724,"statfs":{"total":96636764160,"available":96631525376,"internally_reserved":0,"allocated":249856,"data_stored":91530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1659999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4049999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.552}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9449999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0990000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0329999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":237568,"data_stored":91466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3650000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0059999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.137}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.645}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6930000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8959999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2480000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5148,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366692,"statfs":{"total":96636764160,"available":96631492608,"internally_reserved":0,"allocated":282624,"data_stored":93563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.173}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3839999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8439999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2719999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6040000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6669999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5070000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5104,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366736,"statfs":{"total":96636764160,"available":96631537664,"internally_reserved":0,"allocated":237568,"data_stored":91466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0539999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2450000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4289999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1749999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3719999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5116,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366724,"statfs":{"total":96636764160,"available":96631525376,"internally_reserved":0,"allocated":249856,"data_stored":92131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.865}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.774}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7330000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.024}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.552}]}]}]}} 2021-01-06T22:04:06.623 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-06T22:04:06.745 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.743+0000 7f58984ec700 1 Processor -- start 2021-01-06T22:04:06.745 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.743+0000 7f58984ec700 1 -- start start 2021-01-06T22:04:06.746 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.743+0000 7f58984ec700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f58900602b0 0x7f58900bb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.743+0000 7f58984ec700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f589014f090 con 0x7f5890057060 2021-01-06T22:04:06.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.743+0000 7f58984ec700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5890068780 con 0x7f5890148320 2021-01-06T22:04:06.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.743+0000 7f58984ec700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58900badc0 con 0x7f58900602b0 2021-01-06T22:04:06.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f5896288700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f5890148320 0x7f58900626e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48068/0 (socket says 172.21.15.203:48068) 2021-01-06T22:04:06.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f5896288700 1 -- 172.21.15.203:0/2068069987 learned_addr learned my addr 172.21.15.203:0/2068069987 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:04:06.748 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f5895a87700 1 --2- 172.21.15.203:0/2068069987 >> v2:172.21.15.91:3300/0 conn(0x7f58900602b0 0x7f58900bb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.749 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f5895286700 1 -- 172.21.15.203:0/2068069987 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912245545 0 0) 0x7f5890068780 con 0x7f5890148320 2021-01-06T22:04:06.749 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f5895286700 1 -- 172.21.15.203:0/2068069987 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5878003230 con 0x7f5890148320 2021-01-06T22:04:06.749 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f5895a87700 1 -- 172.21.15.203:0/2068069987 >> v1:172.21.15.203:6789/0 conn(0x7f5890148320 legacy=0x7f58900626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f5895a87700 1 -- 172.21.15.203:0/2068069987 >> v1:172.21.15.91:6789/0 conn(0x7f5890057060 legacy=0x7f5890158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f5895a87700 1 -- 172.21.15.203:0/2068069987 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589014e170 con 0x7f58900602b0 2021-01-06T22:04:06.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f5895a87700 1 --2- 172.21.15.203:0/2068069987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58900602b0 0x7f58900bb650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f588c00b980 tx=0x7f588c006de0).ready entity=mon.0 client_cookie=bd8d87c437659bed server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:06.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f5895286700 1 -- 172.21.15.203:0/2068069987 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c0097b0 con 0x7f58900602b0 2021-01-06T22:04:06.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f5895286700 1 -- 172.21.15.203:0/2068069987 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588c009910 con 0x7f58900602b0 2021-01-06T22:04:06.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f5895286700 1 -- 172.21.15.203:0/2068069987 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c009c90 con 0x7f58900602b0 2021-01-06T22:04:06.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f58984ec700 1 -- 172.21.15.203:0/2068069987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58900602b0 msgr2=0x7f58900bb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f58984ec700 1 --2- 172.21.15.203:0/2068069987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58900602b0 0x7f58900bb650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f588c00b980 tx=0x7f588c006de0).stop 2021-01-06T22:04:06.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f58984ec700 1 -- 172.21.15.203:0/2068069987 shutdown_connections 2021-01-06T22:04:06.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f58984ec700 1 --2- 172.21.15.203:0/2068069987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58900602b0 0x7f58900bb650 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f58984ec700 1 -- 172.21.15.203:0/2068069987 >> 172.21.15.203:0/2068069987 conn(0x7f58900c0cb0 msgr2=0x7f58900c1090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:06.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f58984ec700 1 -- 172.21.15.203:0/2068069987 shutdown_connections 2021-01-06T22:04:06.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f58984ec700 1 -- 172.21.15.203:0/2068069987 wait complete. 2021-01-06T22:04:06.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f58984ec700 1 Processor -- start 2021-01-06T22:04:06.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.747+0000 7f58984ec700 1 -- start start 2021-01-06T22:04:06.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f58984ec700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5890057060 0x7f5890143f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f58984ec700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58900602b0 0x7f5890161d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f58984ec700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5890148320 0x7f58900b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f58984ec700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5890144450 con 0x7f58900602b0 2021-01-06T22:04:06.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f58984ec700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58900b3080 con 0x7f5890057060 2021-01-06T22:04:06.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f58984ec700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58901621f0 con 0x7f5890148320 2021-01-06T22:04:06.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f5896288700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5890057060 0x7f5890143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f5896288700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5890057060 0x7f5890143f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49478/0 (socket says 172.21.15.203:49478) 2021-01-06T22:04:06.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f5896288700 1 -- 172.21.15.203:0/3052300856 learned_addr learned my addr 172.21.15.203:0/3052300856 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:06.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f5895a87700 1 --2- 172.21.15.203:0/3052300856 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58900602b0 0x7f5890161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f5896a89700 1 --2- 172.21.15.203:0/3052300856 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5890148320 0x7f58900b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f5896288700 1 -- 172.21.15.203:0/3052300856 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5890148320 msgr2=0x7f58900b7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f5896288700 1 --2- 172.21.15.203:0/3052300856 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5890148320 0x7f58900b7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f5896288700 1 -- 172.21.15.203:0/3052300856 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58900602b0 msgr2=0x7f5890161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f5896288700 1 --2- 172.21.15.203:0/3052300856 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58900602b0 0x7f5890161d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f5896288700 1 -- 172.21.15.203:0/3052300856 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58900c0150 con 0x7f5890057060 2021-01-06T22:04:06.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f5896288700 1 --2- 172.21.15.203:0/3052300856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5890057060 0x7f5890143f60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f588400c740 tx=0x7f5884007ac0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:06.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f58827fc700 1 -- 172.21.15.203:0/3052300856 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58840072e0 con 0x7f5890057060 2021-01-06T22:04:06.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f58827fc700 1 -- 172.21.15.203:0/3052300856 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5884007440 con 0x7f5890057060 2021-01-06T22:04:06.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f58984ec700 1 -- 172.21.15.203:0/3052300856 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58900b32e0 con 0x7f5890057060 2021-01-06T22:04:06.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f58827fc700 1 -- 172.21.15.203:0/3052300856 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58840057f0 con 0x7f5890057060 2021-01-06T22:04:06.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f58984ec700 1 -- 172.21.15.203:0/3052300856 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58900b3820 con 0x7f5890057060 2021-01-06T22:04:06.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.751+0000 7f58984ec700 1 -- 172.21.15.203:0/3052300856 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f589014edd0 con 0x7f5890057060 2021-01-06T22:04:06.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.755+0000 7f58827fc700 1 -- 172.21.15.203:0/3052300856 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f588401a060 con 0x7f5890057060 2021-01-06T22:04:06.760 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.755+0000 7f58827fc700 1 --2- 172.21.15.203:0/3052300856 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58700563d0 0x7f5870058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:06.760 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.755+0000 7f58827fc700 1 -- 172.21.15.203:0/3052300856 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f588406ed20 con 0x7f5890057060 2021-01-06T22:04:06.760 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.755+0000 7f5895a87700 1 --2- 172.21.15.203:0/3052300856 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58700563d0 0x7f5870058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:06.760 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.755+0000 7f5895a87700 1 --2- 172.21.15.203:0/3052300856 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58700563d0 0x7f5870058890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f588c00b130 tx=0x7f588c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:06.761 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.759+0000 7f58827fc700 1 -- 172.21.15.203:0/3052300856 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f5884044490 con 0x7f5890057060 2021-01-06T22:04:06.910 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.907+0000 7f58984ec700 1 -- 172.21.15.203:0/3052300856 --> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58900b41b0 con 0x7f58700563d0 2021-01-06T22:04:06.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.915+0000 7f58827fc700 1 -- 172.21.15.203:0/3052300856 <== mgr.4102 v2:172.21.15.203:6800/18146 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82257 (secure 0 0 0) 0x7f58900b41b0 con 0x7f58700563d0 2021-01-06T22:04:06.919 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:06.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.919+0000 7f58984ec700 1 -- 172.21.15.203:0/3052300856 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58700563d0 msgr2=0x7f5870058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.919+0000 7f58984ec700 1 --2- 172.21.15.203:0/3052300856 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58700563d0 0x7f5870058890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f588c00b130 tx=0x7f588c016000).stop 2021-01-06T22:04:06.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.919+0000 7f58984ec700 1 -- 172.21.15.203:0/3052300856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5890057060 msgr2=0x7f5890143f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:06.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.919+0000 7f58984ec700 1 --2- 172.21.15.203:0/3052300856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5890057060 0x7f5890143f60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f588400c740 tx=0x7f5884007ac0).stop 2021-01-06T22:04:06.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.919+0000 7f58984ec700 1 -- 172.21.15.203:0/3052300856 shutdown_connections 2021-01-06T22:04:06.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.919+0000 7f58984ec700 1 --2- 172.21.15.203:0/3052300856 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58700563d0 0x7f5870058890 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.919+0000 7f58984ec700 1 --2- 172.21.15.203:0/3052300856 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5890148320 0x7f58900b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.919+0000 7f58984ec700 1 --2- 172.21.15.203:0/3052300856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5890057060 0x7f5890143f60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.919+0000 7f58984ec700 1 --2- 172.21.15.203:0/3052300856 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58900602b0 0x7f5890161d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:06.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.919+0000 7f58984ec700 1 -- 172.21.15.203:0/3052300856 >> 172.21.15.203:0/3052300856 conn(0x7f58900c0cb0 msgr2=0x7f58900626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:06.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.919+0000 7f58984ec700 1 -- 172.21.15.203:0/3052300856 shutdown_connections 2021-01-06T22:04:06.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:06.919+0000 7f58984ec700 1 -- 172.21.15.203:0/3052300856 wait complete. 2021-01-06T22:04:06.923 INFO:teuthology.orchestra.run.smithi203.stderr:dumped all 2021-01-06T22:04:06.935 INFO:teuthology.orchestra.run.smithi203.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2021-01-06T22:04:05.098797+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4962,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":40988,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":38912,"kb_avail":754933732,"statfs":{"total":773094113280,"available":773052141568,"internally_reserved":0,"allocated":2060288,"data_stored":741074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39845888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004021"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.758566+0000","last_change":"2021-01-06T22:03:39.757200+0000","last_active":"2021-01-06T22:03:39.758566+0000","last_peered":"2021-01-06T22:03:39.758566+0000","last_clean":"2021-01-06T22:03:39.758566+0000","last_became_active":"2021-01-06T22:03:39.756706+0000","last_became_peered":"2021-01-06T22:03:39.756706+0000","last_unstale":"2021-01-06T22:03:39.758566+0000","last_undegraded":"2021-01-06T22:03:39.758566+0000","last_fullsized":"2021-01-06T22:03:39.758566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.755461+0000","last_change":"2021-01-06T22:03:36.733731+0000","last_active":"2021-01-06T22:03:39.755461+0000","last_peered":"2021-01-06T22:03:39.755461+0000","last_clean":"2021-01-06T22:03:39.755461+0000","last_became_active":"2021-01-06T22:03:36.733026+0000","last_became_peered":"2021-01-06T22:03:36.733026+0000","last_unstale":"2021-01-06T22:03:39.755461+0000","last_undegraded":"2021-01-06T22:03:39.755461+0000","last_fullsized":"2021-01-06T22:03:39.755461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"23'10","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.806682+0000","last_change":"2021-01-06T22:03:39.372339+0000","last_active":"2021-01-06T22:03:43.806682+0000","last_peered":"2021-01-06T22:03:43.806682+0000","last_clean":"2021-01-06T22:03:43.806682+0000","last_became_active":"2021-01-06T22:03:39.372000+0000","last_became_peered":"2021-01-06T22:03:39.372000+0000","last_unstale":"2021-01-06T22:03:43.806682+0000","last_undegraded":"2021-01-06T22:03:43.806682+0000","last_fullsized":"2021-01-06T22:03:43.806682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":735,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.758556+0000","last_change":"2021-01-06T22:03:39.757152+0000","last_active":"2021-01-06T22:03:39.758556+0000","last_peered":"2021-01-06T22:03:39.758556+0000","last_clean":"2021-01-06T22:03:39.758556+0000","last_became_active":"2021-01-06T22:03:39.756673+0000","last_became_peered":"2021-01-06T22:03:39.756673+0000","last_unstale":"2021-01-06T22:03:39.758556+0000","last_undegraded":"2021-01-06T22:03:39.758556+0000","last_fullsized":"2021-01-06T22:03:39.758556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747142+0000","last_change":"2021-01-06T22:03:36.731052+0000","last_active":"2021-01-06T22:03:39.747142+0000","last_peered":"2021-01-06T22:03:39.747142+0000","last_clean":"2021-01-06T22:03:39.747142+0000","last_became_active":"2021-01-06T22:03:36.730252+0000","last_became_peered":"2021-01-06T22:03:36.730252+0000","last_unstale":"2021-01-06T22:03:39.747142+0000","last_undegraded":"2021-01-06T22:03:39.747142+0000","last_fullsized":"2021-01-06T22:03:39.747142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"23'18","reported_seq":"28","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.807276+0000","last_change":"2021-01-06T22:03:39.371415+0000","last_active":"2021-01-06T22:03:43.807276+0000","last_peered":"2021-01-06T22:03:43.807276+0000","last_clean":"2021-01-06T22:03:43.807276+0000","last_became_active":"2021-01-06T22:03:39.370981+0000","last_became_peered":"2021-01-06T22:03:39.370981+0000","last_unstale":"2021-01-06T22:03:43.807276+0000","last_undegraded":"2021-01-06T22:03:43.807276+0000","last_fullsized":"2021-01-06T22:03:43.807276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":642,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.776586+0000","last_change":"2021-01-06T22:03:39.775845+0000","last_active":"2021-01-06T22:03:39.776586+0000","last_peered":"2021-01-06T22:03:39.776586+0000","last_clean":"2021-01-06T22:03:39.776586+0000","last_became_active":"2021-01-06T22:03:39.775329+0000","last_became_peered":"2021-01-06T22:03:39.775329+0000","last_unstale":"2021-01-06T22:03:39.776586+0000","last_undegraded":"2021-01-06T22:03:39.776586+0000","last_fullsized":"2021-01-06T22:03:39.776586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.751998+0000","last_change":"2021-01-06T22:03:36.209909+0000","last_active":"2021-01-06T22:03:39.751998+0000","last_peered":"2021-01-06T22:03:39.751998+0000","last_clean":"2021-01-06T22:03:39.751998+0000","last_became_active":"2021-01-06T22:03:36.209405+0000","last_became_peered":"2021-01-06T22:03:36.209405+0000","last_unstale":"2021-01-06T22:03:39.751998+0000","last_undegraded":"2021-01-06T22:03:39.751998+0000","last_fullsized":"2021-01-06T22:03:39.751998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"23'21","reported_seq":"31","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.819440+0000","last_change":"2021-01-06T22:03:39.374989+0000","last_active":"2021-01-06T22:03:43.819440+0000","last_peered":"2021-01-06T22:03:43.819440+0000","last_clean":"2021-01-06T22:03:43.819440+0000","last_became_active":"2021-01-06T22:03:39.374506+0000","last_became_peered":"2021-01-06T22:03:39.374506+0000","last_unstale":"2021-01-06T22:03:43.819440+0000","last_undegraded":"2021-01-06T22:03:43.819440+0000","last_fullsized":"2021-01-06T22:03:43.819440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":112,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:40.227173+0000","last_change":"2021-01-06T22:03:40.226400+0000","last_active":"2021-01-06T22:03:40.227173+0000","last_peered":"2021-01-06T22:03:40.227173+0000","last_clean":"2021-01-06T22:03:40.227173+0000","last_became_active":"2021-01-06T22:03:40.225932+0000","last_became_peered":"2021-01-06T22:03:40.225932+0000","last_unstale":"2021-01-06T22:03:40.227173+0000","last_undegraded":"2021-01-06T22:03:40.227173+0000","last_fullsized":"2021-01-06T22:03:40.227173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747380+0000","last_change":"2021-01-06T22:03:36.243205+0000","last_active":"2021-01-06T22:03:39.747380+0000","last_peered":"2021-01-06T22:03:39.747380+0000","last_clean":"2021-01-06T22:03:39.747380+0000","last_became_active":"2021-01-06T22:03:36.242610+0000","last_became_peered":"2021-01-06T22:03:36.242610+0000","last_unstale":"2021-01-06T22:03:39.747380+0000","last_undegraded":"2021-01-06T22:03:39.747380+0000","last_fullsized":"2021-01-06T22:03:39.747380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"17","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:40.222155+0000","last_change":"2021-01-06T22:03:32.172575+0000","last_active":"2021-01-06T22:03:40.222155+0000","last_peered":"2021-01-06T22:03:40.222155+0000","last_clean":"2021-01-06T22:03:40.222155+0000","last_became_active":"2021-01-06T22:03:32.171025+0000","last_became_peered":"2021-01-06T22:03:32.171025+0000","last_unstale":"2021-01-06T22:03:40.222155+0000","last_undegraded":"2021-01-06T22:03:40.222155+0000","last_fullsized":"2021-01-06T22:03:40.222155+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":8,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:25.162227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:25.162227+0000","last_clean_scrub_stamp":"2021-01-06T22:03:25.162227+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"23'8","reported_seq":"18","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.815252+0000","last_change":"2021-01-06T22:03:39.374385+0000","last_active":"2021-01-06T22:03:43.815252+0000","last_peered":"2021-01-06T22:03:43.815252+0000","last_clean":"2021-01-06T22:03:43.815252+0000","last_became_active":"2021-01-06T22:03:39.374029+0000","last_became_peered":"2021-01-06T22:03:39.374029+0000","last_unstale":"2021-01-06T22:03:43.815252+0000","last_undegraded":"2021-01-06T22:03:43.815252+0000","last_fullsized":"2021-01-06T22:03:43.815252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.765901+0000","last_change":"2021-01-06T22:03:39.762496+0000","last_active":"2021-01-06T22:03:39.765901+0000","last_peered":"2021-01-06T22:03:39.765901+0000","last_clean":"2021-01-06T22:03:39.765901+0000","last_became_active":"2021-01-06T22:03:39.761903+0000","last_became_peered":"2021-01-06T22:03:39.761903+0000","last_unstale":"2021-01-06T22:03:39.765901+0000","last_undegraded":"2021-01-06T22:03:39.765901+0000","last_fullsized":"2021-01-06T22:03:39.765901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747295+0000","last_change":"2021-01-06T22:03:36.206609+0000","last_active":"2021-01-06T22:03:39.747295+0000","last_peered":"2021-01-06T22:03:39.747295+0000","last_clean":"2021-01-06T22:03:39.747295+0000","last_became_active":"2021-01-06T22:03:36.206123+0000","last_became_peered":"2021-01-06T22:03:36.206123+0000","last_unstale":"2021-01-06T22:03:39.747295+0000","last_undegraded":"2021-01-06T22:03:39.747295+0000","last_fullsized":"2021-01-06T22:03:39.747295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"23'12","reported_seq":"22","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.821871+0000","last_change":"2021-01-06T22:03:39.377897+0000","last_active":"2021-01-06T22:03:43.821871+0000","last_peered":"2021-01-06T22:03:43.821871+0000","last_clean":"2021-01-06T22:03:43.821871+0000","last_became_active":"2021-01-06T22:03:39.376836+0000","last_became_peered":"2021-01-06T22:03:39.376836+0000","last_unstale":"2021-01-06T22:03:43.821871+0000","last_undegraded":"2021-01-06T22:03:43.821871+0000","last_fullsized":"2021-01-06T22:03:43.821871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":720,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.778763+0000","last_change":"2021-01-06T22:03:39.777501+0000","last_active":"2021-01-06T22:03:39.778763+0000","last_peered":"2021-01-06T22:03:39.778763+0000","last_clean":"2021-01-06T22:03:39.778763+0000","last_became_active":"2021-01-06T22:03:39.777158+0000","last_became_peered":"2021-01-06T22:03:39.777158+0000","last_unstale":"2021-01-06T22:03:39.778763+0000","last_undegraded":"2021-01-06T22:03:39.778763+0000","last_fullsized":"2021-01-06T22:03:39.778763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747656+0000","last_change":"2021-01-06T22:03:36.208974+0000","last_active":"2021-01-06T22:03:39.747656+0000","last_peered":"2021-01-06T22:03:39.747656+0000","last_clean":"2021-01-06T22:03:39.747656+0000","last_became_active":"2021-01-06T22:03:36.208576+0000","last_became_peered":"2021-01-06T22:03:36.208576+0000","last_unstale":"2021-01-06T22:03:39.747656+0000","last_undegraded":"2021-01-06T22:03:39.747656+0000","last_fullsized":"2021-01-06T22:03:39.747656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"23'13","reported_seq":"23","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.816002+0000","last_change":"2021-01-06T22:03:39.375127+0000","last_active":"2021-01-06T22:03:43.816002+0000","last_peered":"2021-01-06T22:03:43.816002+0000","last_clean":"2021-01-06T22:03:43.816002+0000","last_became_active":"2021-01-06T22:03:39.374751+0000","last_became_peered":"2021-01-06T22:03:39.374751+0000","last_unstale":"2021-01-06T22:03:43.816002+0000","last_undegraded":"2021-01-06T22:03:43.816002+0000","last_fullsized":"2021-01-06T22:03:43.816002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2088,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:40.232168+0000","last_change":"2021-01-06T22:03:40.230887+0000","last_active":"2021-01-06T22:03:40.232168+0000","last_peered":"2021-01-06T22:03:40.232168+0000","last_clean":"2021-01-06T22:03:40.232168+0000","last_became_active":"2021-01-06T22:03:40.230100+0000","last_became_peered":"2021-01-06T22:03:40.230100+0000","last_unstale":"2021-01-06T22:03:40.232168+0000","last_undegraded":"2021-01-06T22:03:40.232168+0000","last_fullsized":"2021-01-06T22:03:40.232168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"23'18","reported_seq":"28","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:46.596414+0000","last_change":"2021-01-06T22:03:39.377489+0000","last_active":"2021-01-06T22:03:46.596414+0000","last_peered":"2021-01-06T22:03:46.596414+0000","last_clean":"2021-01-06T22:03:46.596414+0000","last_became_active":"2021-01-06T22:03:39.377092+0000","last_became_peered":"2021-01-06T22:03:39.377092+0000","last_unstale":"2021-01-06T22:03:46.596414+0000","last_undegraded":"2021-01-06T22:03:46.596414+0000","last_fullsized":"2021-01-06T22:03:46.596414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":665,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.759458+0000","last_change":"2021-01-06T22:03:36.213113+0000","last_active":"2021-01-06T22:03:39.759458+0000","last_peered":"2021-01-06T22:03:39.759458+0000","last_clean":"2021-01-06T22:03:39.759458+0000","last_became_active":"2021-01-06T22:03:36.212790+0000","last_became_peered":"2021-01-06T22:03:36.212790+0000","last_unstale":"2021-01-06T22:03:39.759458+0000","last_undegraded":"2021-01-06T22:03:39.759458+0000","last_fullsized":"2021-01-06T22:03:39.759458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.764030+0000","last_change":"2021-01-06T22:03:39.759582+0000","last_active":"2021-01-06T22:03:39.764030+0000","last_peered":"2021-01-06T22:03:39.764030+0000","last_clean":"2021-01-06T22:03:39.764030+0000","last_became_active":"2021-01-06T22:03:39.759212+0000","last_became_peered":"2021-01-06T22:03:39.759212+0000","last_unstale":"2021-01-06T22:03:39.764030+0000","last_undegraded":"2021-01-06T22:03:39.764030+0000","last_fullsized":"2021-01-06T22:03:39.764030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747367+0000","last_change":"2021-01-06T22:03:36.207481+0000","last_active":"2021-01-06T22:03:39.747367+0000","last_peered":"2021-01-06T22:03:39.747367+0000","last_clean":"2021-01-06T22:03:39.747367+0000","last_became_active":"2021-01-06T22:03:36.206991+0000","last_became_peered":"2021-01-06T22:03:36.206991+0000","last_unstale":"2021-01-06T22:03:39.747367+0000","last_undegraded":"2021-01-06T22:03:39.747367+0000","last_fullsized":"2021-01-06T22:03:39.747367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'9","reported_seq":"19","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.813735+0000","last_change":"2021-01-06T22:03:39.370350+0000","last_active":"2021-01-06T22:03:43.813735+0000","last_peered":"2021-01-06T22:03:43.813735+0000","last_clean":"2021-01-06T22:03:43.813735+0000","last_became_active":"2021-01-06T22:03:39.369929+0000","last_became_peered":"2021-01-06T22:03:39.369929+0000","last_unstale":"2021-01-06T22:03:43.813735+0000","last_undegraded":"2021-01-06T22:03:43.813735+0000","last_fullsized":"2021-01-06T22:03:43.813735+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":4962,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":10367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5112,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366728,"statfs":{"total":96636764160,"available":96631529472,"internally_reserved":0,"allocated":245760,"data_stored":91578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5470000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8879999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3029999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3090000000000002}]}]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5224,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366616,"statfs":{"total":96636764160,"available":96631414784,"internally_reserved":0,"allocated":294912,"data_stored":95674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3580000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1520000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4060000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2149999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.097}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0430000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5128,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366712,"statfs":{"total":96636764160,"available":96631513088,"internally_reserved":0,"allocated":262144,"data_stored":93666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4220000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9550000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.524}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9420000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":512.53099999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8079999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5116,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366724,"statfs":{"total":96636764160,"available":96631525376,"internally_reserved":0,"allocated":249856,"data_stored":91530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1659999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4049999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.552}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9449999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0990000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0329999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":237568,"data_stored":91466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3650000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0059999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.137}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.645}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6930000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8959999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2480000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5148,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366692,"statfs":{"total":96636764160,"available":96631492608,"internally_reserved":0,"allocated":282624,"data_stored":93563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.173}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3839999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8439999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2719999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6040000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6669999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5070000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5104,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366736,"statfs":{"total":96636764160,"available":96631537664,"internally_reserved":0,"allocated":237568,"data_stored":91466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0539999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2450000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4289999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1749999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3719999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5116,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366724,"statfs":{"total":96636764160,"available":96631525376,"internally_reserved":0,"allocated":249856,"data_stored":92131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.865}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.774}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7330000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.024}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.552}]}]}]}} 2021-01-06T22:04:06.937 INFO:tasks.mds_thrash.ceph_manager:clean! 2021-01-06T22:04:06.938 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-06T22:04:07.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f24502c3700 1 Processor -- start 2021-01-06T22:04:07.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f24502c3700 1 -- start start 2021-01-06T22:04:07.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f24502c3700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f24480679a0 0x7f24480be300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f24502c3700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24480b8d60 con 0x7f244814fd40 2021-01-06T22:04:07.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f24502c3700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24480b8f90 con 0x7f24480bdf20 2021-01-06T22:04:07.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f24502c3700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24480b9230 con 0x7f24480679a0 2021-01-06T22:04:07.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244e05f700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f24480bdf20 0x7f244814f630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48080/0 (socket says 172.21.15.203:48080) 2021-01-06T22:04:07.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244e05f700 1 -- 172.21.15.203:0/1993192030 learned_addr learned my addr 172.21.15.203:0/1993192030 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:04:07.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d05d700 1 -- 172.21.15.203:0/1993192030 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942715503 0 0) 0x7f24480b8f90 con 0x7f24480bdf20 2021-01-06T22:04:07.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244e860700 1 --2- 172.21.15.203:0/1993192030 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f24480679a0 0x7f24480be300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d05d700 1 -- 172.21.15.203:0/1993192030 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2430003230 con 0x7f24480bdf20 2021-01-06T22:04:07.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d05d700 1 -- 172.21.15.203:0/1993192030 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2591230534 0 0) 0x7f2430003230 con 0x7f24480bdf20 2021-01-06T22:04:07.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d05d700 1 -- 172.21.15.203:0/1993192030 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f24480679a0 msgr2=0x7f24480be300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:07.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d05d700 1 --2- 172.21.15.203:0/1993192030 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f24480679a0 0x7f24480be300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d05d700 1 -- 172.21.15.203:0/1993192030 >> v1:172.21.15.91:6789/0 conn(0x7f244814fd40 legacy=0x7f244815af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d05d700 1 -- 172.21.15.203:0/1993192030 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2448155c10 con 0x7f24480bdf20 2021-01-06T22:04:07.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d05d700 1 -- 172.21.15.203:0/1993192030 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f24440014e0 con 0x7f24480bdf20 2021-01-06T22:04:07.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d05d700 1 -- 172.21.15.203:0/1993192030 >> v1:172.21.15.203:6789/0 conn(0x7f24480bdf20 legacy=0x7f244814f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d05d700 1 --2- 172.21.15.203:0/1993192030 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2430003720 0x7f2430003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d05d700 1 --2- 172.21.15.203:0/1993192030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2430004060 0x7f2430004530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d05d700 1 --2- 172.21.15.203:0/1993192030 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f24480679a0 0x7f2430004b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d05d700 1 -- 172.21.15.203:0/1993192030 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24480b9230 con 0x7f2430003720 2021-01-06T22:04:07.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d05d700 1 -- 172.21.15.203:0/1993192030 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2430005100 con 0x7f2430004060 2021-01-06T22:04:07.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d05d700 1 -- 172.21.15.203:0/1993192030 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24300052c0 con 0x7f24480679a0 2021-01-06T22:04:07.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d85e700 1 --2- 172.21.15.203:0/1993192030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2430004060 0x7f2430004530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244e05f700 1 --2- 172.21.15.203:0/1993192030 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2430003720 0x7f2430003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244e860700 1 --2- 172.21.15.203:0/1993192030 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f24480679a0 0x7f2430004b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d85e700 1 -- 172.21.15.203:0/1993192030 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f24480679a0 msgr2=0x7f2430004b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:07.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.063+0000 7f244d85e700 1 --2- 172.21.15.203:0/1993192030 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f24480679a0 0x7f2430004b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.067+0000 7f244d85e700 1 -- 172.21.15.203:0/1993192030 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2430003720 msgr2=0x7f2430003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:07.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.067+0000 7f244d85e700 1 --2- 172.21.15.203:0/1993192030 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2430003720 0x7f2430003b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.067+0000 7f244d85e700 1 -- 172.21.15.203:0/1993192030 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2430003230 con 0x7f2430004060 2021-01-06T22:04:07.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.067+0000 7f244d85e700 1 --2- 172.21.15.203:0/1993192030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2430004060 0x7f2430004530 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f2438001a00 tx=0x7f2438007b60).ready entity=mon.1 client_cookie=3e6b2ce662f5b174 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:07.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.067+0000 7f244d05d700 1 -- 172.21.15.203:0/1993192030 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800c830 con 0x7f2430004060 2021-01-06T22:04:07.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.067+0000 7f244d05d700 1 -- 172.21.15.203:0/1993192030 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243800c990 con 0x7f2430004060 2021-01-06T22:04:07.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.067+0000 7f244d05d700 1 -- 172.21.15.203:0/1993192030 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800cd50 con 0x7f2430004060 2021-01-06T22:04:07.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.067+0000 7f24502c3700 1 -- 172.21.15.203:0/1993192030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2430004060 msgr2=0x7f2430004530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.067+0000 7f24502c3700 1 --2- 172.21.15.203:0/1993192030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2430004060 0x7f2430004530 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f2438001a00 tx=0x7f2438007b60).stop 2021-01-06T22:04:07.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.067+0000 7f244e860700 1 -- 172.21.15.203:0/1993192030 reap_dead start 2021-01-06T22:04:07.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.067+0000 7f24502c3700 1 -- 172.21.15.203:0/1993192030 shutdown_connections 2021-01-06T22:04:07.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.067+0000 7f24502c3700 1 -- 172.21.15.203:0/1993192030 >> 172.21.15.203:0/1993192030 conn(0x7f2448056cd0 msgr2=0x7f24480570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:07.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.067+0000 7f24502c3700 1 -- 172.21.15.203:0/1993192030 shutdown_connections 2021-01-06T22:04:07.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.067+0000 7f24502c3700 1 -- 172.21.15.203:0/1993192030 wait complete. 2021-01-06T22:04:07.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.067+0000 7f24502c3700 1 Processor -- start 2021-01-06T22:04:07.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.067+0000 7f24502c3700 1 -- start start 2021-01-06T22:04:07.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f24502c3700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2448157740 0x7f24480bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f24502c3700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24481518d0 0x7f24481495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f24502c3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2448157b20 0x7f2448149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f24502c3700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24480c0390 con 0x7f24481518d0 2021-01-06T22:04:07.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f24502c3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2448162fa0 con 0x7f2448157b20 2021-01-06T22:04:07.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f24502c3700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2448163240 con 0x7f2448157740 2021-01-06T22:04:07.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f244e860700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2448157b20 0x7f2448149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f244e05f700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2448157740 0x7f24480bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f244e860700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2448157b20 0x7f2448149ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49500/0 (socket says 172.21.15.203:49500) 2021-01-06T22:04:07.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f244e860700 1 -- 172.21.15.203:0/1627591543 learned_addr learned my addr 172.21.15.203:0/1627591543 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:07.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f244d85e700 1 --2- 172.21.15.203:0/1627591543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24481518d0 0x7f24481495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f244e860700 1 -- 172.21.15.203:0/1627591543 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2448157740 msgr2=0x7f24480bfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f244e860700 1 --2- 172.21.15.203:0/1627591543 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2448157740 0x7f24480bfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f244e860700 1 -- 172.21.15.203:0/1627591543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24481518d0 msgr2=0x7f24481495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f244e860700 1 --2- 172.21.15.203:0/1627591543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24481518d0 0x7f24481495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f244e860700 1 -- 172.21.15.203:0/1627591543 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24480bf4c0 con 0x7f2448157b20 2021-01-06T22:04:07.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f244e860700 1 --2- 172.21.15.203:0/1627591543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2448157b20 0x7f2448149ac0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f24400090e0 tx=0x7f24400091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:07.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f24367fc700 1 -- 172.21.15.203:0/1627591543 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244000fa40 con 0x7f2448157b20 2021-01-06T22:04:07.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f24367fc700 1 -- 172.21.15.203:0/1627591543 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244000fba0 con 0x7f2448157b20 2021-01-06T22:04:07.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f24502c3700 1 -- 172.21.15.203:0/1627591543 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24480bf760 con 0x7f2448157b20 2021-01-06T22:04:07.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f24367fc700 1 -- 172.21.15.203:0/1627591543 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2440014440 con 0x7f2448157b20 2021-01-06T22:04:07.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.071+0000 7f24502c3700 1 -- 172.21.15.203:0/1627591543 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2448149fb0 con 0x7f2448157b20 2021-01-06T22:04:07.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.075+0000 7f24502c3700 1 -- 172.21.15.203:0/1627591543 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244805d600 con 0x7f2448157b20 2021-01-06T22:04:07.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.075+0000 7f24367fc700 1 -- 172.21.15.203:0/1627591543 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f2440014870 con 0x7f2448157b20 2021-01-06T22:04:07.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.075+0000 7f24367fc700 1 --2- 172.21.15.203:0/1627591543 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2428056400 0x7f24280588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.075+0000 7f24367fc700 1 -- 172.21.15.203:0/1627591543 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2440070140 con 0x7f2448157b20 2021-01-06T22:04:07.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.075+0000 7f244e05f700 1 --2- 172.21.15.203:0/1627591543 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2428056400 0x7f24280588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.075+0000 7f244e05f700 1 --2- 172.21.15.203:0/1627591543 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2428056400 0x7f24280588c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f2444000e10 tx=0x7f24440077c0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:07.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.083+0000 7f24367fc700 1 -- 172.21.15.203:0/1627591543 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f24400458b0 con 0x7f2448157b20 2021-01-06T22:04:07.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.231+0000 7f24502c3700 1 -- 172.21.15.203:0/1627591543 --> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24481559f0 con 0x7f2428056400 2021-01-06T22:04:07.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.239+0000 7f24367fc700 1 -- 172.21.15.203:0/1627591543 <== mgr.4102 v2:172.21.15.203:6800/18146 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82258 (secure 0 0 0) 0x7f24481559f0 con 0x7f2428056400 2021-01-06T22:04:07.241 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:07.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.239+0000 7f24502c3700 1 -- 172.21.15.203:0/1627591543 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2428056400 msgr2=0x7f24280588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.239+0000 7f24502c3700 1 --2- 172.21.15.203:0/1627591543 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2428056400 0x7f24280588c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f2444000e10 tx=0x7f24440077c0).stop 2021-01-06T22:04:07.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.239+0000 7f24502c3700 1 -- 172.21.15.203:0/1627591543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2448157b20 msgr2=0x7f2448149ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.239+0000 7f24502c3700 1 --2- 172.21.15.203:0/1627591543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2448157b20 0x7f2448149ac0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f24400090e0 tx=0x7f24400091d0).stop 2021-01-06T22:04:07.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.239+0000 7f24502c3700 1 -- 172.21.15.203:0/1627591543 shutdown_connections 2021-01-06T22:04:07.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.239+0000 7f24502c3700 1 --2- 172.21.15.203:0/1627591543 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2428056400 0x7f24280588c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.239+0000 7f24502c3700 1 --2- 172.21.15.203:0/1627591543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2448157b20 0x7f2448149ac0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.239+0000 7f24502c3700 1 --2- 172.21.15.203:0/1627591543 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2448157740 0x7f24480bfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.239+0000 7f24502c3700 1 --2- 172.21.15.203:0/1627591543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24481518d0 0x7f24481495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.239+0000 7f24502c3700 1 -- 172.21.15.203:0/1627591543 >> 172.21.15.203:0/1627591543 conn(0x7f2448056cd0 msgr2=0x7f24480602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:07.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.239+0000 7f24502c3700 1 -- 172.21.15.203:0/1627591543 shutdown_connections 2021-01-06T22:04:07.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.239+0000 7f24502c3700 1 -- 172.21.15.203:0/1627591543 wait complete. 2021-01-06T22:04:07.245 INFO:teuthology.orchestra.run.smithi203.stderr:dumped all 2021-01-06T22:04:07.258 INFO:teuthology.orchestra.run.smithi203.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2021-01-06T22:04:07.099354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5405,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41052,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":38976,"kb_avail":754933668,"statfs":{"total":773094113280,"available":773052076032,"internally_reserved":0,"allocated":2060288,"data_stored":741960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39911424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":443,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003832"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.758566+0000","last_change":"2021-01-06T22:03:39.757200+0000","last_active":"2021-01-06T22:03:39.758566+0000","last_peered":"2021-01-06T22:03:39.758566+0000","last_clean":"2021-01-06T22:03:39.758566+0000","last_became_active":"2021-01-06T22:03:39.756706+0000","last_became_peered":"2021-01-06T22:03:39.756706+0000","last_unstale":"2021-01-06T22:03:39.758566+0000","last_undegraded":"2021-01-06T22:03:39.758566+0000","last_fullsized":"2021-01-06T22:03:39.758566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.755461+0000","last_change":"2021-01-06T22:03:36.733731+0000","last_active":"2021-01-06T22:03:39.755461+0000","last_peered":"2021-01-06T22:03:39.755461+0000","last_clean":"2021-01-06T22:03:39.755461+0000","last_became_active":"2021-01-06T22:03:36.733026+0000","last_became_peered":"2021-01-06T22:03:36.733026+0000","last_unstale":"2021-01-06T22:03:39.755461+0000","last_undegraded":"2021-01-06T22:03:39.755461+0000","last_fullsized":"2021-01-06T22:03:39.755461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"23'10","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.806682+0000","last_change":"2021-01-06T22:03:39.372339+0000","last_active":"2021-01-06T22:03:43.806682+0000","last_peered":"2021-01-06T22:03:43.806682+0000","last_clean":"2021-01-06T22:03:43.806682+0000","last_became_active":"2021-01-06T22:03:39.372000+0000","last_became_peered":"2021-01-06T22:03:39.372000+0000","last_unstale":"2021-01-06T22:03:43.806682+0000","last_undegraded":"2021-01-06T22:03:43.806682+0000","last_fullsized":"2021-01-06T22:03:43.806682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":735,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.758556+0000","last_change":"2021-01-06T22:03:39.757152+0000","last_active":"2021-01-06T22:03:39.758556+0000","last_peered":"2021-01-06T22:03:39.758556+0000","last_clean":"2021-01-06T22:03:39.758556+0000","last_became_active":"2021-01-06T22:03:39.756673+0000","last_became_peered":"2021-01-06T22:03:39.756673+0000","last_unstale":"2021-01-06T22:03:39.758556+0000","last_undegraded":"2021-01-06T22:03:39.758556+0000","last_fullsized":"2021-01-06T22:03:39.758556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747142+0000","last_change":"2021-01-06T22:03:36.731052+0000","last_active":"2021-01-06T22:03:39.747142+0000","last_peered":"2021-01-06T22:03:39.747142+0000","last_clean":"2021-01-06T22:03:39.747142+0000","last_became_active":"2021-01-06T22:03:36.730252+0000","last_became_peered":"2021-01-06T22:03:36.730252+0000","last_unstale":"2021-01-06T22:03:39.747142+0000","last_undegraded":"2021-01-06T22:03:39.747142+0000","last_fullsized":"2021-01-06T22:03:39.747142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"23'19","reported_seq":"29","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.807539+0000","last_change":"2021-01-06T22:03:39.371415+0000","last_active":"2021-01-06T22:03:43.807539+0000","last_peered":"2021-01-06T22:03:43.807539+0000","last_clean":"2021-01-06T22:03:43.807539+0000","last_became_active":"2021-01-06T22:03:39.370981+0000","last_became_peered":"2021-01-06T22:03:39.370981+0000","last_unstale":"2021-01-06T22:03:43.807539+0000","last_undegraded":"2021-01-06T22:03:43.807539+0000","last_fullsized":"2021-01-06T22:03:43.807539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1085,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.776586+0000","last_change":"2021-01-06T22:03:39.775845+0000","last_active":"2021-01-06T22:03:39.776586+0000","last_peered":"2021-01-06T22:03:39.776586+0000","last_clean":"2021-01-06T22:03:39.776586+0000","last_became_active":"2021-01-06T22:03:39.775329+0000","last_became_peered":"2021-01-06T22:03:39.775329+0000","last_unstale":"2021-01-06T22:03:39.776586+0000","last_undegraded":"2021-01-06T22:03:39.776586+0000","last_fullsized":"2021-01-06T22:03:39.776586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.751998+0000","last_change":"2021-01-06T22:03:36.209909+0000","last_active":"2021-01-06T22:03:39.751998+0000","last_peered":"2021-01-06T22:03:39.751998+0000","last_clean":"2021-01-06T22:03:39.751998+0000","last_became_active":"2021-01-06T22:03:36.209405+0000","last_became_peered":"2021-01-06T22:03:36.209405+0000","last_unstale":"2021-01-06T22:03:39.751998+0000","last_undegraded":"2021-01-06T22:03:39.751998+0000","last_fullsized":"2021-01-06T22:03:39.751998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"23'21","reported_seq":"31","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.819440+0000","last_change":"2021-01-06T22:03:39.374989+0000","last_active":"2021-01-06T22:03:43.819440+0000","last_peered":"2021-01-06T22:03:43.819440+0000","last_clean":"2021-01-06T22:03:43.819440+0000","last_became_active":"2021-01-06T22:03:39.374506+0000","last_became_peered":"2021-01-06T22:03:39.374506+0000","last_unstale":"2021-01-06T22:03:43.819440+0000","last_undegraded":"2021-01-06T22:03:43.819440+0000","last_fullsized":"2021-01-06T22:03:43.819440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":112,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:40.227173+0000","last_change":"2021-01-06T22:03:40.226400+0000","last_active":"2021-01-06T22:03:40.227173+0000","last_peered":"2021-01-06T22:03:40.227173+0000","last_clean":"2021-01-06T22:03:40.227173+0000","last_became_active":"2021-01-06T22:03:40.225932+0000","last_became_peered":"2021-01-06T22:03:40.225932+0000","last_unstale":"2021-01-06T22:03:40.227173+0000","last_undegraded":"2021-01-06T22:03:40.227173+0000","last_fullsized":"2021-01-06T22:03:40.227173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747380+0000","last_change":"2021-01-06T22:03:36.243205+0000","last_active":"2021-01-06T22:03:39.747380+0000","last_peered":"2021-01-06T22:03:39.747380+0000","last_clean":"2021-01-06T22:03:39.747380+0000","last_became_active":"2021-01-06T22:03:36.242610+0000","last_became_peered":"2021-01-06T22:03:36.242610+0000","last_unstale":"2021-01-06T22:03:39.747380+0000","last_undegraded":"2021-01-06T22:03:39.747380+0000","last_fullsized":"2021-01-06T22:03:39.747380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"17","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:40.222155+0000","last_change":"2021-01-06T22:03:32.172575+0000","last_active":"2021-01-06T22:03:40.222155+0000","last_peered":"2021-01-06T22:03:40.222155+0000","last_clean":"2021-01-06T22:03:40.222155+0000","last_became_active":"2021-01-06T22:03:32.171025+0000","last_became_peered":"2021-01-06T22:03:32.171025+0000","last_unstale":"2021-01-06T22:03:40.222155+0000","last_undegraded":"2021-01-06T22:03:40.222155+0000","last_fullsized":"2021-01-06T22:03:40.222155+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":8,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:25.162227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:25.162227+0000","last_clean_scrub_stamp":"2021-01-06T22:03:25.162227+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"23'8","reported_seq":"18","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.815252+0000","last_change":"2021-01-06T22:03:39.374385+0000","last_active":"2021-01-06T22:03:43.815252+0000","last_peered":"2021-01-06T22:03:43.815252+0000","last_clean":"2021-01-06T22:03:43.815252+0000","last_became_active":"2021-01-06T22:03:39.374029+0000","last_became_peered":"2021-01-06T22:03:39.374029+0000","last_unstale":"2021-01-06T22:03:43.815252+0000","last_undegraded":"2021-01-06T22:03:43.815252+0000","last_fullsized":"2021-01-06T22:03:43.815252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.765901+0000","last_change":"2021-01-06T22:03:39.762496+0000","last_active":"2021-01-06T22:03:39.765901+0000","last_peered":"2021-01-06T22:03:39.765901+0000","last_clean":"2021-01-06T22:03:39.765901+0000","last_became_active":"2021-01-06T22:03:39.761903+0000","last_became_peered":"2021-01-06T22:03:39.761903+0000","last_unstale":"2021-01-06T22:03:39.765901+0000","last_undegraded":"2021-01-06T22:03:39.765901+0000","last_fullsized":"2021-01-06T22:03:39.765901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747295+0000","last_change":"2021-01-06T22:03:36.206609+0000","last_active":"2021-01-06T22:03:39.747295+0000","last_peered":"2021-01-06T22:03:39.747295+0000","last_clean":"2021-01-06T22:03:39.747295+0000","last_became_active":"2021-01-06T22:03:36.206123+0000","last_became_peered":"2021-01-06T22:03:36.206123+0000","last_unstale":"2021-01-06T22:03:39.747295+0000","last_undegraded":"2021-01-06T22:03:39.747295+0000","last_fullsized":"2021-01-06T22:03:39.747295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"23'12","reported_seq":"22","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.821871+0000","last_change":"2021-01-06T22:03:39.377897+0000","last_active":"2021-01-06T22:03:43.821871+0000","last_peered":"2021-01-06T22:03:43.821871+0000","last_clean":"2021-01-06T22:03:43.821871+0000","last_became_active":"2021-01-06T22:03:39.376836+0000","last_became_peered":"2021-01-06T22:03:39.376836+0000","last_unstale":"2021-01-06T22:03:43.821871+0000","last_undegraded":"2021-01-06T22:03:43.821871+0000","last_fullsized":"2021-01-06T22:03:43.821871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":720,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.778763+0000","last_change":"2021-01-06T22:03:39.777501+0000","last_active":"2021-01-06T22:03:39.778763+0000","last_peered":"2021-01-06T22:03:39.778763+0000","last_clean":"2021-01-06T22:03:39.778763+0000","last_became_active":"2021-01-06T22:03:39.777158+0000","last_became_peered":"2021-01-06T22:03:39.777158+0000","last_unstale":"2021-01-06T22:03:39.778763+0000","last_undegraded":"2021-01-06T22:03:39.778763+0000","last_fullsized":"2021-01-06T22:03:39.778763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747656+0000","last_change":"2021-01-06T22:03:36.208974+0000","last_active":"2021-01-06T22:03:39.747656+0000","last_peered":"2021-01-06T22:03:39.747656+0000","last_clean":"2021-01-06T22:03:39.747656+0000","last_became_active":"2021-01-06T22:03:36.208576+0000","last_became_peered":"2021-01-06T22:03:36.208576+0000","last_unstale":"2021-01-06T22:03:39.747656+0000","last_undegraded":"2021-01-06T22:03:39.747656+0000","last_fullsized":"2021-01-06T22:03:39.747656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"23'13","reported_seq":"23","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.816002+0000","last_change":"2021-01-06T22:03:39.375127+0000","last_active":"2021-01-06T22:03:43.816002+0000","last_peered":"2021-01-06T22:03:43.816002+0000","last_clean":"2021-01-06T22:03:43.816002+0000","last_became_active":"2021-01-06T22:03:39.374751+0000","last_became_peered":"2021-01-06T22:03:39.374751+0000","last_unstale":"2021-01-06T22:03:43.816002+0000","last_undegraded":"2021-01-06T22:03:43.816002+0000","last_fullsized":"2021-01-06T22:03:43.816002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2088,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:40.232168+0000","last_change":"2021-01-06T22:03:40.230887+0000","last_active":"2021-01-06T22:03:40.232168+0000","last_peered":"2021-01-06T22:03:40.232168+0000","last_clean":"2021-01-06T22:03:40.232168+0000","last_became_active":"2021-01-06T22:03:40.230100+0000","last_became_peered":"2021-01-06T22:03:40.230100+0000","last_unstale":"2021-01-06T22:03:40.232168+0000","last_undegraded":"2021-01-06T22:03:40.232168+0000","last_fullsized":"2021-01-06T22:03:40.232168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"23'18","reported_seq":"28","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:46.596414+0000","last_change":"2021-01-06T22:03:39.377489+0000","last_active":"2021-01-06T22:03:46.596414+0000","last_peered":"2021-01-06T22:03:46.596414+0000","last_clean":"2021-01-06T22:03:46.596414+0000","last_became_active":"2021-01-06T22:03:39.377092+0000","last_became_peered":"2021-01-06T22:03:39.377092+0000","last_unstale":"2021-01-06T22:03:46.596414+0000","last_undegraded":"2021-01-06T22:03:46.596414+0000","last_fullsized":"2021-01-06T22:03:46.596414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":665,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.759458+0000","last_change":"2021-01-06T22:03:36.213113+0000","last_active":"2021-01-06T22:03:39.759458+0000","last_peered":"2021-01-06T22:03:39.759458+0000","last_clean":"2021-01-06T22:03:39.759458+0000","last_became_active":"2021-01-06T22:03:36.212790+0000","last_became_peered":"2021-01-06T22:03:36.212790+0000","last_unstale":"2021-01-06T22:03:39.759458+0000","last_undegraded":"2021-01-06T22:03:39.759458+0000","last_fullsized":"2021-01-06T22:03:39.759458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.764030+0000","last_change":"2021-01-06T22:03:39.759582+0000","last_active":"2021-01-06T22:03:39.764030+0000","last_peered":"2021-01-06T22:03:39.764030+0000","last_clean":"2021-01-06T22:03:39.764030+0000","last_became_active":"2021-01-06T22:03:39.759212+0000","last_became_peered":"2021-01-06T22:03:39.759212+0000","last_unstale":"2021-01-06T22:03:39.764030+0000","last_undegraded":"2021-01-06T22:03:39.764030+0000","last_fullsized":"2021-01-06T22:03:39.764030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:39.349016+0000","last_clean_scrub_stamp":"2021-01-06T22:03:39.349016+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-06T22:03:39.747367+0000","last_change":"2021-01-06T22:03:36.207481+0000","last_active":"2021-01-06T22:03:39.747367+0000","last_peered":"2021-01-06T22:03:39.747367+0000","last_clean":"2021-01-06T22:03:39.747367+0000","last_became_active":"2021-01-06T22:03:36.206991+0000","last_became_peered":"2021-01-06T22:03:36.206991+0000","last_unstale":"2021-01-06T22:03:39.747367+0000","last_undegraded":"2021-01-06T22:03:39.747367+0000","last_fullsized":"2021-01-06T22:03:39.747367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:35.177632+0000","last_clean_scrub_stamp":"2021-01-06T22:03:35.177632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'9","reported_seq":"19","reported_epoch":"23","state":"active+clean","last_fresh":"2021-01-06T22:03:43.813735+0000","last_change":"2021-01-06T22:03:39.370350+0000","last_active":"2021-01-06T22:03:43.813735+0000","last_peered":"2021-01-06T22:03:43.813735+0000","last_clean":"2021-01-06T22:03:43.813735+0000","last_became_active":"2021-01-06T22:03:39.369929+0000","last_became_peered":"2021-01-06T22:03:39.369929+0000","last_unstale":"2021-01-06T22:03:43.813735+0000","last_undegraded":"2021-01-06T22:03:43.813735+0000","last_fullsized":"2021-01-06T22:03:43.813735+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-06T22:03:38.343068+0000","last_clean_scrub_stamp":"2021-01-06T22:03:38.343068+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":5405,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":11253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5112,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366728,"statfs":{"total":96636764160,"available":96631529472,"internally_reserved":0,"allocated":245760,"data_stored":91578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5470000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8879999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3029999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3090000000000002}]}]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5224,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366616,"statfs":{"total":96636764160,"available":96631414784,"internally_reserved":0,"allocated":294912,"data_stored":95674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3580000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1520000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4060000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2149999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.097}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0430000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5192,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366648,"statfs":{"total":96636764160,"available":96631447552,"internally_reserved":0,"allocated":262144,"data_stored":94109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.302}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2040000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0649999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0419999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4060000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5070000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5116,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366724,"statfs":{"total":96636764160,"available":96631525376,"internally_reserved":0,"allocated":249856,"data_stored":91973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.944}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3650000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.907}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1259999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2650000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5569999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":237568,"data_stored":91466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3650000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0059999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.137}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.645}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6930000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8959999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2480000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5148,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366692,"statfs":{"total":96636764160,"available":96631492608,"internally_reserved":0,"allocated":282624,"data_stored":93563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.173}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3839999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8439999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2719999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6040000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6669999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5070000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5104,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366736,"statfs":{"total":96636764160,"available":96631537664,"internally_reserved":0,"allocated":237568,"data_stored":91466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0539999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2450000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4289999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1749999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3719999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5116,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366724,"statfs":{"total":96636764160,"available":96631525376,"internally_reserved":0,"allocated":249856,"data_stored":92131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.865}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.774}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7330000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.024}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.552}]}]}]}} 2021-01-06T22:04:07.262 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-06T22:04:07.389 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.387+0000 7f34f1c36700 1 Processor -- start 2021-01-06T22:04:07.389 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.387+0000 7f34f1c36700 1 -- start start 2021-01-06T22:04:07.390 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.387+0000 7f34f1c36700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f34ec0617c0 0x7f34ec14e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.390 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.387+0000 7f34f1c36700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ec0a6880 con 0x7f34ec14a520 2021-01-06T22:04:07.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.387+0000 7f34f1c36700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ec14a970 con 0x7f34ec0646b0 2021-01-06T22:04:07.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.387+0000 7f34f1c36700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ec147ff0 con 0x7f34ec0617c0 2021-01-06T22:04:07.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34eb7fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f34ec0646b0 0x7f34ec0c1330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48100/0 (socket says 172.21.15.203:48100) 2021-01-06T22:04:07.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34eb7fe700 1 -- 172.21.15.203:0/3680356790 learned_addr learned my addr 172.21.15.203:0/3680356790 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:04:07.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34eaffd700 1 --2- 172.21.15.203:0/3680356790 >> v2:172.21.15.91:3300/0 conn(0x7f34ec0617c0 0x7f34ec14e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 -- 172.21.15.203:0/3680356790 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2451808272 0 0) 0x7f34ec14a970 con 0x7f34ec0646b0 2021-01-06T22:04:07.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 -- 172.21.15.203:0/3680356790 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34d8003230 con 0x7f34ec0646b0 2021-01-06T22:04:07.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 -- 172.21.15.203:0/3680356790 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443569135 0 0) 0x7f34ec0a6880 con 0x7f34ec14a520 2021-01-06T22:04:07.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 -- 172.21.15.203:0/3680356790 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34ec14a970 con 0x7f34ec14a520 2021-01-06T22:04:07.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 -- 172.21.15.203:0/3680356790 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2818877650 0 0) 0x7f34d8003230 con 0x7f34ec0646b0 2021-01-06T22:04:07.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 -- 172.21.15.203:0/3680356790 >> v2:172.21.15.91:3300/0 conn(0x7f34ec0617c0 msgr2=0x7f34ec14e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:07.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 --2- 172.21.15.203:0/3680356790 >> v2:172.21.15.91:3300/0 conn(0x7f34ec0617c0 0x7f34ec14e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 -- 172.21.15.203:0/3680356790 >> v1:172.21.15.91:6789/0 conn(0x7f34ec14a520 legacy=0x7f34ec0ba720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 -- 172.21.15.203:0/3680356790 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec0a6ab0 con 0x7f34ec0646b0 2021-01-06T22:04:07.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 -- 172.21.15.203:0/3680356790 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f34e00014e0 con 0x7f34ec0646b0 2021-01-06T22:04:07.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 -- 172.21.15.203:0/3680356790 >> v1:172.21.15.203:6789/0 conn(0x7f34ec0646b0 legacy=0x7f34ec0c1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 --2- 172.21.15.203:0/3680356790 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34d80038b0 0x7f34d8003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 --2- 172.21.15.203:0/3680356790 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34d80044d0 0x7f34d8004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 --2- 172.21.15.203:0/3680356790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34d8004e40 0x7f34d800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 -- 172.21.15.203:0/3680356790 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec147ff0 con 0x7f34d80038b0 2021-01-06T22:04:07.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 -- 172.21.15.203:0/3680356790 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d80053a0 con 0x7f34d8004e40 2021-01-06T22:04:07.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 -- 172.21.15.203:0/3680356790 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d800bc90 con 0x7f34d80044d0 2021-01-06T22:04:07.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34eaffd700 1 --2- 172.21.15.203:0/3680356790 >> v2:172.21.15.91:3300/0 conn(0x7f34ec0617c0 0x7f34ec14e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:04:07.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ebfff700 1 --2- 172.21.15.203:0/3680356790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34d8004e40 0x7f34d800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34eb7fe700 1 --2- 172.21.15.203:0/3680356790 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34d80038b0 0x7f34d8003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34eaffd700 1 --2- 172.21.15.203:0/3680356790 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34d80044d0 0x7f34d8004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34eb7fe700 1 -- 172.21.15.203:0/3680356790 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34d80044d0 msgr2=0x7f34d8004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:07.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34eb7fe700 1 --2- 172.21.15.203:0/3680356790 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34d80044d0 0x7f34d8004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34eb7fe700 1 -- 172.21.15.203:0/3680356790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34d8004e40 msgr2=0x7f34d800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:07.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34eb7fe700 1 --2- 172.21.15.203:0/3680356790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34d8004e40 0x7f34d800b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34eb7fe700 1 -- 172.21.15.203:0/3680356790 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d8003230 con 0x7f34d80038b0 2021-01-06T22:04:07.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ebfff700 1 --2- 172.21.15.203:0/3680356790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34d8004e40 0x7f34d800b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:04:07.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ebfff700 1 -- 172.21.15.203:0/3680356790 reap_dead start 2021-01-06T22:04:07.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34eb7fe700 1 --2- 172.21.15.203:0/3680356790 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34d80038b0 0x7f34d8003cb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f34e000c230 tx=0x7f34ec0656c0).ready entity=mon.0 client_cookie=e0d50ed68431c643 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:07.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 -- 172.21.15.203:0/3680356790 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e00070d0 con 0x7f34d80038b0 2021-01-06T22:04:07.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 -- 172.21.15.203:0/3680356790 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e0007230 con 0x7f34d80038b0 2021-01-06T22:04:07.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34ea7fc700 1 -- 172.21.15.203:0/3680356790 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e00055e0 con 0x7f34d80038b0 2021-01-06T22:04:07.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34f1c36700 1 -- 172.21.15.203:0/3680356790 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34d80038b0 msgr2=0x7f34d8003cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34f1c36700 1 --2- 172.21.15.203:0/3680356790 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34d80038b0 0x7f34d8003cb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f34e000c230 tx=0x7f34ec0656c0).stop 2021-01-06T22:04:07.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34f1c36700 1 -- 172.21.15.203:0/3680356790 shutdown_connections 2021-01-06T22:04:07.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34f1c36700 1 --2- 172.21.15.203:0/3680356790 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34d80038b0 0x7f34d8003cb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34f1c36700 1 -- 172.21.15.203:0/3680356790 >> 172.21.15.203:0/3680356790 conn(0x7f34ec0631b0 msgr2=0x7f34ec0c0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:07.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34f1c36700 1 -- 172.21.15.203:0/3680356790 shutdown_connections 2021-01-06T22:04:07.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.391+0000 7f34f1c36700 1 -- 172.21.15.203:0/3680356790 wait complete. 2021-01-06T22:04:07.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34f1c36700 1 Processor -- start 2021-01-06T22:04:07.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34f1c36700 1 -- start start 2021-01-06T22:04:07.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34f1c36700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34d80038b0 0x7f34ec13d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34f1c36700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34ec13d6d0 0x7f34ec13ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34f1c36700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34ec13ef10 0x7f34ec139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34f1c36700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0beb10 con 0x7f34d80038b0 2021-01-06T22:04:07.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34f1c36700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec13f390 con 0x7f34ec13ef10 2021-01-06T22:04:07.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34f1c36700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec13dc70 con 0x7f34ec13d6d0 2021-01-06T22:04:07.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34ebfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34ec13ef10 0x7f34ec139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34ebfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34ec13ef10 0x7f34ec139bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49520/0 (socket says 172.21.15.203:49520) 2021-01-06T22:04:07.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34ebfff700 1 -- 172.21.15.203:0/3268800261 learned_addr learned my addr 172.21.15.203:0/3268800261 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:07.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34eb7fe700 1 --2- 172.21.15.203:0/3268800261 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34d80038b0 0x7f34ec13d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34eaffd700 1 --2- 172.21.15.203:0/3268800261 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34ec13d6d0 0x7f34ec13ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34ebfff700 1 -- 172.21.15.203:0/3268800261 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34ec13d6d0 msgr2=0x7f34ec13ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34ebfff700 1 --2- 172.21.15.203:0/3268800261 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34ec13d6d0 0x7f34ec13ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34ebfff700 1 -- 172.21.15.203:0/3268800261 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34d80038b0 msgr2=0x7f34ec13d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34ebfff700 1 --2- 172.21.15.203:0/3268800261 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34d80038b0 0x7f34ec13d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.395+0000 7f34ebfff700 1 -- 172.21.15.203:0/3268800261 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec14c510 con 0x7f34ec13ef10 2021-01-06T22:04:07.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.399+0000 7f34ebfff700 1 --2- 172.21.15.203:0/3268800261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34ec13ef10 0x7f34ec139bb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f34ec066590 tx=0x7f34e4007b60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:07.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.399+0000 7f34d37fe700 1 -- 172.21.15.203:0/3268800261 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e4007d00 con 0x7f34ec13ef10 2021-01-06T22:04:07.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.399+0000 7f34d37fe700 1 -- 172.21.15.203:0/3268800261 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e4007240 con 0x7f34ec13ef10 2021-01-06T22:04:07.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.399+0000 7f34f1c36700 1 -- 172.21.15.203:0/3268800261 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ec0bed40 con 0x7f34ec13ef10 2021-01-06T22:04:07.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.399+0000 7f34d37fe700 1 -- 172.21.15.203:0/3268800261 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e4012870 con 0x7f34ec13ef10 2021-01-06T22:04:07.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.399+0000 7f34f1c36700 1 -- 172.21.15.203:0/3268800261 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ec14c260 con 0x7f34ec13ef10 2021-01-06T22:04:07.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.399+0000 7f34f1c36700 1 -- 172.21.15.203:0/3268800261 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ec05cfe0 con 0x7f34ec13ef10 2021-01-06T22:04:07.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.399+0000 7f34d37fe700 1 -- 172.21.15.203:0/3268800261 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f34e4022070 con 0x7f34ec13ef10 2021-01-06T22:04:07.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.399+0000 7f34d37fe700 1 --2- 172.21.15.203:0/3268800261 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f34cc0563d0 0x7f34cc058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.399+0000 7f34d37fe700 1 -- 172.21.15.203:0/3268800261 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f34e4071380 con 0x7f34ec13ef10 2021-01-06T22:04:07.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.399+0000 7f34eb7fe700 1 --2- 172.21.15.203:0/3268800261 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f34cc0563d0 0x7f34cc058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.403+0000 7f34eb7fe700 1 --2- 172.21.15.203:0/3268800261 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f34cc0563d0 0x7f34cc058890 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f34e0007840 tx=0x7f34e0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:07.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.407+0000 7f34d37fe700 1 -- 172.21.15.203:0/3268800261 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f34e4046c00 con 0x7f34ec13ef10 2021-01-06T22:04:07.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.563+0000 7f34f1c36700 1 -- 172.21.15.203:0/3268800261 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34ec061cc0 con 0x7f34ec13ef10 2021-01-06T22:04:07.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.567+0000 7f34d37fe700 1 -- 172.21.15.203:0/3268800261 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f34e40073b0 con 0x7f34ec13ef10 2021-01-06T22:04:07.568 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:07.568 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":23,"fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","created":"2021-01-06T22:03:21.322671+0000","modified":"2021-01-06T22:03:40.744826+0000","last_up_change":"2021-01-06T22:03:31.115611+0000","last_in_change":"2021-01-06T22:03:31.115611+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-06T22:03:25.161465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-06T22:03:35.070364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-01-06T22:03:37.862354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-01-06T22:03:38.738416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e0b35c09-147c-4ccc-b566-c48b81e25412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":18347},{"type":"v1","addr":"172.21.15.203:6815","nonce":18347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":18347},{"type":"v1","addr":"172.21.15.203:6818","nonce":18347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":18347},{"type":"v1","addr":"172.21.15.203:6831","nonce":18347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6821","nonce":18347},{"type":"v1","addr":"172.21.15.203:6827","nonce":18347}]},"public_addr":"172.21.15.203:6815/18347","cluster_addr":"172.21.15.203:6818/18347","heartbeat_back_addr":"172.21.15.203:6831/18347","heartbeat_front_addr":"172.21.15.203:6827/18347","state":["exists","up"]},{"osd":1,"uuid":"8add59a0-2d58-4c1b-a8d1-62049ee9d85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":18337},{"type":"v1","addr":"172.21.15.203:6803","nonce":18337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":18337},{"type":"v1","addr":"172.21.15.203:6805","nonce":18337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":18337},{"type":"v1","addr":"172.21.15.203:6809","nonce":18337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":18337},{"type":"v1","addr":"172.21.15.203:6807","nonce":18337}]},"public_addr":"172.21.15.203:6803/18337","cluster_addr":"172.21.15.203:6805/18337","heartbeat_back_addr":"172.21.15.203:6809/18337","heartbeat_front_addr":"172.21.15.203:6807/18337","state":["exists","up"]},{"osd":2,"uuid":"b294a411-c7c9-477e-8eae-3b7bd0563ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":18354},{"type":"v1","addr":"172.21.15.203:6811","nonce":18354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":18354},{"type":"v1","addr":"172.21.15.203:6819","nonce":18354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6825","nonce":18354},{"type":"v1","addr":"172.21.15.203:6828","nonce":18354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6823","nonce":18354},{"type":"v1","addr":"172.21.15.203:6824","nonce":18354}]},"public_addr":"172.21.15.203:6811/18354","cluster_addr":"172.21.15.203:6819/18354","heartbeat_back_addr":"172.21.15.203:6828/18354","heartbeat_front_addr":"172.21.15.203:6824/18354","state":["exists","up"]},{"osd":3,"uuid":"5878a8f7-1a42-41c3-9810-0ac41a54dd6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":18346},{"type":"v1","addr":"172.21.15.203:6817","nonce":18346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":18346},{"type":"v1","addr":"172.21.15.203:6822","nonce":18346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":18346},{"type":"v1","addr":"172.21.15.203:6833","nonce":18346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":18346},{"type":"v1","addr":"172.21.15.203:6829","nonce":18346}]},"public_addr":"172.21.15.203:6817/18346","cluster_addr":"172.21.15.203:6822/18346","heartbeat_back_addr":"172.21.15.203:6833/18346","heartbeat_front_addr":"172.21.15.203:6829/18346","state":["exists","up"]},{"osd":4,"uuid":"f641a345-81e4-4400-a44c-d2f82fef7f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":18370},{"type":"v1","addr":"172.21.15.91:6809","nonce":18370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":18370},{"type":"v1","addr":"172.21.15.91:6811","nonce":18370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":18370},{"type":"v1","addr":"172.21.15.91:6816","nonce":18370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":18370},{"type":"v1","addr":"172.21.15.91:6813","nonce":18370}]},"public_addr":"172.21.15.91:6809/18370","cluster_addr":"172.21.15.91:6811/18370","heartbeat_back_addr":"172.21.15.91:6816/18370","heartbeat_front_addr":"172.21.15.91:6813/18370","state":["exists","up"]},{"osd":5,"uuid":"c1ff150d-68c6-4e6b-abd4-76af27bb2103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":18372},{"type":"v1","addr":"172.21.15.91:6801","nonce":18372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":18372},{"type":"v1","addr":"172.21.15.91:6803","nonce":18372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":18372},{"type":"v1","addr":"172.21.15.91:6807","nonce":18372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":18372},{"type":"v1","addr":"172.21.15.91:6805","nonce":18372}]},"public_addr":"172.21.15.91:6801/18372","cluster_addr":"172.21.15.91:6803/18372","heartbeat_back_addr":"172.21.15.91:6807/18372","heartbeat_front_addr":"172.21.15.91:6805/18372","state":["exists","up"]},{"osd":6,"uuid":"35a58e1a-0a0a-4f02-a477-adc029038d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6815","nonce":18371},{"type":"v1","addr":"172.21.15.91:6817","nonce":18371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":18371},{"type":"v1","addr":"172.21.15.91:6819","nonce":18371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":18371},{"type":"v1","addr":"172.21.15.91:6823","nonce":18371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":18371},{"type":"v1","addr":"172.21.15.91:6821","nonce":18371}]},"public_addr":"172.21.15.91:6817/18371","cluster_addr":"172.21.15.91:6819/18371","heartbeat_back_addr":"172.21.15.91:6823/18371","heartbeat_front_addr":"172.21.15.91:6821/18371","state":["exists","up"]},{"osd":7,"uuid":"809a2d89-4303-482a-8558-aef907814234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":18373},{"type":"v1","addr":"172.21.15.91:6825","nonce":18373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":18373},{"type":"v1","addr":"172.21.15.91:6827","nonce":18373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":18373},{"type":"v1","addr":"172.21.15.91:6831","nonce":18373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":18373},{"type":"v1","addr":"172.21.15.91:6829","nonce":18373}]},"public_addr":"172.21.15.91:6825/18373","cluster_addr":"172.21.15.91:6827/18373","heartbeat_back_addr":"172.21.15.91:6831/18373","heartbeat_front_addr":"172.21.15.91:6829/18373","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.875052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.103604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.179225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.158512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.206073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.894840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.229708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.248192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-06T22:04:07.569 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.567+0000 7f34f1c36700 1 -- 172.21.15.203:0/3268800261 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f34cc0563d0 msgr2=0x7f34cc058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.569 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.567+0000 7f34f1c36700 1 --2- 172.21.15.203:0/3268800261 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f34cc0563d0 0x7f34cc058890 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f34e0007840 tx=0x7f34e0015000).stop 2021-01-06T22:04:07.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.567+0000 7f34f1c36700 1 -- 172.21.15.203:0/3268800261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34ec13ef10 msgr2=0x7f34ec139bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.567+0000 7f34f1c36700 1 --2- 172.21.15.203:0/3268800261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34ec13ef10 0x7f34ec139bb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f34ec066590 tx=0x7f34e4007b60).stop 2021-01-06T22:04:07.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.567+0000 7f34f1c36700 1 -- 172.21.15.203:0/3268800261 shutdown_connections 2021-01-06T22:04:07.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.567+0000 7f34f1c36700 1 --2- 172.21.15.203:0/3268800261 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f34cc0563d0 0x7f34cc058890 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.567+0000 7f34f1c36700 1 --2- 172.21.15.203:0/3268800261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34ec13ef10 0x7f34ec139bb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.567+0000 7f34f1c36700 1 --2- 172.21.15.203:0/3268800261 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34d80038b0 0x7f34ec13d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.567+0000 7f34f1c36700 1 --2- 172.21.15.203:0/3268800261 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34ec13d6d0 0x7f34ec13ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.567+0000 7f34f1c36700 1 -- 172.21.15.203:0/3268800261 >> 172.21.15.203:0/3268800261 conn(0x7f34ec0631b0 msgr2=0x7f34ec14ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:07.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.567+0000 7f34f1c36700 1 -- 172.21.15.203:0/3268800261 shutdown_connections 2021-01-06T22:04:07.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.567+0000 7f34f1c36700 1 -- 172.21.15.203:0/3268800261 wait complete. 2021-01-06T22:04:07.582 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-01-06T22:03:25.161465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-01-06T22:03:35.070364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-01-06T22:03:37.862354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-01-06T22:03:38.738416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-01-06T22:04:07.583 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-01-06T22:04:07.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe43a86700 1 Processor -- start 2021-01-06T22:04:07.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe43a86700 1 -- start start 2021-01-06T22:04:07.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe43a86700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbe3c14b7c0 0x7fbe3c0bd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe43a86700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbe3c064bd0 0x7fbe3c14ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe43a86700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c0bb740 con 0x7fbe3c0647f0 2021-01-06T22:04:07.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe43a86700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3c056cd0 con 0x7fbe3c14b7c0 2021-01-06T22:04:07.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe43a86700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c057400 con 0x7fbe3c064bd0 2021-01-06T22:04:07.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe41021700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbe3c14b7c0 0x7fbe3c0bd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe41021700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbe3c14b7c0 0x7fbe3c0bd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49530/0 (socket says 172.21.15.203:49530) 2021-01-06T22:04:07.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe41021700 1 -- 172.21.15.203:0/3916016925 learned_addr learned my addr 172.21.15.203:0/3916016925 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:07.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe41822700 1 --1- 172.21.15.203:0/3916016925 >> v1:172.21.15.91:6789/0 conn(0x7fbe3c0647f0 0x7fbe3c14b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:42774/0 (socket says 172.21.15.203:42774) 2021-01-06T22:04:07.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe42023700 1 --2- 172.21.15.203:0/3916016925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbe3c064bd0 0x7fbe3c14ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe41021700 1 -- 172.21.15.203:0/3916016925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbe3c064bd0 msgr2=0x7fbe3c14ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:07.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe41021700 1 --2- 172.21.15.203:0/3916016925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbe3c064bd0 0x7fbe3c14ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe41021700 1 -- 172.21.15.203:0/3916016925 >> v1:172.21.15.91:6789/0 conn(0x7fbe3c0647f0 legacy=0x7fbe3c14b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe41021700 1 -- 172.21.15.203:0/3916016925 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c14ef10 con 0x7fbe3c14b7c0 2021-01-06T22:04:07.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe42023700 1 --2- 172.21.15.203:0/3916016925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbe3c064bd0 0x7fbe3c14ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:04:07.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe41021700 1 --2- 172.21.15.203:0/3916016925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe3c14b7c0 0x7fbe3c0bd150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fbe2c00b9e0 tx=0x7fbe2c006ea0).ready entity=mon.1 client_cookie=68aa15d54a5843e server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:07.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe40820700 1 -- 172.21.15.203:0/3916016925 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c0098d0 con 0x7fbe3c14b7c0 2021-01-06T22:04:07.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe40820700 1 -- 172.21.15.203:0/3916016925 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c009a30 con 0x7fbe3c14b7c0 2021-01-06T22:04:07.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.715+0000 7fbe40820700 1 -- 172.21.15.203:0/3916016925 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c009db0 con 0x7fbe3c14b7c0 2021-01-06T22:04:07.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.719+0000 7fbe43a86700 1 -- 172.21.15.203:0/3916016925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe3c14b7c0 msgr2=0x7fbe3c0bd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.719+0000 7fbe43a86700 1 --2- 172.21.15.203:0/3916016925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe3c14b7c0 0x7fbe3c0bd150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fbe2c00b9e0 tx=0x7fbe2c006ea0).stop 2021-01-06T22:04:07.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.719+0000 7fbe43a86700 1 -- 172.21.15.203:0/3916016925 shutdown_connections 2021-01-06T22:04:07.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.719+0000 7fbe43a86700 1 --2- 172.21.15.203:0/3916016925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbe3c064bd0 0x7fbe3c14ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.719+0000 7fbe43a86700 1 --2- 172.21.15.203:0/3916016925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe3c14b7c0 0x7fbe3c0bd150 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.719+0000 7fbe43a86700 1 -- 172.21.15.203:0/3916016925 >> 172.21.15.203:0/3916016925 conn(0x7fbe3c0631b0 msgr2=0x7fbe3c0c0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:07.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.719+0000 7fbe43a86700 1 -- 172.21.15.203:0/3916016925 shutdown_connections 2021-01-06T22:04:07.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.719+0000 7fbe43a86700 1 -- 172.21.15.203:0/3916016925 wait complete. 2021-01-06T22:04:07.729 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.719+0000 7fbe43a86700 1 Processor -- start 2021-01-06T22:04:07.729 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.719+0000 7fbe43a86700 1 -- start start 2021-01-06T22:04:07.729 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe43a86700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe3c0647f0 0x7fbe3c15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe43a86700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbe3c064bd0 0x7fbe3c060ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe43a86700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe3c14b7c0 0x7fbe3c0b6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe43a86700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c14f2b0 con 0x7fbe3c14b7c0 2021-01-06T22:04:07.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe43a86700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0a32b0 con 0x7fbe3c0647f0 2021-01-06T22:04:07.731 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe43a86700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0a2ad0 con 0x7fbe3c064bd0 2021-01-06T22:04:07.731 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe41822700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe3c0647f0 0x7fbe3c15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.731 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe41822700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe3c0647f0 0x7fbe3c15e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49532/0 (socket says 172.21.15.203:49532) 2021-01-06T22:04:07.731 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe41822700 1 -- 172.21.15.203:0/51904102 learned_addr learned my addr 172.21.15.203:0/51904102 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:07.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe42023700 1 --2- 172.21.15.203:0/51904102 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe3c14b7c0 0x7fbe3c0b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe41021700 1 --2- 172.21.15.203:0/51904102 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbe3c064bd0 0x7fbe3c060ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe41822700 1 -- 172.21.15.203:0/51904102 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbe3c064bd0 msgr2=0x7fbe3c060ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe41822700 1 --2- 172.21.15.203:0/51904102 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbe3c064bd0 0x7fbe3c060ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe41822700 1 -- 172.21.15.203:0/51904102 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe3c14b7c0 msgr2=0x7fbe3c0b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.733 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe41822700 1 --2- 172.21.15.203:0/51904102 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe3c14b7c0 0x7fbe3c0b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.733 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe41822700 1 -- 172.21.15.203:0/51904102 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c0a2d70 con 0x7fbe3c0647f0 2021-01-06T22:04:07.733 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe42023700 1 --2- 172.21.15.203:0/51904102 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe3c14b7c0 0x7fbe3c0b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:04:07.733 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe41021700 1 --2- 172.21.15.203:0/51904102 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbe3c064bd0 0x7fbe3c060ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:04:07.734 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe41822700 1 --2- 172.21.15.203:0/51904102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe3c0647f0 0x7fbe3c15e8a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fbe3400cac0 tx=0x7fbe34007f70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:07.734 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe29ffb700 1 -- 172.21.15.203:0/51904102 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34005ab0 con 0x7fbe3c0647f0 2021-01-06T22:04:07.734 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe29ffb700 1 -- 172.21.15.203:0/51904102 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe34005c10 con 0x7fbe3c0647f0 2021-01-06T22:04:07.734 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe43a86700 1 -- 172.21.15.203:0/51904102 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c0a3010 con 0x7fbe3c0647f0 2021-01-06T22:04:07.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe29ffb700 1 -- 172.21.15.203:0/51904102 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34007110 con 0x7fbe3c0647f0 2021-01-06T22:04:07.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe43a86700 1 -- 172.21.15.203:0/51904102 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3c0b5ef0 con 0x7fbe3c0647f0 2021-01-06T22:04:07.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.723+0000 7fbe43a86700 1 -- 172.21.15.203:0/51904102 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3c0bd2e0 con 0x7fbe3c0647f0 2021-01-06T22:04:07.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.727+0000 7fbe29ffb700 1 -- 172.21.15.203:0/51904102 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fbe34010030 con 0x7fbe3c0647f0 2021-01-06T22:04:07.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.727+0000 7fbe29ffb700 1 --2- 172.21.15.203:0/51904102 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbe1c056400 0x7fbe1c0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:07.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.727+0000 7fbe29ffb700 1 -- 172.21.15.203:0/51904102 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbe3406f500 con 0x7fbe3c0647f0 2021-01-06T22:04:07.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.727+0000 7fbe41021700 1 --2- 172.21.15.203:0/51904102 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbe1c056400 0x7fbe1c0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:07.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.727+0000 7fbe41021700 1 --2- 172.21.15.203:0/51904102 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbe1c056400 0x7fbe1c0588c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fbe2c00b190 tx=0x7fbe2c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:07.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.735+0000 7fbe29ffb700 1 -- 172.21.15.203:0/51904102 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fbe3c0bd2e0 con 0x7fbe3c0647f0 2021-01-06T22:04:07.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.891+0000 7fbe43a86700 1 -- 172.21.15.203:0/51904102 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fbe3c15ed90 con 0x7fbe3c0647f0 2021-01-06T22:04:07.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.891+0000 7fbe29ffb700 1 -- 172.21.15.203:0/51904102 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7fbe34044c70 con 0x7fbe3c0647f0 2021-01-06T22:04:07.892 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 1 2021-01-06T22:04:07.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.891+0000 7fbe43a86700 1 -- 172.21.15.203:0/51904102 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbe1c056400 msgr2=0x7fbe1c0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.891+0000 7fbe43a86700 1 --2- 172.21.15.203:0/51904102 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbe1c056400 0x7fbe1c0588c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fbe2c00b190 tx=0x7fbe2c016000).stop 2021-01-06T22:04:07.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.891+0000 7fbe43a86700 1 -- 172.21.15.203:0/51904102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe3c0647f0 msgr2=0x7fbe3c15e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:07.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.891+0000 7fbe43a86700 1 --2- 172.21.15.203:0/51904102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe3c0647f0 0x7fbe3c15e8a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fbe3400cac0 tx=0x7fbe34007f70).stop 2021-01-06T22:04:07.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.891+0000 7fbe43a86700 1 -- 172.21.15.203:0/51904102 shutdown_connections 2021-01-06T22:04:07.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.891+0000 7fbe43a86700 1 --2- 172.21.15.203:0/51904102 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbe1c056400 0x7fbe1c0588c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.891+0000 7fbe43a86700 1 --2- 172.21.15.203:0/51904102 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe3c14b7c0 0x7fbe3c0b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.891+0000 7fbe43a86700 1 --2- 172.21.15.203:0/51904102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe3c0647f0 0x7fbe3c15e8a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.891+0000 7fbe43a86700 1 --2- 172.21.15.203:0/51904102 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbe3c064bd0 0x7fbe3c060ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:07.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.891+0000 7fbe43a86700 1 -- 172.21.15.203:0/51904102 >> 172.21.15.203:0/51904102 conn(0x7fbe3c0631b0 msgr2=0x7fbe3c14de50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:07.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.891+0000 7fbe43a86700 1 -- 172.21.15.203:0/51904102 shutdown_connections 2021-01-06T22:04:07.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:07.891+0000 7fbe43a86700 1 -- 172.21.15.203:0/51904102 wait complete. 2021-01-06T22:04:07.907 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-01-06T22:04:08.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.035+0000 7f3262917700 1 Processor -- start 2021-01-06T22:04:08.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.035+0000 7f3262917700 1 -- start start 2021-01-06T22:04:08.039 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.035+0000 7f3262917700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f325c066dc0 0x7f325c151530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.039 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.035+0000 7f3262917700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f325c0671a0 0x7f325c0bf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.035+0000 7f3262917700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f325c0bf810 0x7f325c0c17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.035+0000 7f3262917700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f325c151a20 con 0x7f325c066dc0 2021-01-06T22:04:08.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.035+0000 7f3262917700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f325c151c70 con 0x7f325c0bf810 2021-01-06T22:04:08.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.035+0000 7f3262917700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325c151f10 con 0x7f325c0671a0 2021-01-06T22:04:08.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.035+0000 7f3260eb4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f325c0bf810 0x7f325c0c17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.035+0000 7f3260eb4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f325c0bf810 0x7f325c0c17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49544/0 (socket says 172.21.15.203:49544) 2021-01-06T22:04:08.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.035+0000 7f3260eb4700 1 -- 172.21.15.203:0/125207393 learned_addr learned my addr 172.21.15.203:0/125207393 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:08.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f325b7fe700 1 --2- 172.21.15.203:0/125207393 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f325c0671a0 0x7f325c0bf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f325bfff700 1 --2- 172.21.15.203:0/125207393 >> v2:172.21.15.91:3300/0 conn(0x7f325c066dc0 0x7f325c151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3260eb4700 1 -- 172.21.15.203:0/125207393 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f325c0671a0 msgr2=0x7f325c0bf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:08.042 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3260eb4700 1 --2- 172.21.15.203:0/125207393 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f325c0671a0 0x7f325c0bf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.042 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3260eb4700 1 -- 172.21.15.203:0/125207393 >> v2:172.21.15.91:3300/0 conn(0x7f325c066dc0 msgr2=0x7f325c151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:08.042 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3260eb4700 1 --2- 172.21.15.203:0/125207393 >> v2:172.21.15.91:3300/0 conn(0x7f325c066dc0 0x7f325c151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.042 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3260eb4700 1 -- 172.21.15.203:0/125207393 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325c1521b0 con 0x7f325c0bf810 2021-01-06T22:04:08.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f325b7fe700 1 --2- 172.21.15.203:0/125207393 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f325c0671a0 0x7f325c0bf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:04:08.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3260eb4700 1 --2- 172.21.15.203:0/125207393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325c0bf810 0x7f325c0c17a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f325c0c0e00 tx=0x7f3254009290).ready entity=mon.1 client_cookie=1f20ca98c7273ccf server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:08.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f325affd700 1 -- 172.21.15.203:0/125207393 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325400bbc0 con 0x7f325c0bf810 2021-01-06T22:04:08.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f325affd700 1 -- 172.21.15.203:0/125207393 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325400bd20 con 0x7f325c0bf810 2021-01-06T22:04:08.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f325affd700 1 -- 172.21.15.203:0/125207393 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254011610 con 0x7f325c0bf810 2021-01-06T22:04:08.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3262917700 1 -- 172.21.15.203:0/125207393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325c0bf810 msgr2=0x7f325c0c17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3262917700 1 --2- 172.21.15.203:0/125207393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325c0bf810 0x7f325c0c17a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f325c0c0e00 tx=0x7f3254009290).stop 2021-01-06T22:04:08.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3262917700 1 -- 172.21.15.203:0/125207393 shutdown_connections 2021-01-06T22:04:08.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3262917700 1 --2- 172.21.15.203:0/125207393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325c0bf810 0x7f325c0c17a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3262917700 1 --2- 172.21.15.203:0/125207393 >> v2:172.21.15.91:3300/0 conn(0x7f325c066dc0 0x7f325c151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3262917700 1 --2- 172.21.15.203:0/125207393 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f325c0671a0 0x7f325c0bf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3262917700 1 -- 172.21.15.203:0/125207393 >> 172.21.15.203:0/125207393 conn(0x7f325c0626e0 msgr2=0x7f325c061a30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:08.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3262917700 1 -- 172.21.15.203:0/125207393 shutdown_connections 2021-01-06T22:04:08.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3262917700 1 -- 172.21.15.203:0/125207393 wait complete. 2021-01-06T22:04:08.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3262917700 1 Processor -- start 2021-01-06T22:04:08.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.039+0000 7f3262917700 1 -- start start 2021-01-06T22:04:08.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f3262917700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f325c066dc0 0x7f325c0be840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f3262917700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325c0671a0 0x7f325c063cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f3262917700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f325c0bbac0 0x7f325c14c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f3262917700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c0c3d80 con 0x7f325c066dc0 2021-01-06T22:04:08.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f3262917700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c15e450 con 0x7f325c0671a0 2021-01-06T22:04:08.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f3262917700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325c0c3fb0 con 0x7f325c0bbac0 2021-01-06T22:04:08.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f325b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325c0671a0 0x7f325c063cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f325b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325c0671a0 0x7f325c063cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49548/0 (socket says 172.21.15.203:49548) 2021-01-06T22:04:08.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f325b7fe700 1 -- 172.21.15.203:0/3760111882 learned_addr learned my addr 172.21.15.203:0/3760111882 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:08.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f3260eb4700 1 --2- 172.21.15.203:0/3760111882 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f325c0bbac0 0x7f325c14c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f325bfff700 1 --2- 172.21.15.203:0/3760111882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f325c066dc0 0x7f325c0be840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f325b7fe700 1 -- 172.21.15.203:0/3760111882 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f325c0bbac0 msgr2=0x7f325c14c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f325b7fe700 1 --2- 172.21.15.203:0/3760111882 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f325c0bbac0 0x7f325c14c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f325b7fe700 1 -- 172.21.15.203:0/3760111882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f325c066dc0 msgr2=0x7f325c0be840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f325b7fe700 1 --2- 172.21.15.203:0/3760111882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f325c066dc0 0x7f325c0be840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f325b7fe700 1 -- 172.21.15.203:0/3760111882 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325c0c4250 con 0x7f325c0671a0 2021-01-06T22:04:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f325b7fe700 1 --2- 172.21.15.203:0/3760111882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325c0671a0 0x7f325c063cb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f325000db60 tx=0x7f3250008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:08.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f3243fff700 1 -- 172.21.15.203:0/3760111882 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325000be50 con 0x7f325c0671a0 2021-01-06T22:04:08.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f3243fff700 1 -- 172.21.15.203:0/3760111882 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3250009820 con 0x7f325c0671a0 2021-01-06T22:04:08.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f3262917700 1 -- 172.21.15.203:0/3760111882 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325c14be10 con 0x7f325c0671a0 2021-01-06T22:04:08.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f3243fff700 1 -- 172.21.15.203:0/3760111882 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3250009ba0 con 0x7f325c0671a0 2021-01-06T22:04:08.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f3262917700 1 -- 172.21.15.203:0/3760111882 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325c14e9f0 con 0x7f325c0671a0 2021-01-06T22:04:08.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.043+0000 7f3262917700 1 -- 172.21.15.203:0/3760111882 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325c14bfa0 con 0x7f325c0671a0 2021-01-06T22:04:08.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.047+0000 7f3243fff700 1 -- 172.21.15.203:0/3760111882 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f3250013480 con 0x7f325c0671a0 2021-01-06T22:04:08.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.047+0000 7f3243fff700 1 --2- 172.21.15.203:0/3760111882 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f323c0563d0 0x7f323c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.047+0000 7f3243fff700 1 -- 172.21.15.203:0/3760111882 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f325006fba0 con 0x7f325c0671a0 2021-01-06T22:04:08.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.047+0000 7f325bfff700 1 --2- 172.21.15.203:0/3760111882 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f323c0563d0 0x7f323c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.047+0000 7f325bfff700 1 --2- 172.21.15.203:0/3760111882 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f323c0563d0 0x7f323c058890 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f324c00d9f0 tx=0x7f324c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:08.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.055+0000 7f3243fff700 1 -- 172.21.15.203:0/3760111882 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f32500452b0 con 0x7f325c0671a0 2021-01-06T22:04:08.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.203+0000 7f3262917700 1 -- 172.21.15.203:0/3760111882 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f325c05cfe0 con 0x7f325c0671a0 2021-01-06T22:04:08.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.207+0000 7f3243fff700 1 -- 172.21.15.203:0/3760111882 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f32500488d0 con 0x7f325c0671a0 2021-01-06T22:04:08.207 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:04:08.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.207+0000 7f3262917700 1 -- 172.21.15.203:0/3760111882 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f323c0563d0 msgr2=0x7f323c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.207+0000 7f3262917700 1 --2- 172.21.15.203:0/3760111882 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f323c0563d0 0x7f323c058890 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f324c00d9f0 tx=0x7f324c008040).stop 2021-01-06T22:04:08.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.207+0000 7f3262917700 1 -- 172.21.15.203:0/3760111882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325c0671a0 msgr2=0x7f325c063cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.207+0000 7f3262917700 1 --2- 172.21.15.203:0/3760111882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325c0671a0 0x7f325c063cb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f325000db60 tx=0x7f3250008040).stop 2021-01-06T22:04:08.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.207+0000 7f3262917700 1 -- 172.21.15.203:0/3760111882 shutdown_connections 2021-01-06T22:04:08.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.207+0000 7f3262917700 1 --2- 172.21.15.203:0/3760111882 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f323c0563d0 0x7f323c058890 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.207+0000 7f3262917700 1 --2- 172.21.15.203:0/3760111882 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f325c0bbac0 0x7f325c14c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.207+0000 7f3262917700 1 --2- 172.21.15.203:0/3760111882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f325c066dc0 0x7f325c0be840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.207+0000 7f3262917700 1 --2- 172.21.15.203:0/3760111882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325c0671a0 0x7f325c063cb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.207+0000 7f3262917700 1 -- 172.21.15.203:0/3760111882 >> 172.21.15.203:0/3760111882 conn(0x7f325c0626e0 msgr2=0x7f325c149770 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:08.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.207+0000 7f3262917700 1 -- 172.21.15.203:0/3760111882 shutdown_connections 2021-01-06T22:04:08.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.207+0000 7f3262917700 1 -- 172.21.15.203:0/3760111882 wait complete. 2021-01-06T22:04:08.222 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-01-06T22:04:08.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42c57700 1 Processor -- start 2021-01-06T22:04:08.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42c57700 1 -- start start 2021-01-06T22:04:08.348 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42c57700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a3c0662b0 0x7f0a3c0b9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42c57700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0a3c0602b0 0x7f0a3c155790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42c57700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3c0be330 con 0x7f0a3c065ed0 2021-01-06T22:04:08.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42c57700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3c0baa00 con 0x7f0a3c0602b0 2021-01-06T22:04:08.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42c57700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0bddf0 con 0x7f0a3c0662b0 2021-01-06T22:04:08.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42456700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0a3c0602b0 0x7f0a3c155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.350 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42456700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0a3c0602b0 0x7f0a3c155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49558/0 (socket says 172.21.15.203:49558) 2021-01-06T22:04:08.350 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42456700 1 -- 172.21.15.203:0/113355507 learned_addr learned my addr 172.21.15.203:0/113355507 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:08.350 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a41454700 1 --2- 172.21.15.203:0/113355507 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a3c0662b0 0x7f0a3c0b9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.350 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42456700 1 -- 172.21.15.203:0/113355507 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a3c0662b0 msgr2=0x7f0a3c0b9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:08.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42456700 1 --2- 172.21.15.203:0/113355507 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a3c0662b0 0x7f0a3c0b9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42456700 1 -- 172.21.15.203:0/113355507 >> v1:172.21.15.91:6789/0 conn(0x7f0a3c065ed0 legacy=0x7f0a3c0bcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42456700 1 -- 172.21.15.203:0/113355507 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c0be090 con 0x7f0a3c0602b0 2021-01-06T22:04:08.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42456700 1 --2- 172.21.15.203:0/113355507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c0602b0 0x7f0a3c155790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f0a3400b9e0 tx=0x7f0a34006ea0).ready entity=mon.1 client_cookie=ea9c3817c1e00014 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:08.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a40c53700 1 -- 172.21.15.203:0/113355507 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34009960 con 0x7f0a3c0602b0 2021-01-06T22:04:08.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a40c53700 1 -- 172.21.15.203:0/113355507 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a34009ac0 con 0x7f0a3c0602b0 2021-01-06T22:04:08.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a40c53700 1 -- 172.21.15.203:0/113355507 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34009e40 con 0x7f0a3c0602b0 2021-01-06T22:04:08.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42c57700 1 -- 172.21.15.203:0/113355507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c0602b0 msgr2=0x7f0a3c155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42c57700 1 --2- 172.21.15.203:0/113355507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c0602b0 0x7f0a3c155790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f0a3400b9e0 tx=0x7f0a34006ea0).stop 2021-01-06T22:04:08.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42c57700 1 -- 172.21.15.203:0/113355507 shutdown_connections 2021-01-06T22:04:08.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42c57700 1 --2- 172.21.15.203:0/113355507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c0602b0 0x7f0a3c155790 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42c57700 1 --2- 172.21.15.203:0/113355507 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a3c0662b0 0x7f0a3c0b9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42c57700 1 -- 172.21.15.203:0/113355507 >> 172.21.15.203:0/113355507 conn(0x7f0a3c0c2240 msgr2=0x7f0a3c0c2620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:08.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42c57700 1 -- 172.21.15.203:0/113355507 shutdown_connections 2021-01-06T22:04:08.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.347+0000 7f0a42c57700 1 -- 172.21.15.203:0/113355507 wait complete. 2021-01-06T22:04:08.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a42c57700 1 Processor -- start 2021-01-06T22:04:08.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a42c57700 1 -- start start 2021-01-06T22:04:08.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a42c57700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a3c0602b0 0x7f0a3c0630b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a42c57700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c065ed0 0x7f0a3c143be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a42c57700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a3c0662b0 0x7f0a3c13a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a42c57700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c139db0 con 0x7f0a3c0662b0 2021-01-06T22:04:08.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a42c57700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c145a90 con 0x7f0a3c065ed0 2021-01-06T22:04:08.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a42c57700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0b60f0 con 0x7f0a3c0602b0 2021-01-06T22:04:08.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a41454700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c065ed0 0x7f0a3c143be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a41454700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c065ed0 0x7f0a3c143be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49562/0 (socket says 172.21.15.203:49562) 2021-01-06T22:04:08.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a41454700 1 -- 172.21.15.203:0/1216436005 learned_addr learned my addr 172.21.15.203:0/1216436005 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:08.359 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a42456700 1 --2- 172.21.15.203:0/1216436005 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a3c0662b0 0x7f0a3c13a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.359 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a41c55700 1 --2- 172.21.15.203:0/1216436005 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a3c0602b0 0x7f0a3c0630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.359 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a41454700 1 -- 172.21.15.203:0/1216436005 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a3c0602b0 msgr2=0x7f0a3c0630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.359 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a41454700 1 --2- 172.21.15.203:0/1216436005 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a3c0602b0 0x7f0a3c0630b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.360 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a41454700 1 -- 172.21.15.203:0/1216436005 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a3c0662b0 msgr2=0x7f0a3c13a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.360 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a41454700 1 --2- 172.21.15.203:0/1216436005 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a3c0662b0 0x7f0a3c13a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.360 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a41454700 1 -- 172.21.15.203:0/1216436005 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c0b6390 con 0x7f0a3c065ed0 2021-01-06T22:04:08.360 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a41c55700 1 --2- 172.21.15.203:0/1216436005 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a3c0602b0 0x7f0a3c0630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:04:08.360 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a41454700 1 --2- 172.21.15.203:0/1216436005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c065ed0 0x7f0a3c143be0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f0a2c00d8e0 tx=0x7f0a2c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:08.361 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a29ffb700 1 -- 172.21.15.203:0/1216436005 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c009650 con 0x7f0a3c065ed0 2021-01-06T22:04:08.361 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a29ffb700 1 -- 172.21.15.203:0/1216436005 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c0097b0 con 0x7f0a3c065ed0 2021-01-06T22:04:08.361 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a29ffb700 1 -- 172.21.15.203:0/1216436005 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c009b50 con 0x7f0a3c065ed0 2021-01-06T22:04:08.361 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a42c57700 1 -- 172.21.15.203:0/1216436005 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c0b6630 con 0x7f0a3c065ed0 2021-01-06T22:04:08.361 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a42c57700 1 -- 172.21.15.203:0/1216436005 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c0635a0 con 0x7f0a3c065ed0 2021-01-06T22:04:08.362 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.351+0000 7f0a42c57700 1 -- 172.21.15.203:0/1216436005 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3c0450a0 con 0x7f0a3c065ed0 2021-01-06T22:04:08.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.355+0000 7f0a29ffb700 1 -- 172.21.15.203:0/1216436005 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f0a2c013480 con 0x7f0a3c065ed0 2021-01-06T22:04:08.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.355+0000 7f0a29ffb700 1 --2- 172.21.15.203:0/1216436005 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0a1c0563d0 0x7f0a1c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.355+0000 7f0a29ffb700 1 -- 172.21.15.203:0/1216436005 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0a2c06fab0 con 0x7f0a3c065ed0 2021-01-06T22:04:08.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.355+0000 7f0a41c55700 1 --2- 172.21.15.203:0/1216436005 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0a1c0563d0 0x7f0a1c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.355+0000 7f0a41c55700 1 --2- 172.21.15.203:0/1216436005 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0a1c0563d0 0x7f0a1c058890 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f0a38007fd0 tx=0x7f0a38009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:08.364 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.359+0000 7f0a29ffb700 1 -- 172.21.15.203:0/1216436005 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f0a2c0452d0 con 0x7f0a3c065ed0 2021-01-06T22:04:08.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.507+0000 7f0a42c57700 1 -- 172.21.15.203:0/1216436005 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a3c05ca20 con 0x7f0a3c065ed0 2021-01-06T22:04:08.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.507+0000 7f0a29ffb700 1 -- 172.21.15.203:0/1216436005 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0a2c0488f0 con 0x7f0a3c065ed0 2021-01-06T22:04:08.510 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:04:08.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.511+0000 7f0a42c57700 1 -- 172.21.15.203:0/1216436005 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0a1c0563d0 msgr2=0x7f0a1c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.511+0000 7f0a42c57700 1 --2- 172.21.15.203:0/1216436005 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0a1c0563d0 0x7f0a1c058890 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f0a38007fd0 tx=0x7f0a38009040).stop 2021-01-06T22:04:08.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.511+0000 7f0a42c57700 1 -- 172.21.15.203:0/1216436005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c065ed0 msgr2=0x7f0a3c143be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.511+0000 7f0a42c57700 1 --2- 172.21.15.203:0/1216436005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c065ed0 0x7f0a3c143be0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f0a2c00d8e0 tx=0x7f0a2c008040).stop 2021-01-06T22:04:08.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.511+0000 7f0a42c57700 1 -- 172.21.15.203:0/1216436005 shutdown_connections 2021-01-06T22:04:08.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.511+0000 7f0a42c57700 1 --2- 172.21.15.203:0/1216436005 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0a1c0563d0 0x7f0a1c058890 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.513 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.511+0000 7f0a42c57700 1 --2- 172.21.15.203:0/1216436005 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a3c0662b0 0x7f0a3c13a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.513 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.511+0000 7f0a42c57700 1 --2- 172.21.15.203:0/1216436005 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a3c0602b0 0x7f0a3c0630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.513 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.511+0000 7f0a42c57700 1 --2- 172.21.15.203:0/1216436005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c065ed0 0x7f0a3c143be0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.513 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.511+0000 7f0a42c57700 1 -- 172.21.15.203:0/1216436005 >> 172.21.15.203:0/1216436005 conn(0x7f0a3c0c2240 msgr2=0x7f0a3c0c2620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:08.514 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.511+0000 7f0a42c57700 1 -- 172.21.15.203:0/1216436005 shutdown_connections 2021-01-06T22:04:08.514 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.511+0000 7f0a42c57700 1 -- 172.21.15.203:0/1216436005 wait complete. 2021-01-06T22:04:08.524 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-01-06T22:04:08.655 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd154b8700 1 Processor -- start 2021-01-06T22:04:08.656 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd154b8700 1 -- start start 2021-01-06T22:04:08.657 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd154b8700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdd100633f0 0x7fdd1014e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.657 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd154b8700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd100be1b0 0x7fdd10147790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.657 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd154b8700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd100be990 con 0x7fdd10063010 2021-01-06T22:04:08.657 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd154b8700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1014ec60 con 0x7fdd100633f0 2021-01-06T22:04:08.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd154b8700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd100be6f0 con 0x7fdd100be1b0 2021-01-06T22:04:08.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd0f7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdd100633f0 0x7fdd1014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd0f7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdd100633f0 0x7fdd1014e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49572/0 (socket says 172.21.15.203:49572) 2021-01-06T22:04:08.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd0f7fe700 1 -- 172.21.15.203:0/2762747169 learned_addr learned my addr 172.21.15.203:0/2762747169 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:08.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd14cb7700 1 --2- 172.21.15.203:0/2762747169 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd100be1b0 0x7fdd10147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.660 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd0f7fe700 1 -- 172.21.15.203:0/2762747169 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd100be1b0 msgr2=0x7fdd10147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:08.660 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd0f7fe700 1 --2- 172.21.15.203:0/2762747169 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd100be1b0 0x7fdd10147790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.660 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd0f7fe700 1 -- 172.21.15.203:0/2762747169 >> v1:172.21.15.91:6789/0 conn(0x7fdd10063010 legacy=0x7fdd10064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.661 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd0f7fe700 1 -- 172.21.15.203:0/2762747169 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd100c1050 con 0x7fdd100633f0 2021-01-06T22:04:08.661 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd0f7fe700 1 --2- 172.21.15.203:0/2762747169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd100633f0 0x7fdd1014e770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fdd0400b9e0 tx=0x7fdd04006ea0).ready entity=mon.1 client_cookie=672b3748efcc571 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:08.661 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd0effd700 1 -- 172.21.15.203:0/2762747169 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04009960 con 0x7fdd100633f0 2021-01-06T22:04:08.661 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd0effd700 1 -- 172.21.15.203:0/2762747169 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd04009ac0 con 0x7fdd100633f0 2021-01-06T22:04:08.662 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.655+0000 7fdd0effd700 1 -- 172.21.15.203:0/2762747169 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04009e40 con 0x7fdd100633f0 2021-01-06T22:04:08.662 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd154b8700 1 -- 172.21.15.203:0/2762747169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd100633f0 msgr2=0x7fdd1014e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.662 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd154b8700 1 --2- 172.21.15.203:0/2762747169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd100633f0 0x7fdd1014e770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fdd0400b9e0 tx=0x7fdd04006ea0).stop 2021-01-06T22:04:08.662 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd154b8700 1 -- 172.21.15.203:0/2762747169 shutdown_connections 2021-01-06T22:04:08.662 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd154b8700 1 --2- 172.21.15.203:0/2762747169 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd100be1b0 0x7fdd10147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.663 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd154b8700 1 --2- 172.21.15.203:0/2762747169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd100633f0 0x7fdd1014e770 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.663 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd154b8700 1 -- 172.21.15.203:0/2762747169 >> 172.21.15.203:0/2762747169 conn(0x7fdd10067e20 msgr2=0x7fdd10068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:08.663 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd154b8700 1 -- 172.21.15.203:0/2762747169 shutdown_connections 2021-01-06T22:04:08.663 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd154b8700 1 -- 172.21.15.203:0/2762747169 wait complete. 2021-01-06T22:04:08.663 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd154b8700 1 Processor -- start 2021-01-06T22:04:08.664 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd154b8700 1 -- start start 2021-01-06T22:04:08.665 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd154b8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd10063010 0x7fdd100bb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd154b8700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd100633f0 0x7fdd101466d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd154b8700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd100be1b0 0x7fdd10146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd154b8700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd10148c90 con 0x7fdd100633f0 2021-01-06T22:04:08.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd154b8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1013d6a0 con 0x7fdd10063010 2021-01-06T22:04:08.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd154b8700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1015e0d0 con 0x7fdd100be1b0 2021-01-06T22:04:08.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd0ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd10063010 0x7fdd100bb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd0ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd10063010 0x7fdd100bb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49574/0 (socket says 172.21.15.203:49574) 2021-01-06T22:04:08.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd0ffff700 1 -- 172.21.15.203:0/3850044945 learned_addr learned my addr 172.21.15.203:0/3850044945 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:08.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd0f7fe700 1 --2- 172.21.15.203:0/3850044945 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd100633f0 0x7fdd101466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.668 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd0ffff700 1 -- 172.21.15.203:0/3850044945 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd100be1b0 msgr2=0x7fdd10146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.668 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd14cb7700 1 --2- 172.21.15.203:0/3850044945 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd100be1b0 0x7fdd10146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.668 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd0ffff700 1 --2- 172.21.15.203:0/3850044945 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd100be1b0 0x7fdd10146bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.668 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd0ffff700 1 -- 172.21.15.203:0/3850044945 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd100633f0 msgr2=0x7fdd101466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd0ffff700 1 --2- 172.21.15.203:0/3850044945 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd100633f0 0x7fdd101466d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd0ffff700 1 -- 172.21.15.203:0/3850044945 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1015e370 con 0x7fdd10063010 2021-01-06T22:04:08.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd14cb7700 1 --2- 172.21.15.203:0/3850044945 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd100be1b0 0x7fdd10146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:04:08.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdd0ffff700 1 --2- 172.21.15.203:0/3850044945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd10063010 0x7fdd100bb980 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fdd0000c740 tx=0x7fdd00007c00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:08.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.659+0000 7fdcf7fff700 1 -- 172.21.15.203:0/3850044945 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0000e560 con 0x7fdd10063010 2021-01-06T22:04:08.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.663+0000 7fdcf7fff700 1 -- 172.21.15.203:0/3850044945 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0000e6c0 con 0x7fdd10063010 2021-01-06T22:04:08.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.663+0000 7fdd154b8700 1 -- 172.21.15.203:0/3850044945 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd10148e80 con 0x7fdd10063010 2021-01-06T22:04:08.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.663+0000 7fdcf7fff700 1 -- 172.21.15.203:0/3850044945 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0000ea60 con 0x7fdd10063010 2021-01-06T22:04:08.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.663+0000 7fdd154b8700 1 -- 172.21.15.203:0/3850044945 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd10147d80 con 0x7fdd10063010 2021-01-06T22:04:08.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.663+0000 7fdcf7fff700 1 -- 172.21.15.203:0/3850044945 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fdd00019030 con 0x7fdd10063010 2021-01-06T22:04:08.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.663+0000 7fdd154b8700 1 -- 172.21.15.203:0/3850044945 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd10149320 con 0x7fdd10063010 2021-01-06T22:04:08.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.663+0000 7fdcf7fff700 1 --2- 172.21.15.203:0/3850044945 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdcf00563d0 0x7fdcf0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.663+0000 7fdd0f7fe700 1 --2- 172.21.15.203:0/3850044945 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdcf00563d0 0x7fdcf0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.663+0000 7fdcf7fff700 1 -- 172.21.15.203:0/3850044945 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdd0006eb20 con 0x7fdd10063010 2021-01-06T22:04:08.672 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.667+0000 7fdd0f7fe700 1 --2- 172.21.15.203:0/3850044945 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdcf00563d0 0x7fdcf0058890 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fdd0400b190 tx=0x7fdd04016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:08.672 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.671+0000 7fdcf7fff700 1 -- 172.21.15.203:0/3850044945 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fdd10149320 con 0x7fdd10063010 2021-01-06T22:04:08.825 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.823+0000 7fdd154b8700 1 -- 172.21.15.203:0/3850044945 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd1005c770 con 0x7fdd10063010 2021-01-06T22:04:08.826 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.823+0000 7fdcf7fff700 1 -- 172.21.15.203:0/3850044945 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7fdd00017030 con 0x7fdd10063010 2021-01-06T22:04:08.826 INFO:teuthology.orchestra.run.smithi203.stdout:pg_num: 8 2021-01-06T22:04:08.827 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.827+0000 7fdd154b8700 1 -- 172.21.15.203:0/3850044945 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdcf00563d0 msgr2=0x7fdcf0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.828 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.827+0000 7fdd154b8700 1 --2- 172.21.15.203:0/3850044945 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdcf00563d0 0x7fdcf0058890 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fdd0400b190 tx=0x7fdd04016000).stop 2021-01-06T22:04:08.828 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.827+0000 7fdd154b8700 1 -- 172.21.15.203:0/3850044945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd10063010 msgr2=0x7fdd100bb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.828 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.827+0000 7fdd154b8700 1 --2- 172.21.15.203:0/3850044945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd10063010 0x7fdd100bb980 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fdd0000c740 tx=0x7fdd00007c00).stop 2021-01-06T22:04:08.828 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.827+0000 7fdd154b8700 1 -- 172.21.15.203:0/3850044945 shutdown_connections 2021-01-06T22:04:08.829 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.827+0000 7fdd154b8700 1 --2- 172.21.15.203:0/3850044945 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdcf00563d0 0x7fdcf0058890 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.829 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.827+0000 7fdd154b8700 1 --2- 172.21.15.203:0/3850044945 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd100be1b0 0x7fdd10146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.829 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.827+0000 7fdd154b8700 1 --2- 172.21.15.203:0/3850044945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd10063010 0x7fdd100bb980 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.829 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.827+0000 7fdd154b8700 1 --2- 172.21.15.203:0/3850044945 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd100633f0 0x7fdd101466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.830 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.827+0000 7fdd154b8700 1 -- 172.21.15.203:0/3850044945 >> 172.21.15.203:0/3850044945 conn(0x7fdd10067e20 msgr2=0x7fdd1005c060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:08.830 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.827+0000 7fdd154b8700 1 -- 172.21.15.203:0/3850044945 shutdown_connections 2021-01-06T22:04:08.830 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.827+0000 7fdd154b8700 1 -- 172.21.15.203:0/3850044945 wait complete. 2021-01-06T22:04:08.841 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:04:08.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.959+0000 7fd3988da700 1 Processor -- start 2021-01-06T22:04:08.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.959+0000 7fd3988da700 1 -- start start 2021-01-06T22:04:08.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.959+0000 7fd3988da700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd394067b40 0x7fd39414e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.959+0000 7fd3988da700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3940beb40 con 0x7fd394067f20 2021-01-06T22:04:08.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.959+0000 7fd3988da700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd394152630 con 0x7fd3941521b0 2021-01-06T22:04:08.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.959+0000 7fd3988da700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3940be600 con 0x7fd394067b40 2021-01-06T22:04:08.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd39259c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd394067b40 0x7fd39414e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd392d9d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fd3941521b0 0x7fd3940bddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48180/0 (socket says 172.21.15.203:48180) 2021-01-06T22:04:08.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd392d9d700 1 -- 172.21.15.203:0/904988803 learned_addr learned my addr 172.21.15.203:0/904988803 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:04:08.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd39259c700 1 -- 172.21.15.203:0/904988803 >> v1:172.21.15.203:6789/0 conn(0x7fd3941521b0 legacy=0x7fd3940bddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd39159a700 1 -- 172.21.15.203:0/904988803 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 144705437 0 0) 0x7fd38c001450 con 0x7fd3941521b0 2021-01-06T22:04:08.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd39259c700 1 -- 172.21.15.203:0/904988803 >> v1:172.21.15.91:6789/0 conn(0x7fd394067f20 legacy=0x7fd3940b8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd39259c700 1 -- 172.21.15.203:0/904988803 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3940be8a0 con 0x7fd394067b40 2021-01-06T22:04:08.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd39259c700 1 --2- 172.21.15.203:0/904988803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd394067b40 0x7fd39414e160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fd38400b980 tx=0x7fd384006de0).ready entity=mon.1 client_cookie=fb8310d9c5864124 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:08.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd39159a700 1 -- 172.21.15.203:0/904988803 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3840096b0 con 0x7fd394067b40 2021-01-06T22:04:08.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd39159a700 1 -- 172.21.15.203:0/904988803 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd384009810 con 0x7fd394067b40 2021-01-06T22:04:08.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd39159a700 1 -- 172.21.15.203:0/904988803 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384009b90 con 0x7fd394067b40 2021-01-06T22:04:08.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd3988da700 1 -- 172.21.15.203:0/904988803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd394067b40 msgr2=0x7fd39414e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd3988da700 1 --2- 172.21.15.203:0/904988803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd394067b40 0x7fd39414e160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fd38400b980 tx=0x7fd384006de0).stop 2021-01-06T22:04:08.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd3988da700 1 -- 172.21.15.203:0/904988803 shutdown_connections 2021-01-06T22:04:08.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd3988da700 1 --2- 172.21.15.203:0/904988803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd394067b40 0x7fd39414e160 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd3988da700 1 -- 172.21.15.203:0/904988803 >> 172.21.15.203:0/904988803 conn(0x7fd394056cd0 msgr2=0x7fd3940570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:08.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd3988da700 1 -- 172.21.15.203:0/904988803 shutdown_connections 2021-01-06T22:04:08.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd3988da700 1 -- 172.21.15.203:0/904988803 wait complete. 2021-01-06T22:04:08.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd3988da700 1 Processor -- start 2021-01-06T22:04:08.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.963+0000 7fd3988da700 1 -- start start 2021-01-06T22:04:08.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd3988da700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd394067b40 0x7fd3940bf440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd3988da700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd394067f20 0x7fd394152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd3988da700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3941521b0 0x7fd39415dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd3988da700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940b9d30 con 0x7fd394067f20 2021-01-06T22:04:08.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd3988da700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd394161210 con 0x7fd394067b40 2021-01-06T22:04:08.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd3988da700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940bb9c0 con 0x7fd3941521b0 2021-01-06T22:04:08.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd39259c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd394067b40 0x7fd3940bf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd39259c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd394067b40 0x7fd3940bf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49588/0 (socket says 172.21.15.203:49588) 2021-01-06T22:04:08.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd39259c700 1 -- 172.21.15.203:0/683129875 learned_addr learned my addr 172.21.15.203:0/683129875 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:08.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd391d9b700 1 --2- 172.21.15.203:0/683129875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd394067f20 0x7fd394152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd392d9d700 1 --2- 172.21.15.203:0/683129875 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3941521b0 0x7fd39415dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd39259c700 1 -- 172.21.15.203:0/683129875 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3941521b0 msgr2=0x7fd39415dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd39259c700 1 --2- 172.21.15.203:0/683129875 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3941521b0 0x7fd39415dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd39259c700 1 -- 172.21.15.203:0/683129875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd394067f20 msgr2=0x7fd394152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:08.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd39259c700 1 --2- 172.21.15.203:0/683129875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd394067f20 0x7fd394152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:08.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd39259c700 1 -- 172.21.15.203:0/683129875 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd394162080 con 0x7fd394067b40 2021-01-06T22:04:08.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd39259c700 1 --2- 172.21.15.203:0/683129875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd394067b40 0x7fd3940bf440 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fd394067030 tx=0x7fd384016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:08.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd3827fc700 1 -- 172.21.15.203:0/683129875 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384011cc0 con 0x7fd394067b40 2021-01-06T22:04:08.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd3827fc700 1 -- 172.21.15.203:0/683129875 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd384011e20 con 0x7fd394067b40 2021-01-06T22:04:08.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd3988da700 1 -- 172.21.15.203:0/683129875 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3940b9a90 con 0x7fd394067b40 2021-01-06T22:04:08.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd3827fc700 1 -- 172.21.15.203:0/683129875 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38401d640 con 0x7fd394067b40 2021-01-06T22:04:08.975 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd3988da700 1 -- 172.21.15.203:0/683129875 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3941616a0 con 0x7fd394067b40 2021-01-06T22:04:08.975 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.967+0000 7fd3988da700 1 -- 172.21.15.203:0/683129875 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd394064e20 con 0x7fd394067b40 2021-01-06T22:04:08.976 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.971+0000 7fd3827fc700 1 -- 172.21.15.203:0/683129875 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fd38400f070 con 0x7fd394067b40 2021-01-06T22:04:08.976 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.971+0000 7fd3827fc700 1 --2- 172.21.15.203:0/683129875 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd3700563d0 0x7fd370058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:08.976 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.971+0000 7fd3827fc700 1 -- 172.21.15.203:0/683129875 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd384071100 con 0x7fd394067b40 2021-01-06T22:04:08.976 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.971+0000 7fd391d9b700 1 --2- 172.21.15.203:0/683129875 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd3700563d0 0x7fd370058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:08.977 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.971+0000 7fd391d9b700 1 --2- 172.21.15.203:0/683129875 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd3700563d0 0x7fd370058890 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fd39415ed80 tx=0x7fd388007a80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:08.978 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:08.975+0000 7fd3827fc700 1 -- 172.21.15.203:0/683129875 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fd384046870 con 0x7fd394067b40 2021-01-06T22:04:09.177 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.175+0000 7fd3988da700 1 -- 172.21.15.203:0/683129875 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd394061dc0 con 0x7fd394067b40 2021-01-06T22:04:09.178 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.175+0000 7fd3827fc700 1 -- 172.21.15.203:0/683129875 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+3574 (secure 0 0 0) 0x7fd384049e90 con 0x7fd394067b40 2021-01-06T22:04:09.178 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:09.178 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":5}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:03:44.792784+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1116012025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1116012025},{"type":"v1","addr":"172.21.15.203:6835","nonce":1116012025}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:04:09.180 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.179+0000 7fd3988da700 1 -- 172.21.15.203:0/683129875 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd3700563d0 msgr2=0x7fd370058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:09.180 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.179+0000 7fd3988da700 1 --2- 172.21.15.203:0/683129875 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd3700563d0 0x7fd370058890 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fd39415ed80 tx=0x7fd388007a80).stop 2021-01-06T22:04:09.180 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.179+0000 7fd3988da700 1 -- 172.21.15.203:0/683129875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd394067b40 msgr2=0x7fd3940bf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:09.180 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.179+0000 7fd3988da700 1 --2- 172.21.15.203:0/683129875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd394067b40 0x7fd3940bf440 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fd394067030 tx=0x7fd384016000).stop 2021-01-06T22:04:09.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.179+0000 7fd3988da700 1 -- 172.21.15.203:0/683129875 shutdown_connections 2021-01-06T22:04:09.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.179+0000 7fd3988da700 1 --2- 172.21.15.203:0/683129875 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd3700563d0 0x7fd370058890 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:09.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.179+0000 7fd3988da700 1 --2- 172.21.15.203:0/683129875 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3941521b0 0x7fd39415dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:09.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.179+0000 7fd3988da700 1 --2- 172.21.15.203:0/683129875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd394067b40 0x7fd3940bf440 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:09.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.179+0000 7fd3988da700 1 --2- 172.21.15.203:0/683129875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd394067f20 0x7fd394152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:09.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.179+0000 7fd3988da700 1 -- 172.21.15.203:0/683129875 >> 172.21.15.203:0/683129875 conn(0x7fd394056cd0 msgr2=0x7fd394065320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:09.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.179+0000 7fd3988da700 1 -- 172.21.15.203:0/683129875 shutdown_connections 2021-01-06T22:04:09.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.179+0000 7fd3988da700 1 -- 172.21.15.203:0/683129875 wait complete. 2021-01-06T22:04:09.183 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 12 2021-01-06T22:04:09.193 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-06T22:04:09.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b7363700 1 Processor -- start 2021-01-06T22:04:09.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b7363700 1 -- start start 2021-01-06T22:04:09.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b7363700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f07b00c44f0 0x7f07b00c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:09.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b7363700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f07b00c48d0 0x7f07b00c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:09.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b7363700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07b0060fc0 0x7f07b0150730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:09.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b7363700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b00c1d60 con 0x7f07b00c44f0 2021-01-06T22:04:09.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b7363700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b00c0f50 con 0x7f07b00c48d0 2021-01-06T22:04:09.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b7363700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b0066cf0 con 0x7f07b0060fc0 2021-01-06T22:04:09.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b5b60700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f07b00c48d0 0x7f07b00c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:09.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b5b60700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f07b00c48d0 0x7f07b00c1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49598/0 (socket says 172.21.15.203:49598) 2021-01-06T22:04:09.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b5b60700 1 -- 172.21.15.203:0/211296301 learned_addr learned my addr 172.21.15.203:0/211296301 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:09.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b6361700 1 --2- 172.21.15.203:0/211296301 >> v2:172.21.15.91:3300/0 conn(0x7f07b00c44f0 0x7f07b00c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:09.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b6b62700 1 --2- 172.21.15.203:0/211296301 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07b0060fc0 0x7f07b0150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:09.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b5b60700 1 -- 172.21.15.203:0/211296301 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07b0060fc0 msgr2=0x7f07b0150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:09.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b5b60700 1 --2- 172.21.15.203:0/211296301 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07b0060fc0 0x7f07b0150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:09.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b5b60700 1 -- 172.21.15.203:0/211296301 >> v2:172.21.15.91:3300/0 conn(0x7f07b00c44f0 msgr2=0x7f07b00c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:09.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b5b60700 1 --2- 172.21.15.203:0/211296301 >> v2:172.21.15.91:3300/0 conn(0x7f07b00c44f0 0x7f07b00c1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:09.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b5b60700 1 -- 172.21.15.203:0/211296301 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b0060750 con 0x7f07b00c48d0 2021-01-06T22:04:09.330 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.323+0000 7f07b5b60700 1 --2- 172.21.15.203:0/211296301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00c48d0 0x7f07b00c1870 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f07b0061480 tx=0x7f07a0006ea0).ready entity=mon.1 client_cookie=652059b24941b36d server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:09.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.327+0000 7f07b535f700 1 -- 172.21.15.203:0/211296301 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a00096e0 con 0x7f07b00c48d0 2021-01-06T22:04:09.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.327+0000 7f07b535f700 1 -- 172.21.15.203:0/211296301 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a0009840 con 0x7f07b00c48d0 2021-01-06T22:04:09.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.327+0000 7f07b535f700 1 -- 172.21.15.203:0/211296301 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0009bc0 con 0x7f07b00c48d0 2021-01-06T22:04:09.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.327+0000 7f07b7363700 1 -- 172.21.15.203:0/211296301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00c48d0 msgr2=0x7f07b00c1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:09.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.327+0000 7f07b7363700 1 --2- 172.21.15.203:0/211296301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00c48d0 0x7f07b00c1870 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f07b0061480 tx=0x7f07a0006ea0).stop 2021-01-06T22:04:09.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.327+0000 7f07b7363700 1 -- 172.21.15.203:0/211296301 shutdown_connections 2021-01-06T22:04:09.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.327+0000 7f07b7363700 1 --2- 172.21.15.203:0/211296301 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07b0060fc0 0x7f07b0150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:09.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.327+0000 7f07b7363700 1 --2- 172.21.15.203:0/211296301 >> v2:172.21.15.91:3300/0 conn(0x7f07b00c44f0 0x7f07b00c1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:09.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.327+0000 7f07b7363700 1 --2- 172.21.15.203:0/211296301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00c48d0 0x7f07b00c1870 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:09.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.327+0000 7f07b7363700 1 -- 172.21.15.203:0/211296301 >> 172.21.15.203:0/211296301 conn(0x7f07b0056cd0 msgr2=0x7f07b0061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:09.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.327+0000 7f07b7363700 1 -- 172.21.15.203:0/211296301 shutdown_connections 2021-01-06T22:04:09.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.327+0000 7f07b7363700 1 -- 172.21.15.203:0/211296301 wait complete. 2021-01-06T22:04:09.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.327+0000 7f07b7363700 1 Processor -- start 2021-01-06T22:04:09.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.327+0000 7f07b7363700 1 -- start start 2021-01-06T22:04:09.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b7363700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07b0060fc0 0x7f07b00b99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:09.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b7363700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00c44f0 0x7f07b014da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:09.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b7363700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f07b00c48d0 0x7f07b0150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:09.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b7363700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b0154210 con 0x7f07b00c48d0 2021-01-06T22:04:09.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b7363700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b015e6e0 con 0x7f07b00c44f0 2021-01-06T22:04:09.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b7363700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b0153a30 con 0x7f07b0060fc0 2021-01-06T22:04:09.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b5b60700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00c44f0 0x7f07b014da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:09.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b5b60700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00c44f0 0x7f07b014da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49604/0 (socket says 172.21.15.203:49604) 2021-01-06T22:04:09.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b5b60700 1 -- 172.21.15.203:0/3951624183 learned_addr learned my addr 172.21.15.203:0/3951624183 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b6b62700 1 --2- 172.21.15.203:0/3951624183 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f07b00c48d0 0x7f07b0150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b6361700 1 --2- 172.21.15.203:0/3951624183 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07b0060fc0 0x7f07b00b99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b5b60700 1 -- 172.21.15.203:0/3951624183 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07b0060fc0 msgr2=0x7f07b00b99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b5b60700 1 --2- 172.21.15.203:0/3951624183 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07b0060fc0 0x7f07b00b99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:09.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b5b60700 1 -- 172.21.15.203:0/3951624183 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f07b00c48d0 msgr2=0x7f07b0150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:09.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b5b60700 1 --2- 172.21.15.203:0/3951624183 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f07b00c48d0 0x7f07b0150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:09.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b5b60700 1 -- 172.21.15.203:0/3951624183 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b0153cd0 con 0x7f07b00c44f0 2021-01-06T22:04:09.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b6b62700 1 --2- 172.21.15.203:0/3951624183 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f07b00c48d0 0x7f07b0150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:04:09.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b5b60700 1 --2- 172.21.15.203:0/3951624183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00c44f0 0x7f07b014da90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f07a0006ee0 tx=0x7f07a0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:09.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f079e7fc700 1 -- 172.21.15.203:0/3951624183 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0011c40 con 0x7f07b00c44f0 2021-01-06T22:04:09.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f079e7fc700 1 -- 172.21.15.203:0/3951624183 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a0011da0 con 0x7f07b00c44f0 2021-01-06T22:04:09.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f079e7fc700 1 -- 172.21.15.203:0/3951624183 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a001c650 con 0x7f07b00c44f0 2021-01-06T22:04:09.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b7363700 1 -- 172.21.15.203:0/3951624183 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b0153f70 con 0x7f07b00c44f0 2021-01-06T22:04:09.342 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b7363700 1 -- 172.21.15.203:0/3951624183 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b00b9fe0 con 0x7f07b00c44f0 2021-01-06T22:04:09.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.331+0000 7f07b7363700 1 -- 172.21.15.203:0/3951624183 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b00450a0 con 0x7f07b00c44f0 2021-01-06T22:04:09.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.335+0000 7f079e7fc700 1 -- 172.21.15.203:0/3951624183 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f07a000f070 con 0x7f07b00c44f0 2021-01-06T22:04:09.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.335+0000 7f079e7fc700 1 --2- 172.21.15.203:0/3951624183 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f07900563d0 0x7f0790058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:09.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.335+0000 7f07b6361700 1 --2- 172.21.15.203:0/3951624183 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f07900563d0 0x7f0790058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:09.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.335+0000 7f079e7fc700 1 -- 172.21.15.203:0/3951624183 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f07a0070b10 con 0x7f07b00c44f0 2021-01-06T22:04:09.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.335+0000 7f07b6361700 1 --2- 172.21.15.203:0/3951624183 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f07900563d0 0x7f0790058890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f07ac00bb80 tx=0x7f07ac008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:09.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.339+0000 7f079e7fc700 1 -- 172.21.15.203:0/3951624183 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f07a0046390 con 0x7f07b00c44f0 2021-01-06T22:04:09.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.499+0000 7f07b7363700 1 -- 172.21.15.203:0/3951624183 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07b005ca60 con 0x7f07b00c44f0 2021-01-06T22:04:09.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.499+0000 7f079e7fc700 1 -- 172.21.15.203:0/3951624183 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f07a00499b0 con 0x7f07b00c44f0 2021-01-06T22:04:09.503 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:09.503 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":23,"fsid":"f96dbde2-6d40-40dc-b34a-9fb254ce805f","created":"2021-01-06T22:03:21.322671+0000","modified":"2021-01-06T22:03:40.744826+0000","last_up_change":"2021-01-06T22:03:31.115611+0000","last_in_change":"2021-01-06T22:03:31.115611+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-06T22:03:25.161465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-06T22:03:35.070364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-01-06T22:03:37.862354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-01-06T22:03:38.738416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e0b35c09-147c-4ccc-b566-c48b81e25412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":18347},{"type":"v1","addr":"172.21.15.203:6815","nonce":18347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":18347},{"type":"v1","addr":"172.21.15.203:6818","nonce":18347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":18347},{"type":"v1","addr":"172.21.15.203:6831","nonce":18347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6821","nonce":18347},{"type":"v1","addr":"172.21.15.203:6827","nonce":18347}]},"public_addr":"172.21.15.203:6815/18347","cluster_addr":"172.21.15.203:6818/18347","heartbeat_back_addr":"172.21.15.203:6831/18347","heartbeat_front_addr":"172.21.15.203:6827/18347","state":["exists","up"]},{"osd":1,"uuid":"8add59a0-2d58-4c1b-a8d1-62049ee9d85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":18337},{"type":"v1","addr":"172.21.15.203:6803","nonce":18337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":18337},{"type":"v1","addr":"172.21.15.203:6805","nonce":18337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":18337},{"type":"v1","addr":"172.21.15.203:6809","nonce":18337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":18337},{"type":"v1","addr":"172.21.15.203:6807","nonce":18337}]},"public_addr":"172.21.15.203:6803/18337","cluster_addr":"172.21.15.203:6805/18337","heartbeat_back_addr":"172.21.15.203:6809/18337","heartbeat_front_addr":"172.21.15.203:6807/18337","state":["exists","up"]},{"osd":2,"uuid":"b294a411-c7c9-477e-8eae-3b7bd0563ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":18354},{"type":"v1","addr":"172.21.15.203:6811","nonce":18354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":18354},{"type":"v1","addr":"172.21.15.203:6819","nonce":18354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6825","nonce":18354},{"type":"v1","addr":"172.21.15.203:6828","nonce":18354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6823","nonce":18354},{"type":"v1","addr":"172.21.15.203:6824","nonce":18354}]},"public_addr":"172.21.15.203:6811/18354","cluster_addr":"172.21.15.203:6819/18354","heartbeat_back_addr":"172.21.15.203:6828/18354","heartbeat_front_addr":"172.21.15.203:6824/18354","state":["exists","up"]},{"osd":3,"uuid":"5878a8f7-1a42-41c3-9810-0ac41a54dd6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":18346},{"type":"v1","addr":"172.21.15.203:6817","nonce":18346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":18346},{"type":"v1","addr":"172.21.15.203:6822","nonce":18346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":18346},{"type":"v1","addr":"172.21.15.203:6833","nonce":18346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":18346},{"type":"v1","addr":"172.21.15.203:6829","nonce":18346}]},"public_addr":"172.21.15.203:6817/18346","cluster_addr":"172.21.15.203:6822/18346","heartbeat_back_addr":"172.21.15.203:6833/18346","heartbeat_front_addr":"172.21.15.203:6829/18346","state":["exists","up"]},{"osd":4,"uuid":"f641a345-81e4-4400-a44c-d2f82fef7f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":18370},{"type":"v1","addr":"172.21.15.91:6809","nonce":18370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":18370},{"type":"v1","addr":"172.21.15.91:6811","nonce":18370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":18370},{"type":"v1","addr":"172.21.15.91:6816","nonce":18370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":18370},{"type":"v1","addr":"172.21.15.91:6813","nonce":18370}]},"public_addr":"172.21.15.91:6809/18370","cluster_addr":"172.21.15.91:6811/18370","heartbeat_back_addr":"172.21.15.91:6816/18370","heartbeat_front_addr":"172.21.15.91:6813/18370","state":["exists","up"]},{"osd":5,"uuid":"c1ff150d-68c6-4e6b-abd4-76af27bb2103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":18372},{"type":"v1","addr":"172.21.15.91:6801","nonce":18372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":18372},{"type":"v1","addr":"172.21.15.91:6803","nonce":18372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":18372},{"type":"v1","addr":"172.21.15.91:6807","nonce":18372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":18372},{"type":"v1","addr":"172.21.15.91:6805","nonce":18372}]},"public_addr":"172.21.15.91:6801/18372","cluster_addr":"172.21.15.91:6803/18372","heartbeat_back_addr":"172.21.15.91:6807/18372","heartbeat_front_addr":"172.21.15.91:6805/18372","state":["exists","up"]},{"osd":6,"uuid":"35a58e1a-0a0a-4f02-a477-adc029038d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6815","nonce":18371},{"type":"v1","addr":"172.21.15.91:6817","nonce":18371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":18371},{"type":"v1","addr":"172.21.15.91:6819","nonce":18371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":18371},{"type":"v1","addr":"172.21.15.91:6823","nonce":18371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":18371},{"type":"v1","addr":"172.21.15.91:6821","nonce":18371}]},"public_addr":"172.21.15.91:6817/18371","cluster_addr":"172.21.15.91:6819/18371","heartbeat_back_addr":"172.21.15.91:6823/18371","heartbeat_front_addr":"172.21.15.91:6821/18371","state":["exists","up"]},{"osd":7,"uuid":"809a2d89-4303-482a-8558-aef907814234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":18373},{"type":"v1","addr":"172.21.15.91:6825","nonce":18373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":18373},{"type":"v1","addr":"172.21.15.91:6827","nonce":18373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":18373},{"type":"v1","addr":"172.21.15.91:6831","nonce":18373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":18373},{"type":"v1","addr":"172.21.15.91:6829","nonce":18373}]},"public_addr":"172.21.15.91:6825/18373","cluster_addr":"172.21.15.91:6827/18373","heartbeat_back_addr":"172.21.15.91:6831/18373","heartbeat_front_addr":"172.21.15.91:6829/18373","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.875052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.103604+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.179225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.158512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.206073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:28.894840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.229708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-06T22:03:29.248192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-06T22:04:09.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.503+0000 7f07b7363700 1 -- 172.21.15.203:0/3951624183 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f07900563d0 msgr2=0x7f0790058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:09.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.503+0000 7f07b7363700 1 --2- 172.21.15.203:0/3951624183 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f07900563d0 0x7f0790058890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f07ac00bb80 tx=0x7f07ac008040).stop 2021-01-06T22:04:09.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.503+0000 7f07b7363700 1 -- 172.21.15.203:0/3951624183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00c44f0 msgr2=0x7f07b014da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:09.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.503+0000 7f07b7363700 1 --2- 172.21.15.203:0/3951624183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00c44f0 0x7f07b014da90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f07a0006ee0 tx=0x7f07a0016000).stop 2021-01-06T22:04:09.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.503+0000 7f07b7363700 1 -- 172.21.15.203:0/3951624183 shutdown_connections 2021-01-06T22:04:09.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.503+0000 7f07b7363700 1 --2- 172.21.15.203:0/3951624183 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f07900563d0 0x7f0790058890 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:09.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.503+0000 7f07b7363700 1 --2- 172.21.15.203:0/3951624183 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f07b00c48d0 0x7f07b0150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:09.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.503+0000 7f07b7363700 1 --2- 172.21.15.203:0/3951624183 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07b0060fc0 0x7f07b00b99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:09.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.503+0000 7f07b7363700 1 --2- 172.21.15.203:0/3951624183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00c44f0 0x7f07b014da90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:09.508 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.503+0000 7f07b7363700 1 -- 172.21.15.203:0/3951624183 >> 172.21.15.203:0/3951624183 conn(0x7f07b0056cd0 msgr2=0x7f07b00bcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:09.508 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.503+0000 7f07b7363700 1 -- 172.21.15.203:0/3951624183 shutdown_connections 2021-01-06T22:04:09.508 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:09.503+0000 7f07b7363700 1 -- 172.21.15.203:0/3951624183 wait complete. 2021-01-06T22:04:09.517 DEBUG:tasks.mds_thrash:Yielding 2021-01-06T22:04:09.518 INFO:teuthology.run_tasks:Running task workunit... 2021-01-06T22:04:09.532 INFO:tasks.workunit:Pulling workunits from ref d0ed162b51928c50f20cee111f8292828eda755e 2021-01-06T22:04:09.533 INFO:tasks.workunit:Making a separate scratch dir for every client... 2021-01-06T22:04:09.533 INFO:tasks.workunit:timeout=3h 2021-01-06T22:04:09.534 INFO:tasks.workunit:cleanup=True 2021-01-06T22:04:09.534 DEBUG:teuthology.orchestra.run.smithi203:> stat -- /home/ubuntu/cephtest/mnt.0 2021-01-06T22:04:09.535 INFO:tasks.mds_thrash.fs.[cephfs]:starting mds_do_thrash for fs cephfs 2021-01-06T22:04:09.535 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 8 secs before thrashing 2021-01-06T22:04:09.543 INFO:teuthology.orchestra.run.smithi203.stdout: File: /home/ubuntu/cephtest/mnt.0 2021-01-06T22:04:09.543 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2021-01-06T22:04:09.543 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 32h/50d Inode: 1 Links: 2 2021-01-06T22:04:09.544 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-06T22:04:09.544 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2021-01-06T22:04:09.544 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2021-01-06 22:03:39.747831841 +0000 2021-01-06T22:04:09.544 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2021-01-06 22:04:02.749918697 +0000 2021-01-06T22:04:09.544 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2021-01-06T22:04:09.545 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2021-01-06T22:04:09.545 DEBUG:teuthology.orchestra.run.smithi203:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2021-01-06T22:04:09.614 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d0ed162b51928c50f20cee111f8292828eda755e 2021-01-06T22:04:09.666 INFO:tasks.workunit.client.0.smithi203.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2021-01-06T22:04:10.863 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:04:10.867 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:04:17.537 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:04:17.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.667+0000 7f658091f700 1 Processor -- start 2021-01-06T22:04:17.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.667+0000 7f658091f700 1 -- start start 2021-01-06T22:04:17.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.667+0000 7f658091f700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f657c1478c0 0x7f657c0bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:17.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.667+0000 7f658091f700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657c0c16b0 con 0x7f657c064380 2021-01-06T22:04:17.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.667+0000 7f658091f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657c0bb2d0 con 0x7f657c14e310 2021-01-06T22:04:17.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.667+0000 7f658091f700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657c14a560 con 0x7f657c1478c0 2021-01-06T22:04:17.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.667+0000 7f657a59c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f657c14e310 0x7f657c1471b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48208/0 (socket says 172.21.15.203:48208) 2021-01-06T22:04:17.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.667+0000 7f657a59c700 1 -- 172.21.15.203:0/3888126354 learned_addr learned my addr 172.21.15.203:0/3888126354 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:04:17.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.667+0000 7f657b59e700 1 --2- 172.21.15.203:0/3888126354 >> v2:172.21.15.91:3300/0 conn(0x7f657c1478c0 0x7f657c0bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:17.673 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.667+0000 7f6579d9b700 1 -- 172.21.15.203:0/3888126354 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2727360411 0 0) 0x7f657c0bb2d0 con 0x7f657c14e310 2021-01-06T22:04:17.673 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.667+0000 7f6579d9b700 1 -- 172.21.15.203:0/3888126354 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6568003230 con 0x7f657c14e310 2021-01-06T22:04:17.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.667+0000 7f6579d9b700 1 -- 172.21.15.203:0/3888126354 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766025188 0 0) 0x7f656c0010d0 con 0x7f657c064380 2021-01-06T22:04:17.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.667+0000 7f6579d9b700 1 -- 172.21.15.203:0/3888126354 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f657c0bb2d0 con 0x7f657c064380 2021-01-06T22:04:17.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.667+0000 7f6579d9b700 1 -- 172.21.15.203:0/3888126354 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2716892865 0 0) 0x7f6568003230 con 0x7f657c14e310 2021-01-06T22:04:17.675 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.671+0000 7f657b59e700 1 -- 172.21.15.203:0/3888126354 >> v1:172.21.15.203:6789/0 conn(0x7f657c14e310 legacy=0x7f657c1471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:17.675 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.671+0000 7f657b59e700 1 -- 172.21.15.203:0/3888126354 >> v1:172.21.15.91:6789/0 conn(0x7f657c064380 legacy=0x7f657c0c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:17.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.671+0000 7f657b59e700 1 -- 172.21.15.203:0/3888126354 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c14a800 con 0x7f657c1478c0 2021-01-06T22:04:17.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.671+0000 7f657b59e700 1 --2- 172.21.15.203:0/3888126354 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f657c1478c0 0x7f657c0bb730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f6574009c10 tx=0x7f65740091d0).ready entity=mon.0 client_cookie=9bb86e1ee3ac1d3c server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:17.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.671+0000 7f6579d9b700 1 -- 172.21.15.203:0/3888126354 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65700014e0 con 0x7f657c1478c0 2021-01-06T22:04:17.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.671+0000 7f6579d9b700 1 -- 172.21.15.203:0/3888126354 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657400b7c0 con 0x7f657c1478c0 2021-01-06T22:04:17.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.671+0000 7f6579d9b700 1 -- 172.21.15.203:0/3888126354 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657400bb40 con 0x7f657c1478c0 2021-01-06T22:04:17.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.671+0000 7f658091f700 1 -- 172.21.15.203:0/3888126354 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f657c1478c0 msgr2=0x7f657c0bb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:17.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.671+0000 7f658091f700 1 --2- 172.21.15.203:0/3888126354 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f657c1478c0 0x7f657c0bb730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f6574009c10 tx=0x7f65740091d0).stop 2021-01-06T22:04:17.678 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.671+0000 7f658091f700 1 -- 172.21.15.203:0/3888126354 shutdown_connections 2021-01-06T22:04:17.678 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.671+0000 7f658091f700 1 --2- 172.21.15.203:0/3888126354 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f657c1478c0 0x7f657c0bb730 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:17.678 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.671+0000 7f658091f700 1 -- 172.21.15.203:0/3888126354 >> 172.21.15.203:0/3888126354 conn(0x7f657c0667b0 msgr2=0x7f657c0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:17.679 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.671+0000 7f658091f700 1 -- 172.21.15.203:0/3888126354 shutdown_connections 2021-01-06T22:04:17.679 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.671+0000 7f658091f700 1 -- 172.21.15.203:0/3888126354 wait complete. 2021-01-06T22:04:17.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f658091f700 1 Processor -- start 2021-01-06T22:04:17.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f658091f700 1 -- start start 2021-01-06T22:04:17.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f658091f700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f657c064380 0x7f657c150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:17.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f658091f700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f657c1478c0 0x7f657c150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:17.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f658091f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f657c14e310 0x7f657c146180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:17.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f658091f700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c151050 con 0x7f657c1478c0 2021-01-06T22:04:17.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f658091f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0ae680 con 0x7f657c14e310 2021-01-06T22:04:17.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f658091f700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0aedf0 con 0x7f657c064380 2021-01-06T22:04:17.684 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f657b59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f657c14e310 0x7f657c146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:17.684 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f657b59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f657c14e310 0x7f657c146180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49622/0 (socket says 172.21.15.203:49622) 2021-01-06T22:04:17.685 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f657b59e700 1 -- 172.21.15.203:0/3015769791 learned_addr learned my addr 172.21.15.203:0/3015769791 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:17.685 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f657a59c700 1 --2- 172.21.15.203:0/3015769791 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f657c1478c0 0x7f657c150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:17.685 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f657ad9d700 1 --2- 172.21.15.203:0/3015769791 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f657c064380 0x7f657c150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:17.686 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f657b59e700 1 -- 172.21.15.203:0/3015769791 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f657c064380 msgr2=0x7f657c150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:17.686 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f657b59e700 1 --2- 172.21.15.203:0/3015769791 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f657c064380 0x7f657c150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:17.686 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f657b59e700 1 -- 172.21.15.203:0/3015769791 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f657c1478c0 msgr2=0x7f657c150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:17.686 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f657b59e700 1 --2- 172.21.15.203:0/3015769791 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f657c1478c0 0x7f657c150b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:17.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f657b59e700 1 -- 172.21.15.203:0/3015769791 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c0ae8b0 con 0x7f657c14e310 2021-01-06T22:04:17.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f657b59e700 1 --2- 172.21.15.203:0/3015769791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f657c14e310 0x7f657c146180 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f6574011bc0 tx=0x7f6574017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:17.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f6562ffd700 1 -- 172.21.15.203:0/3015769791 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574011e40 con 0x7f657c14e310 2021-01-06T22:04:17.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f6562ffd700 1 -- 172.21.15.203:0/3015769791 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657401c630 con 0x7f657c14e310 2021-01-06T22:04:17.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f658091f700 1 -- 172.21.15.203:0/3015769791 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657c0aeb50 con 0x7f657c14e310 2021-01-06T22:04:17.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f6562ffd700 1 -- 172.21.15.203:0/3015769791 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657401c7a0 con 0x7f657c14e310 2021-01-06T22:04:17.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f658091f700 1 -- 172.21.15.203:0/3015769791 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c15b3c0 con 0x7f657c14e310 2021-01-06T22:04:17.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.675+0000 7f658091f700 1 -- 172.21.15.203:0/3015769791 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657c0450a0 con 0x7f657c14e310 2021-01-06T22:04:17.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.679+0000 7f6562ffd700 1 -- 172.21.15.203:0/3015769791 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f657402c070 con 0x7f657c14e310 2021-01-06T22:04:17.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.679+0000 7f6562ffd700 1 --2- 172.21.15.203:0/3015769791 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f655c056400 0x7f655c0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:17.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.679+0000 7f6562ffd700 1 -- 172.21.15.203:0/3015769791 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6574071360 con 0x7f657c14e310 2021-01-06T22:04:17.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.679+0000 7f657ad9d700 1 --2- 172.21.15.203:0/3015769791 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f655c056400 0x7f655c0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:17.690 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.679+0000 7f657ad9d700 1 --2- 172.21.15.203:0/3015769791 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f655c056400 0x7f655c0588c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f656c0077c0 tx=0x7f656c0027c0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:17.690 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.687+0000 7f6562ffd700 1 -- 172.21.15.203:0/3015769791 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f6574046ad0 con 0x7f657c14e310 2021-01-06T22:04:17.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.887+0000 7f658091f700 1 -- 172.21.15.203:0/3015769791 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f657c061ae0 con 0x7f657c14e310 2021-01-06T22:04:17.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.887+0000 7f6562ffd700 1 -- 172.21.15.203:0/3015769791 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+3574 (secure 0 0 0) 0x7f657402a020 con 0x7f657c14e310 2021-01-06T22:04:17.888 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:17.888 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":5}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:03:44.792784+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1116012025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1116012025},{"type":"v1","addr":"172.21.15.203:6835","nonce":1116012025}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:04:17.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.891+0000 7f658091f700 1 -- 172.21.15.203:0/3015769791 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f655c056400 msgr2=0x7f655c0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:17.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.891+0000 7f658091f700 1 --2- 172.21.15.203:0/3015769791 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f655c056400 0x7f655c0588c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f656c0077c0 tx=0x7f656c0027c0).stop 2021-01-06T22:04:17.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.891+0000 7f658091f700 1 -- 172.21.15.203:0/3015769791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f657c14e310 msgr2=0x7f657c146180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:17.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.891+0000 7f658091f700 1 --2- 172.21.15.203:0/3015769791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f657c14e310 0x7f657c146180 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f6574011bc0 tx=0x7f6574017000).stop 2021-01-06T22:04:17.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.891+0000 7f658091f700 1 -- 172.21.15.203:0/3015769791 shutdown_connections 2021-01-06T22:04:17.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.891+0000 7f658091f700 1 --2- 172.21.15.203:0/3015769791 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f655c056400 0x7f655c0588c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:17.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.891+0000 7f658091f700 1 --2- 172.21.15.203:0/3015769791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f657c14e310 0x7f657c146180 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:17.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.891+0000 7f658091f700 1 --2- 172.21.15.203:0/3015769791 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f657c064380 0x7f657c150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:17.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.891+0000 7f658091f700 1 --2- 172.21.15.203:0/3015769791 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f657c1478c0 0x7f657c150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:17.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.891+0000 7f658091f700 1 -- 172.21.15.203:0/3015769791 >> 172.21.15.203:0/3015769791 conn(0x7f657c0667b0 msgr2=0x7f657c0bb630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:17.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.891+0000 7f658091f700 1 -- 172.21.15.203:0/3015769791 shutdown_connections 2021-01-06T22:04:17.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:17.891+0000 7f658091f700 1 -- 172.21.15.203:0/3015769791 wait complete. 2021-01-06T22:04:17.894 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 12 2021-01-06T22:04:17.908 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=2) 2021-01-06T22:04:17.909 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-01-06T22:04:17.909 INFO:teuthology.orchestra.run:waiting for 300 2021-01-06T22:04:17.924 INFO:tasks.ceph.mds.c:Stopped 2021-01-06T22:04:17.924 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:04:18.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.091+0000 7fcb98bcb700 1 Processor -- start 2021-01-06T22:04:18.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.091+0000 7fcb98bcb700 1 -- start start 2021-01-06T22:04:18.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb98bcb700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb940bb3f0 0x7fcb940ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:18.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb98bcb700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcb940a5e50 0x7fcb940669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:18.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb98bcb700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb940646b0 con 0x7fcb940bb7d0 2021-01-06T22:04:18.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb98bcb700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb94064450 con 0x7fcb940a5e50 2021-01-06T22:04:18.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb98bcb700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb94068530 con 0x7fcb940bb3f0 2021-01-06T22:04:18.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb93fff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fcb940bb7d0 0x7fcb940602b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48224/0 (socket says 172.21.15.203:48224) 2021-01-06T22:04:18.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb93fff700 1 -- 172.21.15.203:0/351642453 learned_addr learned my addr 172.21.15.203:0/351642453 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:04:18.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb92ffd700 1 --2- 172.21.15.203:0/351642453 >> v2:172.21.15.91:3300/0 conn(0x7fcb940a5e50 0x7fcb940669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:18.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb937fe700 1 --2- 172.21.15.203:0/351642453 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb940bb3f0 0x7fcb940ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:18.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 -- 172.21.15.203:0/351642453 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206533779 0 0) 0x7fcb940646b0 con 0x7fcb940bb7d0 2021-01-06T22:04:18.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 -- 172.21.15.203:0/351642453 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb78003230 con 0x7fcb940bb7d0 2021-01-06T22:04:18.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 -- 172.21.15.203:0/351642453 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 669807081 0 0) 0x7fcb78003230 con 0x7fcb940bb7d0 2021-01-06T22:04:18.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 -- 172.21.15.203:0/351642453 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb940bb3f0 msgr2=0x7fcb940ba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:18.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 --2- 172.21.15.203:0/351642453 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb940bb3f0 0x7fcb940ba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:18.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 -- 172.21.15.203:0/351642453 >> v2:172.21.15.91:3300/0 conn(0x7fcb940a5e50 msgr2=0x7fcb940669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:18.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 --2- 172.21.15.203:0/351642453 >> v2:172.21.15.91:3300/0 conn(0x7fcb940a5e50 0x7fcb940669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:18.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 -- 172.21.15.203:0/351642453 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb94152cc0 con 0x7fcb940bb7d0 2021-01-06T22:04:18.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 -- 172.21.15.203:0/351642453 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7fcb8c0014e0 con 0x7fcb940bb7d0 2021-01-06T22:04:18.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb92ffd700 1 --2- 172.21.15.203:0/351642453 >> v2:172.21.15.91:3300/0 conn(0x7fcb940a5e50 0x7fcb940669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:04:18.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 -- 172.21.15.203:0/351642453 >> v1:172.21.15.203:6789/0 conn(0x7fcb940bb7d0 legacy=0x7fcb940602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:18.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 --2- 172.21.15.203:0/351642453 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb940bb3f0 0x7fcb78003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:18.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 --2- 172.21.15.203:0/351642453 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb78003c30 0x7fcb780044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:18.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 --2- 172.21.15.203:0/351642453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb780049c0 0x7fcb78004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:18.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 -- 172.21.15.203:0/351642453 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb94064450 con 0x7fcb78003c30 2021-01-06T22:04:18.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 -- 172.21.15.203:0/351642453 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb94068530 con 0x7fcb780049c0 2021-01-06T22:04:18.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 -- 172.21.15.203:0/351642453 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb780041f0 con 0x7fcb940bb3f0 2021-01-06T22:04:18.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb93fff700 1 --2- 172.21.15.203:0/351642453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb780049c0 0x7fcb78004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:18.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb937fe700 1 --2- 172.21.15.203:0/351642453 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb940bb3f0 0x7fcb78003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:18.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb92ffd700 1 --2- 172.21.15.203:0/351642453 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb78003c30 0x7fcb780044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:18.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb937fe700 1 -- 172.21.15.203:0/351642453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb780049c0 msgr2=0x7fcb78004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:18.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb937fe700 1 --2- 172.21.15.203:0/351642453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb780049c0 0x7fcb78004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:18.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb937fe700 1 -- 172.21.15.203:0/351642453 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb78003c30 msgr2=0x7fcb780044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:18.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb937fe700 1 --2- 172.21.15.203:0/351642453 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb78003c30 0x7fcb780044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:18.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb937fe700 1 -- 172.21.15.203:0/351642453 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb78003230 con 0x7fcb940bb3f0 2021-01-06T22:04:18.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb93fff700 1 --2- 172.21.15.203:0/351642453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb780049c0 0x7fcb78004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:04:18.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb937fe700 1 --2- 172.21.15.203:0/351642453 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb940bb3f0 0x7fcb78003720 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fcb8400bb90 tx=0x7fcb84008040).ready entity=mon.2 client_cookie=4461a168b9d26e72 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:18.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 -- 172.21.15.203:0/351642453 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8400b2b0 con 0x7fcb940bb3f0 2021-01-06T22:04:18.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 -- 172.21.15.203:0/351642453 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8400e900 con 0x7fcb940bb3f0 2021-01-06T22:04:18.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.095+0000 7fcb927fc700 1 -- 172.21.15.203:0/351642453 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8400eca0 con 0x7fcb940bb3f0 2021-01-06T22:04:18.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.099+0000 7fcb98bcb700 1 -- 172.21.15.203:0/351642453 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb940bb3f0 msgr2=0x7fcb78003720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:18.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.099+0000 7fcb98bcb700 1 --2- 172.21.15.203:0/351642453 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb940bb3f0 0x7fcb78003720 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fcb8400bb90 tx=0x7fcb84008040).stop 2021-01-06T22:04:18.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.099+0000 7fcb93fff700 1 -- 172.21.15.203:0/351642453 reap_dead start 2021-01-06T22:04:18.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.099+0000 7fcb98bcb700 1 -- 172.21.15.203:0/351642453 shutdown_connections 2021-01-06T22:04:18.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.099+0000 7fcb98bcb700 1 -- 172.21.15.203:0/351642453 >> 172.21.15.203:0/351642453 conn(0x7fcb94061530 msgr2=0x7fcb94063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:18.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.099+0000 7fcb98bcb700 1 -- 172.21.15.203:0/351642453 shutdown_connections 2021-01-06T22:04:18.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.099+0000 7fcb98bcb700 1 -- 172.21.15.203:0/351642453 wait complete. 2021-01-06T22:04:18.116 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.099+0000 7fcb98bcb700 1 Processor -- start 2021-01-06T22:04:18.116 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.099+0000 7fcb98bcb700 1 -- start start 2021-01-06T22:04:18.116 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb98bcb700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb940bb3f0 0x7fcb9414faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:18.117 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb98bcb700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb94151e40 0x7fcb940be1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:18.117 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb98bcb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb9414f390 0x7fcb94163e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:18.118 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb98bcb700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb94152310 con 0x7fcb940bb3f0 2021-01-06T22:04:18.118 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb98bcb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb940bac80 con 0x7fcb9414f390 2021-01-06T22:04:18.118 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb98bcb700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9409f9c0 con 0x7fcb94151e40 2021-01-06T22:04:18.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb93fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb9414f390 0x7fcb94163e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:18.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb93fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb9414f390 0x7fcb94163e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49642/0 (socket says 172.21.15.203:49642) 2021-01-06T22:04:18.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb937fe700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb940bb3f0 0x7fcb9414faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:18.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb92ffd700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb94151e40 0x7fcb940be1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:18.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb93fff700 1 -- 172.21.15.203:0/372905478 learned_addr learned my addr 172.21.15.203:0/372905478 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:18.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb93fff700 1 -- 172.21.15.203:0/372905478 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb94151e40 msgr2=0x7fcb940be1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:18.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb93fff700 1 --2- 172.21.15.203:0/372905478 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb94151e40 0x7fcb940be1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:18.122 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb93fff700 1 -- 172.21.15.203:0/372905478 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb940bb3f0 msgr2=0x7fcb9414faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:18.122 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb93fff700 1 --2- 172.21.15.203:0/372905478 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb940bb3f0 0x7fcb9414faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:18.122 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb93fff700 1 -- 172.21.15.203:0/372905478 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9409fc60 con 0x7fcb9414f390 2021-01-06T22:04:18.123 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb93fff700 1 --2- 172.21.15.203:0/372905478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb9414f390 0x7fcb94163e90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fcb940baef0 tx=0x7fcb8c0077f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:18.123 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb8b7fe700 1 -- 172.21.15.203:0/372905478 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c00f8a0 con 0x7fcb9414f390 2021-01-06T22:04:18.124 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb8b7fe700 1 -- 172.21.15.203:0/372905478 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8c00fa00 con 0x7fcb9414f390 2021-01-06T22:04:18.124 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb98bcb700 1 -- 172.21.15.203:0/372905478 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb9409ff00 con 0x7fcb9414f390 2021-01-06T22:04:18.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb8b7fe700 1 -- 172.21.15.203:0/372905478 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c00fd80 con 0x7fcb9414f390 2021-01-06T22:04:18.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.103+0000 7fcb98bcb700 1 -- 172.21.15.203:0/372905478 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb940be6d0 con 0x7fcb9414f390 2021-01-06T22:04:18.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.107+0000 7fcb98bcb700 1 -- 172.21.15.203:0/372905478 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb940450a0 con 0x7fcb9414f390 2021-01-06T22:04:18.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.107+0000 7fcb8b7fe700 1 -- 172.21.15.203:0/372905478 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fcb8c015620 con 0x7fcb9414f390 2021-01-06T22:04:18.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.107+0000 7fcb8b7fe700 1 --2- 172.21.15.203:0/372905478 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fcb6c0563d0 0x7fcb6c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:18.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.107+0000 7fcb937fe700 1 --2- 172.21.15.203:0/372905478 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fcb6c0563d0 0x7fcb6c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:18.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.107+0000 7fcb8b7fe700 1 -- 172.21.15.203:0/372905478 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcb8c06fa10 con 0x7fcb9414f390 2021-01-06T22:04:18.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.107+0000 7fcb937fe700 1 --2- 172.21.15.203:0/372905478 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fcb6c0563d0 0x7fcb6c058890 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fcb84006cf0 tx=0x7fcb8401a000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:18.128 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.111+0000 7fcb8b7fe700 1 -- 172.21.15.203:0/372905478 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fcb8c0451a0 con 0x7fcb9414f390 2021-01-06T22:04:18.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.311+0000 7fcb98bcb700 1 -- 172.21.15.203:0/372905478 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb940a0090 con 0x7fcb9414f390 2021-01-06T22:04:18.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.311+0000 7fcb8b7fe700 1 -- 172.21.15.203:0/372905478 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+3574 (secure 0 0 0) 0x7fcb8c0487c0 con 0x7fcb9414f390 2021-01-06T22:04:18.315 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:18.315 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":5}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:03:44.792784+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1116012025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1116012025},{"type":"v1","addr":"172.21.15.203:6835","nonce":1116012025}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:04:18.317 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.315+0000 7fcb98bcb700 1 -- 172.21.15.203:0/372905478 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fcb6c0563d0 msgr2=0x7fcb6c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:18.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.315+0000 7fcb98bcb700 1 --2- 172.21.15.203:0/372905478 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fcb6c0563d0 0x7fcb6c058890 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fcb84006cf0 tx=0x7fcb8401a000).stop 2021-01-06T22:04:18.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.315+0000 7fcb98bcb700 1 -- 172.21.15.203:0/372905478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb9414f390 msgr2=0x7fcb94163e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:18.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.315+0000 7fcb98bcb700 1 --2- 172.21.15.203:0/372905478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb9414f390 0x7fcb94163e90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fcb940baef0 tx=0x7fcb8c0077f0).stop 2021-01-06T22:04:18.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.315+0000 7fcb98bcb700 1 -- 172.21.15.203:0/372905478 shutdown_connections 2021-01-06T22:04:18.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.315+0000 7fcb98bcb700 1 --2- 172.21.15.203:0/372905478 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fcb6c0563d0 0x7fcb6c058890 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:18.320 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.315+0000 7fcb98bcb700 1 --2- 172.21.15.203:0/372905478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb9414f390 0x7fcb94163e90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:18.320 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.315+0000 7fcb98bcb700 1 --2- 172.21.15.203:0/372905478 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb940bb3f0 0x7fcb9414faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:18.320 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.315+0000 7fcb98bcb700 1 --2- 172.21.15.203:0/372905478 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb94151e40 0x7fcb940be1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:18.321 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.315+0000 7fcb98bcb700 1 -- 172.21.15.203:0/372905478 >> 172.21.15.203:0/372905478 conn(0x7fcb94061530 msgr2=0x7fcb940602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:18.321 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.315+0000 7fcb98bcb700 1 -- 172.21.15.203:0/372905478 shutdown_connections 2021-01-06T22:04:18.321 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:18.315+0000 7fcb98bcb700 1 -- 172.21.15.203:0/372905478 wait complete. 2021-01-06T22:04:18.321 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 12 2021-01-06T22:04:18.329 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:04:20.330 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:04:20.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.463+0000 7f99355fd700 1 Processor -- start 2021-01-06T22:04:20.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.463+0000 7f99355fd700 1 -- start start 2021-01-06T22:04:20.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.463+0000 7f99355fd700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f99300bb3f0 0x7f99300ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:20.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.463+0000 7f99355fd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f99300bb7d0 0x7f993014eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:20.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.463+0000 7f99355fd700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99300646b0 con 0x7f99300a5e50 2021-01-06T22:04:20.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.463+0000 7f99355fd700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99300670c0 con 0x7f99300bb3f0 2021-01-06T22:04:20.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.463+0000 7f99355fd700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9930064450 con 0x7f99300bb7d0 2021-01-06T22:04:20.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.463+0000 7f9934dfc700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f99300bb7d0 0x7f993014eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:20.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f9934dfc700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f99300bb7d0 0x7f993014eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49646/0 (socket says 172.21.15.203:49646) 2021-01-06T22:04:20.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f9934dfc700 1 -- 172.21.15.203:0/847747683 learned_addr learned my addr 172.21.15.203:0/847747683 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:20.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f992ffff700 1 --2- 172.21.15.203:0/847747683 >> v2:172.21.15.91:3300/0 conn(0x7f99300bb3f0 0x7f99300ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:20.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f9934dfc700 1 -- 172.21.15.203:0/847747683 >> v2:172.21.15.91:3300/0 conn(0x7f99300bb3f0 msgr2=0x7f99300ba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:20.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f9934dfc700 1 --2- 172.21.15.203:0/847747683 >> v2:172.21.15.91:3300/0 conn(0x7f99300bb3f0 0x7f99300ba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:20.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f9934dfc700 1 -- 172.21.15.203:0/847747683 >> v1:172.21.15.91:6789/0 conn(0x7f99300a5e50 legacy=0x7f99300669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:20.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f9934dfc700 1 -- 172.21.15.203:0/847747683 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9930152cc0 con 0x7f99300bb7d0 2021-01-06T22:04:20.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f992ffff700 1 --2- 172.21.15.203:0/847747683 >> v2:172.21.15.91:3300/0 conn(0x7f99300bb3f0 0x7f99300ba9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:04:20.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f9934dfc700 1 --2- 172.21.15.203:0/847747683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99300bb7d0 0x7f993014eef0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f99300bb130 tx=0x7f9928009290).ready entity=mon.1 client_cookie=a2c1f915ce8ab4da server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:20.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f992effd700 1 -- 172.21.15.203:0/847747683 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992800bbc0 con 0x7f99300bb7d0 2021-01-06T22:04:20.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f992effd700 1 -- 172.21.15.203:0/847747683 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992800bd20 con 0x7f99300bb7d0 2021-01-06T22:04:20.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f992effd700 1 -- 172.21.15.203:0/847747683 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928011610 con 0x7f99300bb7d0 2021-01-06T22:04:20.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f99355fd700 1 -- 172.21.15.203:0/847747683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99300bb7d0 msgr2=0x7f993014eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:20.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f99355fd700 1 --2- 172.21.15.203:0/847747683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99300bb7d0 0x7f993014eef0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f99300bb130 tx=0x7f9928009290).stop 2021-01-06T22:04:20.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f99355fd700 1 -- 172.21.15.203:0/847747683 shutdown_connections 2021-01-06T22:04:20.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f99355fd700 1 --2- 172.21.15.203:0/847747683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99300bb7d0 0x7f993014eef0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:20.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f99355fd700 1 --2- 172.21.15.203:0/847747683 >> v2:172.21.15.91:3300/0 conn(0x7f99300bb3f0 0x7f99300ba9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:20.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f99355fd700 1 -- 172.21.15.203:0/847747683 >> 172.21.15.203:0/847747683 conn(0x7f9930061530 msgr2=0x7f9930063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:20.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f99355fd700 1 -- 172.21.15.203:0/847747683 shutdown_connections 2021-01-06T22:04:20.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.467+0000 7f99355fd700 1 -- 172.21.15.203:0/847747683 wait complete. 2021-01-06T22:04:20.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.471+0000 7f99355fd700 1 Processor -- start 2021-01-06T22:04:20.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.471+0000 7f99355fd700 1 -- start start 2021-01-06T22:04:20.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.471+0000 7f99355fd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99300a5e50 0x7f993009faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:20.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.471+0000 7f99355fd700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99300bb3f0 0x7f99300a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:20.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.471+0000 7f99355fd700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99300bb7d0 0x7f99300a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:20.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.471+0000 7f99355fd700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993009f6b0 con 0x7f99300bb3f0 2021-01-06T22:04:20.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.471+0000 7f99355fd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9930159e60 con 0x7f99300a5e50 2021-01-06T22:04:20.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.471+0000 7f99355fd700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99300a5ca0 con 0x7f99300bb7d0 2021-01-06T22:04:20.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.471+0000 7f992ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99300a5e50 0x7f993009faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:20.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.471+0000 7f992ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99300a5e50 0x7f993009faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49652/0 (socket says 172.21.15.203:49652) 2021-01-06T22:04:20.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.471+0000 7f992ffff700 1 -- 172.21.15.203:0/1234445289 learned_addr learned my addr 172.21.15.203:0/1234445289 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:20.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.471+0000 7f9934dfc700 1 --2- 172.21.15.203:0/1234445289 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99300bb7d0 0x7f99300a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:20.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.471+0000 7f992f7fe700 1 --2- 172.21.15.203:0/1234445289 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99300bb3f0 0x7f99300a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:20.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.475+0000 7f992ffff700 1 -- 172.21.15.203:0/1234445289 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99300bb7d0 msgr2=0x7f99300a57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:20.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.475+0000 7f992ffff700 1 --2- 172.21.15.203:0/1234445289 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99300bb7d0 0x7f99300a57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:20.478 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.475+0000 7f992ffff700 1 -- 172.21.15.203:0/1234445289 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99300bb3f0 msgr2=0x7f99300a52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:20.478 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.475+0000 7f992ffff700 1 --2- 172.21.15.203:0/1234445289 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99300bb3f0 0x7f99300a52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:20.478 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.475+0000 7f992ffff700 1 -- 172.21.15.203:0/1234445289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99300a15f0 con 0x7f99300a5e50 2021-01-06T22:04:20.478 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.475+0000 7f992f7fe700 1 --2- 172.21.15.203:0/1234445289 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99300bb3f0 0x7f99300a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:04:20.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.475+0000 7f9934dfc700 1 --2- 172.21.15.203:0/1234445289 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99300bb7d0 0x7f99300a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:04:20.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.475+0000 7f992ffff700 1 --2- 172.21.15.203:0/1234445289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99300a5e50 0x7f993009faa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f992000cb80 tx=0x7f9920008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:20.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.475+0000 7f9927fff700 1 -- 172.21.15.203:0/1234445289 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992000a180 con 0x7f99300a5e50 2021-01-06T22:04:20.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.475+0000 7f9927fff700 1 -- 172.21.15.203:0/1234445289 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99200077d0 con 0x7f99300a5e50 2021-01-06T22:04:20.480 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.475+0000 7f99355fd700 1 -- 172.21.15.203:0/1234445289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9930151b80 con 0x7f99300a5e50 2021-01-06T22:04:20.480 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.475+0000 7f9927fff700 1 -- 172.21.15.203:0/1234445289 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920007b50 con 0x7f99300a5e50 2021-01-06T22:04:20.480 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.475+0000 7f99355fd700 1 -- 172.21.15.203:0/1234445289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9930159bc0 con 0x7f99300a5e50 2021-01-06T22:04:20.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.475+0000 7f9927fff700 1 -- 172.21.15.203:0/1234445289 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f992001b460 con 0x7f99300a5e50 2021-01-06T22:04:20.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.475+0000 7f99355fd700 1 -- 172.21.15.203:0/1234445289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99300450a0 con 0x7f99300a5e50 2021-01-06T22:04:20.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.475+0000 7f9927fff700 1 --2- 172.21.15.203:0/1234445289 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f99140563d0 0x7f9914058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:20.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.479+0000 7f9927fff700 1 -- 172.21.15.203:0/1234445289 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f992006f490 con 0x7f99300a5e50 2021-01-06T22:04:20.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.479+0000 7f992f7fe700 1 --2- 172.21.15.203:0/1234445289 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f99140563d0 0x7f9914058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:20.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.479+0000 7f992f7fe700 1 --2- 172.21.15.203:0/1234445289 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f99140563d0 0x7f9914058890 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f9930162970 tx=0x7f9918009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:20.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.483+0000 7f9927fff700 1 -- 172.21.15.203:0/1234445289 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f9920044cb0 con 0x7f99300a5e50 2021-01-06T22:04:20.684 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.683+0000 7f99355fd700 1 -- 172.21.15.203:0/1234445289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f993005ca60 con 0x7f99300a5e50 2021-01-06T22:04:20.685 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.683+0000 7f9927fff700 1 -- 172.21.15.203:0/1234445289 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+3574 (secure 0 0 0) 0x7f99200482d0 con 0x7f99300a5e50 2021-01-06T22:04:20.685 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:20.685 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":5}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:03:44.792784+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1116012025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1116012025},{"type":"v1","addr":"172.21.15.203:6835","nonce":1116012025}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:04:20.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.687+0000 7f99355fd700 1 -- 172.21.15.203:0/1234445289 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f99140563d0 msgr2=0x7f9914058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:20.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.687+0000 7f99355fd700 1 --2- 172.21.15.203:0/1234445289 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f99140563d0 0x7f9914058890 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f9930162970 tx=0x7f9918009040).stop 2021-01-06T22:04:20.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.687+0000 7f99355fd700 1 -- 172.21.15.203:0/1234445289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99300a5e50 msgr2=0x7f993009faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:20.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.687+0000 7f99355fd700 1 --2- 172.21.15.203:0/1234445289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99300a5e50 0x7f993009faa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f992000cb80 tx=0x7f9920008040).stop 2021-01-06T22:04:20.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.687+0000 7f99355fd700 1 -- 172.21.15.203:0/1234445289 shutdown_connections 2021-01-06T22:04:20.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.687+0000 7f99355fd700 1 --2- 172.21.15.203:0/1234445289 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f99140563d0 0x7f9914058890 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:20.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.687+0000 7f99355fd700 1 --2- 172.21.15.203:0/1234445289 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99300bb7d0 0x7f99300a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:20.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.687+0000 7f99355fd700 1 --2- 172.21.15.203:0/1234445289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99300a5e50 0x7f993009faa0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:20.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.687+0000 7f99355fd700 1 --2- 172.21.15.203:0/1234445289 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99300bb3f0 0x7f99300a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:20.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.687+0000 7f99355fd700 1 -- 172.21.15.203:0/1234445289 >> 172.21.15.203:0/1234445289 conn(0x7f9930061530 msgr2=0x7f99300bc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:20.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.687+0000 7f99355fd700 1 -- 172.21.15.203:0/1234445289 shutdown_connections 2021-01-06T22:04:20.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:20.687+0000 7f99355fd700 1 -- 172.21.15.203:0/1234445289 wait complete. 2021-01-06T22:04:20.690 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 12 2021-01-06T22:04:20.699 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:04:22.701 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:04:22.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c6e2f700 1 Processor -- start 2021-01-06T22:04:22.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c6e2f700 1 -- start start 2021-01-06T22:04:22.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c6e2f700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa7c00602b0 0x7fa7c014ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:22.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c6e2f700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7c00bd6d0 0x7fa7c014b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:22.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c6e2f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c00668d0 con 0x7fa7c014bbb0 2021-01-06T22:04:22.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c6e2f700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c0067a90 con 0x7fa7c00602b0 2021-01-06T22:04:22.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c6e2f700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c0066b70 con 0x7fa7c00bd6d0 2021-01-06T22:04:22.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c662e700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fa7c014bbb0 0x7fa7c0060690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48256/0 (socket says 172.21.15.203:48256) 2021-01-06T22:04:22.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c662e700 1 -- 172.21.15.203:0/3280224313 learned_addr learned my addr 172.21.15.203:0/3280224313 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:04:22.843 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c5e2d700 1 --2- 172.21.15.203:0/3280224313 >> v2:172.21.15.91:3300/0 conn(0x7fa7c00602b0 0x7fa7c014ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:22.843 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c562c700 1 --2- 172.21.15.203:0/3280224313 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7c00bd6d0 0x7fa7c014b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:22.843 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c4e2b700 1 -- 172.21.15.203:0/3280224313 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552998491 0 0) 0x7fa7c00668d0 con 0x7fa7c014bbb0 2021-01-06T22:04:22.843 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c4e2b700 1 -- 172.21.15.203:0/3280224313 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a8004340 con 0x7fa7c014bbb0 2021-01-06T22:04:22.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c5e2d700 1 -- 172.21.15.203:0/3280224313 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7c00bd6d0 msgr2=0x7fa7c014b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:22.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c5e2d700 1 --2- 172.21.15.203:0/3280224313 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7c00bd6d0 0x7fa7c014b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:22.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c5e2d700 1 -- 172.21.15.203:0/3280224313 >> v1:172.21.15.203:6789/0 conn(0x7fa7c014bbb0 legacy=0x7fa7c0060690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:22.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c4e2b700 1 -- 172.21.15.203:0/3280224313 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 56311 0 0) 0x7fa7a8004340 con 0x7fa7c014bbb0 2021-01-06T22:04:22.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c5e2d700 1 -- 172.21.15.203:0/3280224313 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c0066e10 con 0x7fa7c00602b0 2021-01-06T22:04:22.845 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c5e2d700 1 --2- 172.21.15.203:0/3280224313 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7c00602b0 0x7fa7c014ac90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fa7bc00b980 tx=0x7fa7bc006de0).ready entity=mon.0 client_cookie=abcfeb0f0f655cc3 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:22.845 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c4e2b700 1 -- 172.21.15.203:0/3280224313 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8001860 con 0x7fa7c00602b0 2021-01-06T22:04:22.845 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c4e2b700 1 -- 172.21.15.203:0/3280224313 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc011400 con 0x7fa7c00602b0 2021-01-06T22:04:22.845 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.839+0000 7fa7c4e2b700 1 -- 172.21.15.203:0/3280224313 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc011780 con 0x7fa7c00602b0 2021-01-06T22:04:22.845 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c6e2f700 1 -- 172.21.15.203:0/3280224313 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7c00602b0 msgr2=0x7fa7c014ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:22.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c6e2f700 1 --2- 172.21.15.203:0/3280224313 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7c00602b0 0x7fa7c014ac90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fa7bc00b980 tx=0x7fa7bc006de0).stop 2021-01-06T22:04:22.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c6e2f700 1 -- 172.21.15.203:0/3280224313 shutdown_connections 2021-01-06T22:04:22.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c6e2f700 1 --2- 172.21.15.203:0/3280224313 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7c00602b0 0x7fa7c014ac90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:22.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c6e2f700 1 --2- 172.21.15.203:0/3280224313 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7c00bd6d0 0x7fa7c014b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:22.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c6e2f700 1 -- 172.21.15.203:0/3280224313 >> 172.21.15.203:0/3280224313 conn(0x7fa7c0061530 msgr2=0x7fa7c0063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:22.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c6e2f700 1 -- 172.21.15.203:0/3280224313 shutdown_connections 2021-01-06T22:04:22.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c6e2f700 1 -- 172.21.15.203:0/3280224313 wait complete. 2021-01-06T22:04:22.848 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c6e2f700 1 Processor -- start 2021-01-06T22:04:22.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c6e2f700 1 -- start start 2021-01-06T22:04:22.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c6e2f700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7c00602b0 0x7fa7c0151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:22.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c6e2f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7c00bd6d0 0x7fa7c0146940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:22.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c6e2f700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7c014bbb0 0x7fa7c0150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:22.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c6e2f700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c0158ee0 con 0x7fa7c014bbb0 2021-01-06T22:04:22.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c6e2f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c015c4b0 con 0x7fa7c00bd6d0 2021-01-06T22:04:22.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c6e2f700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c015c6e0 con 0x7fa7c00602b0 2021-01-06T22:04:22.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c562c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7c00bd6d0 0x7fa7c0146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:22.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c562c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7c00bd6d0 0x7fa7c0146940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49670/0 (socket says 172.21.15.203:49670) 2021-01-06T22:04:22.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c562c700 1 -- 172.21.15.203:0/4157601002 learned_addr learned my addr 172.21.15.203:0/4157601002 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:22.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c5e2d700 1 --2- 172.21.15.203:0/4157601002 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7c00602b0 0x7fa7c0151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:22.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.843+0000 7fa7c662e700 1 --2- 172.21.15.203:0/4157601002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7c014bbb0 0x7fa7c0150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:22.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.847+0000 7fa7c562c700 1 -- 172.21.15.203:0/4157601002 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7c00602b0 msgr2=0x7fa7c0151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:22.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.847+0000 7fa7c562c700 1 --2- 172.21.15.203:0/4157601002 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7c00602b0 0x7fa7c0151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:22.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.847+0000 7fa7c562c700 1 -- 172.21.15.203:0/4157601002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7c014bbb0 msgr2=0x7fa7c0150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:22.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.847+0000 7fa7c562c700 1 --2- 172.21.15.203:0/4157601002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7c014bbb0 0x7fa7c0150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:22.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.847+0000 7fa7c562c700 1 -- 172.21.15.203:0/4157601002 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c015c980 con 0x7fa7c00bd6d0 2021-01-06T22:04:22.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.847+0000 7fa7c562c700 1 --2- 172.21.15.203:0/4157601002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7c00bd6d0 0x7fa7c0146940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fa7b000ebd0 tx=0x7fa7b0006de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:22.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.847+0000 7fa7adffb700 1 -- 172.21.15.203:0/4157601002 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b000c970 con 0x7fa7c00bd6d0 2021-01-06T22:04:22.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.847+0000 7fa7adffb700 1 -- 172.21.15.203:0/4157601002 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b000cad0 con 0x7fa7c00bd6d0 2021-01-06T22:04:22.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.847+0000 7fa7c6e2f700 1 -- 172.21.15.203:0/4157601002 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c015cc20 con 0x7fa7c00bd6d0 2021-01-06T22:04:22.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.847+0000 7fa7adffb700 1 -- 172.21.15.203:0/4157601002 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b000ce50 con 0x7fa7c00bd6d0 2021-01-06T22:04:22.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.847+0000 7fa7c6e2f700 1 -- 172.21.15.203:0/4157601002 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c015d160 con 0x7fa7c00bd6d0 2021-01-06T22:04:22.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.847+0000 7fa7c6e2f700 1 -- 172.21.15.203:0/4157601002 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c015d360 con 0x7fa7c00bd6d0 2021-01-06T22:04:22.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.847+0000 7fa7adffb700 1 -- 172.21.15.203:0/4157601002 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa7b0012600 con 0x7fa7c00bd6d0 2021-01-06T22:04:22.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.847+0000 7fa7adffb700 1 --2- 172.21.15.203:0/4157601002 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa7a00563d0 0x7fa7a0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:22.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.847+0000 7fa7adffb700 1 -- 172.21.15.203:0/4157601002 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa7b006f970 con 0x7fa7c00bd6d0 2021-01-06T22:04:22.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.847+0000 7fa7c5e2d700 1 --2- 172.21.15.203:0/4157601002 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa7a00563d0 0x7fa7a0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:22.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.851+0000 7fa7c5e2d700 1 --2- 172.21.15.203:0/4157601002 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa7a00563d0 0x7fa7a0058890 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fa7bc006e20 tx=0x7fa7bc016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:22.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:22.851+0000 7fa7adffb700 1 -- 172.21.15.203:0/4157601002 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa7b00451f0 con 0x7fa7c00bd6d0 2021-01-06T22:04:23.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:23.051+0000 7fa7c6e2f700 1 -- 172.21.15.203:0/4157601002 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7c00450a0 con 0x7fa7c00bd6d0 2021-01-06T22:04:23.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:23.055+0000 7fa7adffb700 1 -- 172.21.15.203:0/4157601002 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+3574 (secure 0 0 0) 0x7fa7b0048810 con 0x7fa7c00bd6d0 2021-01-06T22:04:23.054 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:23.055 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":5}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:03:44.792784+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1116012025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1116012025},{"type":"v1","addr":"172.21.15.203:6835","nonce":1116012025}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:04:23.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:23.055+0000 7fa7c6e2f700 1 -- 172.21.15.203:0/4157601002 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa7a00563d0 msgr2=0x7fa7a0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:23.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:23.055+0000 7fa7c6e2f700 1 --2- 172.21.15.203:0/4157601002 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa7a00563d0 0x7fa7a0058890 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fa7bc006e20 tx=0x7fa7bc016000).stop 2021-01-06T22:04:23.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:23.055+0000 7fa7c6e2f700 1 -- 172.21.15.203:0/4157601002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7c00bd6d0 msgr2=0x7fa7c0146940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:23.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:23.055+0000 7fa7c6e2f700 1 --2- 172.21.15.203:0/4157601002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7c00bd6d0 0x7fa7c0146940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fa7b000ebd0 tx=0x7fa7b0006de0).stop 2021-01-06T22:04:23.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:23.055+0000 7fa7c6e2f700 1 -- 172.21.15.203:0/4157601002 shutdown_connections 2021-01-06T22:04:23.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:23.055+0000 7fa7c6e2f700 1 --2- 172.21.15.203:0/4157601002 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa7a00563d0 0x7fa7a0058890 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:23.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:23.055+0000 7fa7c6e2f700 1 --2- 172.21.15.203:0/4157601002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7c014bbb0 0x7fa7c0150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:23.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:23.055+0000 7fa7c6e2f700 1 --2- 172.21.15.203:0/4157601002 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7c00602b0 0x7fa7c0151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:23.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:23.055+0000 7fa7c6e2f700 1 --2- 172.21.15.203:0/4157601002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7c00bd6d0 0x7fa7c0146940 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:23.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:23.055+0000 7fa7c6e2f700 1 -- 172.21.15.203:0/4157601002 >> 172.21.15.203:0/4157601002 conn(0x7fa7c0061530 msgr2=0x7fa7c00bf450 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:23.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:23.055+0000 7fa7c6e2f700 1 -- 172.21.15.203:0/4157601002 shutdown_connections 2021-01-06T22:04:23.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:23.055+0000 7fa7c6e2f700 1 -- 172.21.15.203:0/4157601002 wait complete. 2021-01-06T22:04:23.059 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 12 2021-01-06T22:04:23.069 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:04:25.069 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:04:25.203 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.203+0000 7fea8313d700 1 Processor -- start 2021-01-06T22:04:25.203 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.203+0000 7fea8313d700 1 -- start start 2021-01-06T22:04:25.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.203+0000 7fea8313d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fea7c148080 0x7fea7c068110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:25.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.203+0000 7fea8313d700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea7c0bba10 con 0x7fea7c067720 2021-01-06T22:04:25.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.203+0000 7fea8313d700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea7c0bb230 con 0x7fea7c067340 2021-01-06T22:04:25.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.203+0000 7fea8313d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea7c0bb4d0 con 0x7fea7c148080 2021-01-06T22:04:25.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.203+0000 7fea8213b700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fea7c067340 0x7fea7c147970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48272/0 (socket says 172.21.15.203:48272) 2021-01-06T22:04:25.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.203+0000 7fea8213b700 1 -- 172.21.15.203:0/2042321426 learned_addr learned my addr 172.21.15.203:0/2042321426 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:04:25.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.203+0000 7fea8293c700 1 --2- 172.21.15.203:0/2042321426 >> v2:172.21.15.203:3300/0 conn(0x7fea7c148080 0x7fea7c068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:25.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.203+0000 7fea81139700 1 -- 172.21.15.203:0/2042321426 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149662702 0 0) 0x7fea7c0bb230 con 0x7fea7c067340 2021-01-06T22:04:25.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.203+0000 7fea81139700 1 -- 172.21.15.203:0/2042321426 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea68003230 con 0x7fea7c067340 2021-01-06T22:04:25.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.203+0000 7fea8293c700 1 -- 172.21.15.203:0/2042321426 >> v1:172.21.15.203:6789/0 conn(0x7fea7c067340 legacy=0x7fea7c147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:25.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.203+0000 7fea81139700 1 -- 172.21.15.203:0/2042321426 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1402795517 0 0) 0x7fea68003230 con 0x7fea7c067340 2021-01-06T22:04:25.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.203+0000 7fea8293c700 1 -- 172.21.15.203:0/2042321426 >> v1:172.21.15.91:6789/0 conn(0x7fea7c067720 legacy=0x7fea7c1532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:25.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.207+0000 7fea8293c700 1 -- 172.21.15.203:0/2042321426 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c0bb770 con 0x7fea7c148080 2021-01-06T22:04:25.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.207+0000 7fea8293c700 1 --2- 172.21.15.203:0/2042321426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea7c148080 0x7fea7c068110 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fea6c009c10 tx=0x7fea6c0091d0).ready entity=mon.1 client_cookie=87334987d375ede4 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:25.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.207+0000 7fea81139700 1 -- 172.21.15.203:0/2042321426 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea740014e0 con 0x7fea7c148080 2021-01-06T22:04:25.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.207+0000 7fea81139700 1 -- 172.21.15.203:0/2042321426 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6c00b820 con 0x7fea7c148080 2021-01-06T22:04:25.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.207+0000 7fea81139700 1 -- 172.21.15.203:0/2042321426 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c00bba0 con 0x7fea7c148080 2021-01-06T22:04:25.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.207+0000 7fea8313d700 1 -- 172.21.15.203:0/2042321426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea7c148080 msgr2=0x7fea7c068110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:25.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.207+0000 7fea8313d700 1 --2- 172.21.15.203:0/2042321426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea7c148080 0x7fea7c068110 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fea6c009c10 tx=0x7fea6c0091d0).stop 2021-01-06T22:04:25.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.207+0000 7fea8313d700 1 -- 172.21.15.203:0/2042321426 shutdown_connections 2021-01-06T22:04:25.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.207+0000 7fea8313d700 1 --2- 172.21.15.203:0/2042321426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea7c148080 0x7fea7c068110 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:25.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.207+0000 7fea8313d700 1 -- 172.21.15.203:0/2042321426 >> 172.21.15.203:0/2042321426 conn(0x7fea7c0667b0 msgr2=0x7fea7c0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:25.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.207+0000 7fea8313d700 1 -- 172.21.15.203:0/2042321426 shutdown_connections 2021-01-06T22:04:25.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.207+0000 7fea8313d700 1 -- 172.21.15.203:0/2042321426 wait complete. 2021-01-06T22:04:25.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8313d700 1 Processor -- start 2021-01-06T22:04:25.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8313d700 1 -- start start 2021-01-06T22:04:25.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8313d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea7c067340 0x7fea7c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:25.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8313d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea7c067720 0x7fea7c15a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:25.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8313d700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea7c148080 0x7fea7c0609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:25.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8313d700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c15e420 con 0x7fea7c148080 2021-01-06T22:04:25.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8313d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c14eb70 con 0x7fea7c067340 2021-01-06T22:04:25.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8313d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c14e390 con 0x7fea7c067720 2021-01-06T22:04:25.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8213b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea7c067340 0x7fea7c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:25.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8213b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea7c067340 0x7fea7c15df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49684/0 (socket says 172.21.15.203:49684) 2021-01-06T22:04:25.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8213b700 1 -- 172.21.15.203:0/3880269236 learned_addr learned my addr 172.21.15.203:0/3880269236 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:25.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8193a700 1 --2- 172.21.15.203:0/3880269236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea7c067720 0x7fea7c15a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:25.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8293c700 1 --2- 172.21.15.203:0/3880269236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea7c148080 0x7fea7c0609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:25.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8213b700 1 -- 172.21.15.203:0/3880269236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea7c067720 msgr2=0x7fea7c15a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:25.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8213b700 1 --2- 172.21.15.203:0/3880269236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea7c067720 0x7fea7c15a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:25.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8213b700 1 -- 172.21.15.203:0/3880269236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea7c148080 msgr2=0x7fea7c0609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:25.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8213b700 1 --2- 172.21.15.203:0/3880269236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea7c148080 0x7fea7c0609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:25.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8213b700 1 -- 172.21.15.203:0/3880269236 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c14e630 con 0x7fea7c067340 2021-01-06T22:04:25.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8213b700 1 --2- 172.21.15.203:0/3880269236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea7c067340 0x7fea7c15df30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fea740013f0 tx=0x7fea74001af0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:25.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea7a7fc700 1 -- 172.21.15.203:0/3880269236 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7400ce50 con 0x7fea7c067340 2021-01-06T22:04:25.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea7a7fc700 1 -- 172.21.15.203:0/3880269236 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea74007170 con 0x7fea7c067340 2021-01-06T22:04:25.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8313d700 1 -- 172.21.15.203:0/3880269236 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea7c14e8d0 con 0x7fea7c067340 2021-01-06T22:04:25.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea7a7fc700 1 -- 172.21.15.203:0/3880269236 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7400a840 con 0x7fea7c067340 2021-01-06T22:04:25.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.211+0000 7fea8313d700 1 -- 172.21.15.203:0/3880269236 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7c15a3b0 con 0x7fea7c067340 2021-01-06T22:04:25.222 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.215+0000 7fea8313d700 1 -- 172.21.15.203:0/3880269236 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7c15a570 con 0x7fea7c067340 2021-01-06T22:04:25.222 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.215+0000 7fea7a7fc700 1 -- 172.21.15.203:0/3880269236 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fea74010070 con 0x7fea7c067340 2021-01-06T22:04:25.222 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.215+0000 7fea7a7fc700 1 --2- 172.21.15.203:0/3880269236 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fea5c056400 0x7fea5c0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:25.222 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.215+0000 7fea7a7fc700 1 -- 172.21.15.203:0/3880269236 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fea7406f290 con 0x7fea7c067340 2021-01-06T22:04:25.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.215+0000 7fea8193a700 1 --2- 172.21.15.203:0/3880269236 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fea5c056400 0x7fea5c0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:25.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.215+0000 7fea8193a700 1 --2- 172.21.15.203:0/3880269236 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fea5c056400 0x7fea5c0588c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fea7c068370 tx=0x7fea70007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:25.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.219+0000 7fea7a7fc700 1 -- 172.21.15.203:0/3880269236 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fea74044b10 con 0x7fea7c067340 2021-01-06T22:04:25.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.415+0000 7fea8313d700 1 -- 172.21.15.203:0/3880269236 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea7c0450a0 con 0x7fea7c067340 2021-01-06T22:04:25.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.415+0000 7fea7a7fc700 1 -- 172.21.15.203:0/3880269236 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+3574 (secure 0 0 0) 0x7fea74048130 con 0x7fea7c067340 2021-01-06T22:04:25.417 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:25.417 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":5}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:03:44.792784+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1116012025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1116012025},{"type":"v1","addr":"172.21.15.203:6835","nonce":1116012025}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:04:25.419 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.419+0000 7fea8313d700 1 -- 172.21.15.203:0/3880269236 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fea5c056400 msgr2=0x7fea5c0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:25.420 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.419+0000 7fea8313d700 1 --2- 172.21.15.203:0/3880269236 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fea5c056400 0x7fea5c0588c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fea7c068370 tx=0x7fea70007b80).stop 2021-01-06T22:04:25.420 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.419+0000 7fea8313d700 1 -- 172.21.15.203:0/3880269236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea7c067340 msgr2=0x7fea7c15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:25.420 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.419+0000 7fea8313d700 1 --2- 172.21.15.203:0/3880269236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea7c067340 0x7fea7c15df30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fea740013f0 tx=0x7fea74001af0).stop 2021-01-06T22:04:25.421 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.419+0000 7fea8313d700 1 -- 172.21.15.203:0/3880269236 shutdown_connections 2021-01-06T22:04:25.421 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.419+0000 7fea8313d700 1 --2- 172.21.15.203:0/3880269236 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fea5c056400 0x7fea5c0588c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:25.421 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.419+0000 7fea8313d700 1 --2- 172.21.15.203:0/3880269236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea7c148080 0x7fea7c0609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:25.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.419+0000 7fea8313d700 1 --2- 172.21.15.203:0/3880269236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea7c067340 0x7fea7c15df30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:25.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.419+0000 7fea8313d700 1 --2- 172.21.15.203:0/3880269236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea7c067720 0x7fea7c15a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:25.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.419+0000 7fea8313d700 1 -- 172.21.15.203:0/3880269236 >> 172.21.15.203:0/3880269236 conn(0x7fea7c0667b0 msgr2=0x7fea7c14a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:25.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.419+0000 7fea8313d700 1 -- 172.21.15.203:0/3880269236 shutdown_connections 2021-01-06T22:04:25.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:25.419+0000 7fea8313d700 1 -- 172.21.15.203:0/3880269236 wait complete. 2021-01-06T22:04:25.423 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 12 2021-01-06T22:04:25.438 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:04:27.439 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:04:27.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f1720700 1 Processor -- start 2021-01-06T22:04:27.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f1720700 1 -- start start 2021-01-06T22:04:27.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f1720700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa0ec0c2380 0x7fa0ec0a7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:27.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f1720700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa0ec0be6d0 0x7fa0ec0beb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:27.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f1720700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec0bd9d0 con 0x7fa0ec0c2760 2021-01-06T22:04:27.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f1720700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0ec0a7710 con 0x7fa0ec0c2380 2021-01-06T22:04:27.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f1720700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0ec0a79b0 con 0x7fa0ec0be6d0 2021-01-06T22:04:27.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f0f1f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa0ec0be6d0 0x7fa0ec0beb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:27.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0eb7fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fa0ec0c2760 0x7fa0ec0bd2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48294/0 (socket says 172.21.15.203:48294) 2021-01-06T22:04:27.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f0f1f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa0ec0be6d0 0x7fa0ec0beb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49702/0 (socket says 172.21.15.203:49702) 2021-01-06T22:04:27.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0eb7fe700 1 -- 172.21.15.203:0/537377102 learned_addr learned my addr 172.21.15.203:0/537377102 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:04:27.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0ebfff700 1 --2- 172.21.15.203:0/537377102 >> v2:172.21.15.91:3300/0 conn(0x7fa0ec0c2380 0x7fa0ec0a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:27.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f0f1f700 1 -- 172.21.15.203:0/537377102 >> v2:172.21.15.91:3300/0 conn(0x7fa0ec0c2380 msgr2=0x7fa0ec0a7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:27.576 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f0f1f700 1 --2- 172.21.15.203:0/537377102 >> v2:172.21.15.91:3300/0 conn(0x7fa0ec0c2380 0x7fa0ec0a7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:27.576 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f0f1f700 1 -- 172.21.15.203:0/537377102 >> v1:172.21.15.203:6789/0 conn(0x7fa0ec0c2760 legacy=0x7fa0ec0bd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:27.576 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f0f1f700 1 -- 172.21.15.203:0/537377102 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec0a7c50 con 0x7fa0ec0be6d0 2021-01-06T22:04:27.576 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f0f1f700 1 --2- 172.21.15.203:0/537377102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0ec0be6d0 0x7fa0ec0beb30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fa0ec065a30 tx=0x7fa0e4006de0).ready entity=mon.1 client_cookie=ae53733a9818fe6a server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:27.576 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0eaffd700 1 -- 172.21.15.203:0/537377102 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4009750 con 0x7fa0ec0be6d0 2021-01-06T22:04:27.577 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0eaffd700 1 -- 172.21.15.203:0/537377102 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e40098b0 con 0x7fa0ec0be6d0 2021-01-06T22:04:27.577 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0eaffd700 1 -- 172.21.15.203:0/537377102 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4009c30 con 0x7fa0ec0be6d0 2021-01-06T22:04:27.580 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f1720700 1 -- 172.21.15.203:0/537377102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0ec0be6d0 msgr2=0x7fa0ec0beb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:27.581 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f1720700 1 --2- 172.21.15.203:0/537377102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0ec0be6d0 0x7fa0ec0beb30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fa0ec065a30 tx=0x7fa0e4006de0).stop 2021-01-06T22:04:27.581 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f1720700 1 -- 172.21.15.203:0/537377102 shutdown_connections 2021-01-06T22:04:27.581 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f1720700 1 --2- 172.21.15.203:0/537377102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0ec0be6d0 0x7fa0ec0beb30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:27.581 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f1720700 1 --2- 172.21.15.203:0/537377102 >> v2:172.21.15.91:3300/0 conn(0x7fa0ec0c2380 0x7fa0ec0a7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:27.582 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f1720700 1 -- 172.21.15.203:0/537377102 >> 172.21.15.203:0/537377102 conn(0x7fa0ec0680f0 msgr2=0x7fa0ec056cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:27.582 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f1720700 1 -- 172.21.15.203:0/537377102 shutdown_connections 2021-01-06T22:04:27.582 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.571+0000 7fa0f1720700 1 -- 172.21.15.203:0/537377102 wait complete. 2021-01-06T22:04:27.582 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f1720700 1 Processor -- start 2021-01-06T22:04:27.583 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f1720700 1 -- start start 2021-01-06T22:04:27.583 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f1720700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa0ec0be6d0 0x7fa0ec063eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:27.583 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f1720700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0ec0c2380 0x7fa0ec0643a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:27.583 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f1720700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0ec0c2760 0x7fa0ec141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:27.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f1720700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec143fd0 con 0x7fa0ec0c2380 2021-01-06T22:04:27.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f1720700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec0b71e0 con 0x7fa0ec0c2760 2021-01-06T22:04:27.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f1720700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec0b7480 con 0x7fa0ec0be6d0 2021-01-06T22:04:27.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f0f1f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0ec0c2760 0x7fa0ec141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:27.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f0f1f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0ec0c2760 0x7fa0ec141870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49706/0 (socket says 172.21.15.203:49706) 2021-01-06T22:04:27.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f0f1f700 1 -- 172.21.15.203:0/1617968086 learned_addr learned my addr 172.21.15.203:0/1617968086 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:27.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0ebfff700 1 --2- 172.21.15.203:0/1617968086 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa0ec0be6d0 0x7fa0ec063eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:27.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0eb7fe700 1 --2- 172.21.15.203:0/1617968086 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0ec0c2380 0x7fa0ec0643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:27.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f0f1f700 1 -- 172.21.15.203:0/1617968086 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa0ec0be6d0 msgr2=0x7fa0ec063eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:27.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f0f1f700 1 --2- 172.21.15.203:0/1617968086 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa0ec0be6d0 0x7fa0ec063eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:27.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f0f1f700 1 -- 172.21.15.203:0/1617968086 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0ec0c2380 msgr2=0x7fa0ec0643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:27.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f0f1f700 1 --2- 172.21.15.203:0/1617968086 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0ec0c2380 0x7fa0ec0643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:27.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f0f1f700 1 -- 172.21.15.203:0/1617968086 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec1437f0 con 0x7fa0ec0c2760 2021-01-06T22:04:27.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f0f1f700 1 --2- 172.21.15.203:0/1617968086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0ec0c2760 0x7fa0ec141870 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fa0e4007fd0 tx=0x7fa0e4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:27.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0e3fff700 1 -- 172.21.15.203:0/1617968086 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e401b660 con 0x7fa0ec0c2760 2021-01-06T22:04:27.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0e3fff700 1 -- 172.21.15.203:0/1617968086 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e401b7c0 con 0x7fa0ec0c2760 2021-01-06T22:04:27.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f1720700 1 -- 172.21.15.203:0/1617968086 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ec143a90 con 0x7fa0ec0c2760 2021-01-06T22:04:27.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0e3fff700 1 -- 172.21.15.203:0/1617968086 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e401bb40 con 0x7fa0ec0c2760 2021-01-06T22:04:27.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.575+0000 7fa0f1720700 1 -- 172.21.15.203:0/1617968086 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ec1413d0 con 0x7fa0ec0c2760 2021-01-06T22:04:27.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.579+0000 7fa0e3fff700 1 -- 172.21.15.203:0/1617968086 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa0e400f070 con 0x7fa0ec0c2760 2021-01-06T22:04:27.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.579+0000 7fa0f1720700 1 -- 172.21.15.203:0/1617968086 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ec0450a0 con 0x7fa0ec0c2760 2021-01-06T22:04:27.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.579+0000 7fa0e3fff700 1 --2- 172.21.15.203:0/1617968086 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa0c40563d0 0x7fa0c4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:27.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.579+0000 7fa0e3fff700 1 -- 172.21.15.203:0/1617968086 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa0e40711d0 con 0x7fa0ec0c2760 2021-01-06T22:04:27.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.579+0000 7fa0ebfff700 1 --2- 172.21.15.203:0/1617968086 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa0c40563d0 0x7fa0c4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:27.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.579+0000 7fa0ebfff700 1 --2- 172.21.15.203:0/1617968086 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa0c40563d0 0x7fa0c4058890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fa0dc009d70 tx=0x7fa0dc0091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:27.590 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.587+0000 7fa0e3fff700 1 -- 172.21.15.203:0/1617968086 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa0e4046a50 con 0x7fa0ec0c2760 2021-01-06T22:04:27.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.791+0000 7fa0f1720700 1 -- 172.21.15.203:0/1617968086 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0ec05ca60 con 0x7fa0ec0c2760 2021-01-06T22:04:27.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.791+0000 7fa0e3fff700 1 -- 172.21.15.203:0/1617968086 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+3574 (secure 0 0 0) 0x7fa0e404a070 con 0x7fa0ec0c2760 2021-01-06T22:04:27.791 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:27.792 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":5}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:03:44.792784+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1116012025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1116012025},{"type":"v1","addr":"172.21.15.203:6835","nonce":1116012025}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:04:27.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.791+0000 7fa0f1720700 1 -- 172.21.15.203:0/1617968086 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa0c40563d0 msgr2=0x7fa0c4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:27.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.791+0000 7fa0f1720700 1 --2- 172.21.15.203:0/1617968086 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa0c40563d0 0x7fa0c4058890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fa0dc009d70 tx=0x7fa0dc0091d0).stop 2021-01-06T22:04:27.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.791+0000 7fa0f1720700 1 -- 172.21.15.203:0/1617968086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0ec0c2760 msgr2=0x7fa0ec141870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:27.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.791+0000 7fa0f1720700 1 --2- 172.21.15.203:0/1617968086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0ec0c2760 0x7fa0ec141870 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fa0e4007fd0 tx=0x7fa0e4016000).stop 2021-01-06T22:04:27.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.791+0000 7fa0f1720700 1 -- 172.21.15.203:0/1617968086 shutdown_connections 2021-01-06T22:04:27.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.791+0000 7fa0f1720700 1 --2- 172.21.15.203:0/1617968086 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa0c40563d0 0x7fa0c4058890 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:27.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.791+0000 7fa0f1720700 1 --2- 172.21.15.203:0/1617968086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0ec0c2760 0x7fa0ec141870 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:27.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.791+0000 7fa0f1720700 1 --2- 172.21.15.203:0/1617968086 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa0ec0be6d0 0x7fa0ec063eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:27.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.791+0000 7fa0f1720700 1 --2- 172.21.15.203:0/1617968086 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0ec0c2380 0x7fa0ec0643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:27.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.791+0000 7fa0f1720700 1 -- 172.21.15.203:0/1617968086 >> 172.21.15.203:0/1617968086 conn(0x7fa0ec0680f0 msgr2=0x7fa0ec0602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:27.796 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.791+0000 7fa0f1720700 1 -- 172.21.15.203:0/1617968086 shutdown_connections 2021-01-06T22:04:27.796 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:27.791+0000 7fa0f1720700 1 -- 172.21.15.203:0/1617968086 wait complete. 2021-01-06T22:04:27.796 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 12 2021-01-06T22:04:27.806 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:04:29.807 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:04:29.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.939+0000 7fab668ae700 1 Processor -- start 2021-01-06T22:04:29.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.939+0000 7fab668ae700 1 -- start start 2021-01-06T22:04:29.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab668ae700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fab600c0fe0 0x7fab600aa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:29.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab668ae700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab600a9ca0 con 0x7fab600c13c0 2021-01-06T22:04:29.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab668ae700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab60066a50 con 0x7fab600c03f0 2021-01-06T22:04:29.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab668ae700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab60066cf0 con 0x7fab600c0fe0 2021-01-06T22:04:29.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab658ac700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fab600c0fe0 0x7fab600aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:29.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab650ab700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fab600c03f0 0x7fab60060d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48314/0 (socket says 172.21.15.203:48314) 2021-01-06T22:04:29.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab650ab700 1 -- 172.21.15.203:0/2226310 learned_addr learned my addr 172.21.15.203:0/2226310 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:04:29.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab658ac700 1 -- 172.21.15.203:0/2226310 >> v1:172.21.15.203:6789/0 conn(0x7fab600c03f0 legacy=0x7fab60060d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:29.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab658ac700 1 -- 172.21.15.203:0/2226310 >> v1:172.21.15.91:6789/0 conn(0x7fab600c13c0 legacy=0x7fab60150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:29.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab648aa700 1 -- 172.21.15.203:0/2226310 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143547021 0 0) 0x7fab60066a50 con 0x7fab600c03f0 2021-01-06T22:04:29.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab658ac700 1 -- 172.21.15.203:0/2226310 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab60068460 con 0x7fab600c0fe0 2021-01-06T22:04:29.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab658ac700 1 --2- 172.21.15.203:0/2226310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab600c0fe0 0x7fab600aa030 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fab58009c90 tx=0x7fab580091d0).ready entity=mon.1 client_cookie=1ebc8582dad1020c server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:29.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab648aa700 1 -- 172.21.15.203:0/2226310 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5800bac0 con 0x7fab600c0fe0 2021-01-06T22:04:29.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab648aa700 1 -- 172.21.15.203:0/2226310 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5800bc20 con 0x7fab600c0fe0 2021-01-06T22:04:29.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab648aa700 1 -- 172.21.15.203:0/2226310 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58011400 con 0x7fab600c0fe0 2021-01-06T22:04:29.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab668ae700 1 -- 172.21.15.203:0/2226310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab600c0fe0 msgr2=0x7fab600aa030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:29.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab668ae700 1 --2- 172.21.15.203:0/2226310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab600c0fe0 0x7fab600aa030 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fab58009c90 tx=0x7fab580091d0).stop 2021-01-06T22:04:29.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab668ae700 1 -- 172.21.15.203:0/2226310 shutdown_connections 2021-01-06T22:04:29.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab668ae700 1 --2- 172.21.15.203:0/2226310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab600c0fe0 0x7fab600aa030 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:29.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab668ae700 1 -- 172.21.15.203:0/2226310 >> 172.21.15.203:0/2226310 conn(0x7fab60061530 msgr2=0x7fab60061910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:29.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab668ae700 1 -- 172.21.15.203:0/2226310 shutdown_connections 2021-01-06T22:04:29.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.943+0000 7fab668ae700 1 -- 172.21.15.203:0/2226310 wait complete. 2021-01-06T22:04:29.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab668ae700 1 Processor -- start 2021-01-06T22:04:29.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab668ae700 1 -- start start 2021-01-06T22:04:29.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab668ae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab600c03f0 0x7fab600b8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:29.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab668ae700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab600c0fe0 0x7fab600bb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:29.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab668ae700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab600c13c0 0x7fab60149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:29.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab668ae700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab600bea40 con 0x7fab600c0fe0 2021-01-06T22:04:29.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab668ae700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab60160c60 con 0x7fab600c03f0 2021-01-06T22:04:29.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab668ae700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab600b9970 con 0x7fab600c13c0 2021-01-06T22:04:29.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab658ac700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab600c03f0 0x7fab600b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:29.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab658ac700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab600c03f0 0x7fab600b8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49724/0 (socket says 172.21.15.203:49724) 2021-01-06T22:04:29.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab658ac700 1 -- 172.21.15.203:0/4215056453 learned_addr learned my addr 172.21.15.203:0/4215056453 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:29.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab650ab700 1 --2- 172.21.15.203:0/4215056453 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab600c0fe0 0x7fab600bb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:29.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab660ad700 1 --2- 172.21.15.203:0/4215056453 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab600c13c0 0x7fab60149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:29.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab658ac700 1 -- 172.21.15.203:0/4215056453 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab600c13c0 msgr2=0x7fab60149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:29.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab658ac700 1 --2- 172.21.15.203:0/4215056453 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab600c13c0 0x7fab60149d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:29.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab658ac700 1 -- 172.21.15.203:0/4215056453 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab600c0fe0 msgr2=0x7fab600bb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:29.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab658ac700 1 --2- 172.21.15.203:0/4215056453 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab600c0fe0 0x7fab600bb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:29.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab658ac700 1 -- 172.21.15.203:0/4215056453 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab600b9c10 con 0x7fab600c03f0 2021-01-06T22:04:29.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab658ac700 1 --2- 172.21.15.203:0/4215056453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab600c03f0 0x7fab600b8130 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fab60064c30 tx=0x7fab58017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:29.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab4dffb700 1 -- 172.21.15.203:0/4215056453 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58006ad0 con 0x7fab600c03f0 2021-01-06T22:04:29.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab4dffb700 1 -- 172.21.15.203:0/4215056453 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab58011e20 con 0x7fab600c03f0 2021-01-06T22:04:29.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab668ae700 1 -- 172.21.15.203:0/4215056453 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab600b9eb0 con 0x7fab600c03f0 2021-01-06T22:04:29.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab4dffb700 1 -- 172.21.15.203:0/4215056453 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5801c840 con 0x7fab600c03f0 2021-01-06T22:04:29.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab668ae700 1 -- 172.21.15.203:0/4215056453 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6015d710 con 0x7fab600c03f0 2021-01-06T22:04:29.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.947+0000 7fab668ae700 1 -- 172.21.15.203:0/4215056453 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab600b7d60 con 0x7fab600c03f0 2021-01-06T22:04:29.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.951+0000 7fab4dffb700 1 -- 172.21.15.203:0/4215056453 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fab5802c070 con 0x7fab600c03f0 2021-01-06T22:04:29.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.951+0000 7fab4dffb700 1 --2- 172.21.15.203:0/4215056453 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fab400563d0 0x7fab40058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:29.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.951+0000 7fab4dffb700 1 -- 172.21.15.203:0/4215056453 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fab580714f0 con 0x7fab600c03f0 2021-01-06T22:04:29.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.951+0000 7fab650ab700 1 --2- 172.21.15.203:0/4215056453 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fab400563d0 0x7fab40058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:29.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.951+0000 7fab650ab700 1 --2- 172.21.15.203:0/4215056453 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fab400563d0 0x7fab40058890 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fab60066c60 tx=0x7fab50007bf0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:29.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:29.959+0000 7fab4dffb700 1 -- 172.21.15.203:0/4215056453 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fab58046c60 con 0x7fab600c03f0 2021-01-06T22:04:30.160 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:30.159+0000 7fab668ae700 1 -- 172.21.15.203:0/4215056453 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab6005ca30 con 0x7fab600c03f0 2021-01-06T22:04:30.162 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:30.159+0000 7fab4dffb700 1 -- 172.21.15.203:0/4215056453 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+3574 (secure 0 0 0) 0x7fab58020020 con 0x7fab600c03f0 2021-01-06T22:04:30.163 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:30.163 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":5}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:03:44.792784+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1116012025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1116012025},{"type":"v1","addr":"172.21.15.203:6835","nonce":1116012025}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:04:30.164 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:30.163+0000 7fab668ae700 1 -- 172.21.15.203:0/4215056453 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fab400563d0 msgr2=0x7fab40058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:30.164 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:30.163+0000 7fab668ae700 1 --2- 172.21.15.203:0/4215056453 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fab400563d0 0x7fab40058890 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fab60066c60 tx=0x7fab50007bf0).stop 2021-01-06T22:04:30.165 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:30.163+0000 7fab668ae700 1 -- 172.21.15.203:0/4215056453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab600c03f0 msgr2=0x7fab600b8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:30.165 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:30.163+0000 7fab668ae700 1 --2- 172.21.15.203:0/4215056453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab600c03f0 0x7fab600b8130 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fab60064c30 tx=0x7fab58017000).stop 2021-01-06T22:04:30.165 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:30.163+0000 7fab668ae700 1 -- 172.21.15.203:0/4215056453 shutdown_connections 2021-01-06T22:04:30.165 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:30.163+0000 7fab668ae700 1 --2- 172.21.15.203:0/4215056453 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fab400563d0 0x7fab40058890 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:30.165 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:30.163+0000 7fab668ae700 1 --2- 172.21.15.203:0/4215056453 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab600c13c0 0x7fab60149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:30.166 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:30.163+0000 7fab668ae700 1 --2- 172.21.15.203:0/4215056453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab600c03f0 0x7fab600b8130 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:30.166 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:30.163+0000 7fab668ae700 1 --2- 172.21.15.203:0/4215056453 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab600c0fe0 0x7fab600bb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:30.166 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:30.163+0000 7fab668ae700 1 -- 172.21.15.203:0/4215056453 >> 172.21.15.203:0/4215056453 conn(0x7fab60061530 msgr2=0x7fab6014beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:30.166 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:30.163+0000 7fab668ae700 1 -- 172.21.15.203:0/4215056453 shutdown_connections 2021-01-06T22:04:30.167 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:30.163+0000 7fab668ae700 1 -- 172.21.15.203:0/4215056453 wait complete. 2021-01-06T22:04:30.167 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 12 2021-01-06T22:04:30.176 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:04:32.177 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:04:32.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.311+0000 7fd95ec4b700 1 Processor -- start 2021-01-06T22:04:32.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.311+0000 7fd95ec4b700 1 -- start start 2021-01-06T22:04:32.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.311+0000 7fd95ec4b700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd958064380 0x7fd9580c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:32.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.311+0000 7fd95ec4b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd95814e310 0x7fd9581471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:32.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.311+0000 7fd95ec4b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd958064760 0x7fd9581476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:32.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.311+0000 7fd95ec4b700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd958147b90 con 0x7fd958064380 2021-01-06T22:04:32.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.311+0000 7fd95ec4b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9580bb2d0 con 0x7fd95814e310 2021-01-06T22:04:32.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.311+0000 7fd95ec4b700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95814a560 con 0x7fd958064760 2021-01-06T22:04:32.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.311+0000 7fd95d448700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd95814e310 0x7fd9581471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:32.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.311+0000 7fd95d448700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd95814e310 0x7fd9581471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49730/0 (socket says 172.21.15.203:49730) 2021-01-06T22:04:32.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.311+0000 7fd95d448700 1 -- 172.21.15.203:0/3852750647 learned_addr learned my addr 172.21.15.203:0/3852750647 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:32.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.311+0000 7fd95e44a700 1 --2- 172.21.15.203:0/3852750647 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd958064760 0x7fd9581476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:32.317 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.311+0000 7fd95dc49700 1 --2- 172.21.15.203:0/3852750647 >> v2:172.21.15.91:3300/0 conn(0x7fd958064380 0x7fd9580c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:32.317 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95d448700 1 -- 172.21.15.203:0/3852750647 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd958064760 msgr2=0x7fd9581476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:32.317 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95d448700 1 --2- 172.21.15.203:0/3852750647 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd958064760 0x7fd9581476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.317 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95d448700 1 -- 172.21.15.203:0/3852750647 >> v2:172.21.15.91:3300/0 conn(0x7fd958064380 msgr2=0x7fd9580c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:32.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95d448700 1 --2- 172.21.15.203:0/3852750647 >> v2:172.21.15.91:3300/0 conn(0x7fd958064380 0x7fd9580c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95d448700 1 -- 172.21.15.203:0/3852750647 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95814a800 con 0x7fd95814e310 2021-01-06T22:04:32.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95e44a700 1 --2- 172.21.15.203:0/3852750647 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd958064760 0x7fd9581476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:04:32.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95d448700 1 --2- 172.21.15.203:0/3852750647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd95814e310 0x7fd9581471b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7fd958061c90 tx=0x7fd948006f30).ready entity=mon.1 client_cookie=fad45e8f2dca9768 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:32.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95cc47700 1 -- 172.21.15.203:0/3852750647 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94800ca90 con 0x7fd95814e310 2021-01-06T22:04:32.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95cc47700 1 -- 172.21.15.203:0/3852750647 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94800cbf0 con 0x7fd95814e310 2021-01-06T22:04:32.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95cc47700 1 -- 172.21.15.203:0/3852750647 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94800f470 con 0x7fd95814e310 2021-01-06T22:04:32.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95ec4b700 1 -- 172.21.15.203:0/3852750647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd95814e310 msgr2=0x7fd9581471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:32.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95ec4b700 1 --2- 172.21.15.203:0/3852750647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd95814e310 0x7fd9581471b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7fd958061c90 tx=0x7fd948006f30).stop 2021-01-06T22:04:32.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95ec4b700 1 -- 172.21.15.203:0/3852750647 shutdown_connections 2021-01-06T22:04:32.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95ec4b700 1 --2- 172.21.15.203:0/3852750647 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd958064760 0x7fd9581476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95ec4b700 1 --2- 172.21.15.203:0/3852750647 >> v2:172.21.15.91:3300/0 conn(0x7fd958064380 0x7fd9580c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95ec4b700 1 --2- 172.21.15.203:0/3852750647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd95814e310 0x7fd9581471b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95ec4b700 1 -- 172.21.15.203:0/3852750647 >> 172.21.15.203:0/3852750647 conn(0x7fd9580667b0 msgr2=0x7fd9580c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:32.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95ec4b700 1 -- 172.21.15.203:0/3852750647 shutdown_connections 2021-01-06T22:04:32.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.315+0000 7fd95ec4b700 1 -- 172.21.15.203:0/3852750647 wait complete. 2021-01-06T22:04:32.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95ec4b700 1 Processor -- start 2021-01-06T22:04:32.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95ec4b700 1 -- start start 2021-01-06T22:04:32.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95ec4b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd958064380 0x7fd95815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:32.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95ec4b700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd958064760 0x7fd9581496d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:32.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95ec4b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd95814e310 0x7fd9580b99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:32.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95ec4b700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9580b5820 con 0x7fd958064760 2021-01-06T22:04:32.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95ec4b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9580b5f90 con 0x7fd95814e310 2021-01-06T22:04:32.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95ec4b700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95815d4c0 con 0x7fd958064380 2021-01-06T22:04:32.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95e44a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd95814e310 0x7fd9580b99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:32.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95e44a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd95814e310 0x7fd9580b99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49740/0 (socket says 172.21.15.203:49740) 2021-01-06T22:04:32.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95e44a700 1 -- 172.21.15.203:0/1924776409 learned_addr learned my addr 172.21.15.203:0/1924776409 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:32.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95dc49700 1 --2- 172.21.15.203:0/1924776409 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd958064380 0x7fd95815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:32.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95d448700 1 --2- 172.21.15.203:0/1924776409 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd958064760 0x7fd9581496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:32.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95e44a700 1 -- 172.21.15.203:0/1924776409 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd958064380 msgr2=0x7fd95815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:32.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95e44a700 1 --2- 172.21.15.203:0/1924776409 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd958064380 0x7fd95815cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95e44a700 1 -- 172.21.15.203:0/1924776409 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd958064760 msgr2=0x7fd9581496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:32.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95e44a700 1 --2- 172.21.15.203:0/1924776409 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd958064760 0x7fd9581496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95e44a700 1 -- 172.21.15.203:0/1924776409 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9580b5a50 con 0x7fd95814e310 2021-01-06T22:04:32.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95e44a700 1 --2- 172.21.15.203:0/1924776409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd95814e310 0x7fd9580b99c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fd95000ebf0 tx=0x7fd950008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:32.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd945ffb700 1 -- 172.21.15.203:0/1924776409 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95000c770 con 0x7fd95814e310 2021-01-06T22:04:32.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd945ffb700 1 -- 172.21.15.203:0/1924776409 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95000c8d0 con 0x7fd95814e310 2021-01-06T22:04:32.330 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95ec4b700 1 -- 172.21.15.203:0/1924776409 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9580b5cf0 con 0x7fd95814e310 2021-01-06T22:04:32.330 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd945ffb700 1 -- 172.21.15.203:0/1924776409 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95000cc70 con 0x7fd95814e310 2021-01-06T22:04:32.330 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.319+0000 7fd95ec4b700 1 -- 172.21.15.203:0/1924776409 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9580b8320 con 0x7fd95814e310 2021-01-06T22:04:32.331 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.323+0000 7fd945ffb700 1 -- 172.21.15.203:0/1924776409 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fd95001d460 con 0x7fd95814e310 2021-01-06T22:04:32.331 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.323+0000 7fd95ec4b700 1 -- 172.21.15.203:0/1924776409 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9580b84e0 con 0x7fd95814e310 2021-01-06T22:04:32.331 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.323+0000 7fd945ffb700 1 --2- 172.21.15.203:0/1924776409 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9380563d0 0x7fd938058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:32.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.323+0000 7fd945ffb700 1 -- 172.21.15.203:0/1924776409 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd95006f580 con 0x7fd95814e310 2021-01-06T22:04:32.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.323+0000 7fd95dc49700 1 --2- 172.21.15.203:0/1924776409 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9380563d0 0x7fd938058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:32.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.323+0000 7fd95dc49700 1 --2- 172.21.15.203:0/1924776409 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9380563d0 0x7fd938058890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fd95400cac0 tx=0x7fd954006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:32.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.327+0000 7fd945ffb700 1 -- 172.21.15.203:0/1924776409 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fd950044d50 con 0x7fd95814e310 2021-01-06T22:04:32.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.531+0000 7fd95ec4b700 1 -- 172.21.15.203:0/1924776409 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9580450a0 con 0x7fd95814e310 2021-01-06T22:04:32.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.531+0000 7fd945ffb700 1 -- 172.21.15.203:0/1924776409 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+3222 (secure 0 0 0) 0x7fd950048370 con 0x7fd95814e310 2021-01-06T22:04:32.533 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:32.533 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:04:32.460169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":2,"incarnation":13,"state":"up:resolve","state_seq":15,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:04:32.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.535+0000 7fd95ec4b700 1 -- 172.21.15.203:0/1924776409 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9380563d0 msgr2=0x7fd938058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:32.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.535+0000 7fd95ec4b700 1 --2- 172.21.15.203:0/1924776409 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9380563d0 0x7fd938058890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fd95400cac0 tx=0x7fd954006de0).stop 2021-01-06T22:04:32.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.535+0000 7fd95ec4b700 1 -- 172.21.15.203:0/1924776409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd95814e310 msgr2=0x7fd9580b99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:32.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.535+0000 7fd95ec4b700 1 --2- 172.21.15.203:0/1924776409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd95814e310 0x7fd9580b99c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fd95000ebf0 tx=0x7fd950008040).stop 2021-01-06T22:04:32.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.535+0000 7fd95ec4b700 1 -- 172.21.15.203:0/1924776409 shutdown_connections 2021-01-06T22:04:32.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.535+0000 7fd95ec4b700 1 --2- 172.21.15.203:0/1924776409 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9380563d0 0x7fd938058890 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.535+0000 7fd95ec4b700 1 --2- 172.21.15.203:0/1924776409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd95814e310 0x7fd9580b99c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.535+0000 7fd95ec4b700 1 --2- 172.21.15.203:0/1924776409 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd958064380 0x7fd95815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.535+0000 7fd95ec4b700 1 --2- 172.21.15.203:0/1924776409 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd958064760 0x7fd9581496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.535+0000 7fd95ec4b700 1 -- 172.21.15.203:0/1924776409 >> 172.21.15.203:0/1924776409 conn(0x7fd9580667b0 msgr2=0x7fd9580bf420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:32.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.535+0000 7fd95ec4b700 1 -- 172.21.15.203:0/1924776409 shutdown_connections 2021-01-06T22:04:32.539 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.535+0000 7fd95ec4b700 1 -- 172.21.15.203:0/1924776409 wait complete. 2021-01-06T22:04:32.539 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 14 2021-01-06T22:04:32.551 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2021-01-06T22:04:32.551 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2021-01-06T22:04:32.551 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:04:32.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.675+0000 7ff75affe700 1 Processor -- start 2021-01-06T22:04:32.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.675+0000 7ff75affe700 1 -- start start 2021-01-06T22:04:32.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff75affe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff7540c0fa0 0x7ff7540bccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:32.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff75affe700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff75414ad90 0x7ff75414b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:32.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff75affe700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7540bacb0 con 0x7ff7541513f0 2021-01-06T22:04:32.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff75affe700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75414b680 con 0x7ff75414ad90 2021-01-06T22:04:32.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff75affe700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff754152000 con 0x7ff7540c0fa0 2021-01-06T22:04:32.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff7597fb700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff7540c0fa0 0x7ff7540bccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:32.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff7597fb700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff7540c0fa0 0x7ff7540bccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49746/0 (socket says 172.21.15.203:49746) 2021-01-06T22:04:32.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff7597fb700 1 -- 172.21.15.203:0/102584027 learned_addr learned my addr 172.21.15.203:0/102584027 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:32.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff75a7fd700 1 --2- 172.21.15.203:0/102584027 >> v2:172.21.15.91:3300/0 conn(0x7ff75414ad90 0x7ff75414b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:32.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff7597fb700 1 -- 172.21.15.203:0/102584027 >> v2:172.21.15.91:3300/0 conn(0x7ff75414ad90 msgr2=0x7ff75414b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:32.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff7597fb700 1 --2- 172.21.15.203:0/102584027 >> v2:172.21.15.91:3300/0 conn(0x7ff75414ad90 0x7ff75414b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff7597fb700 1 -- 172.21.15.203:0/102584027 >> v1:172.21.15.91:6789/0 conn(0x7ff7541513f0 legacy=0x7ff7541517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:32.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff7597fb700 1 -- 172.21.15.203:0/102584027 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff754067e50 con 0x7ff7540c0fa0 2021-01-06T22:04:32.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff75a7fd700 1 --2- 172.21.15.203:0/102584027 >> v2:172.21.15.91:3300/0 conn(0x7ff75414ad90 0x7ff75414b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:04:32.684 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff7597fb700 1 --2- 172.21.15.203:0/102584027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7540c0fa0 0x7ff7540bccb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7ff7540be730 tx=0x7ff744006ea0).ready entity=mon.1 client_cookie=b9a57fa55a2eb77 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:32.684 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff758ffa700 1 -- 172.21.15.203:0/102584027 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7440098d0 con 0x7ff7540c0fa0 2021-01-06T22:04:32.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff758ffa700 1 -- 172.21.15.203:0/102584027 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff744009a30 con 0x7ff7540c0fa0 2021-01-06T22:04:32.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff758ffa700 1 -- 172.21.15.203:0/102584027 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744009db0 con 0x7ff7540c0fa0 2021-01-06T22:04:32.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff75affe700 1 -- 172.21.15.203:0/102584027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7540c0fa0 msgr2=0x7ff7540bccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:32.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.679+0000 7ff75affe700 1 --2- 172.21.15.203:0/102584027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7540c0fa0 0x7ff7540bccb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7ff7540be730 tx=0x7ff744006ea0).stop 2021-01-06T22:04:32.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75affe700 1 -- 172.21.15.203:0/102584027 shutdown_connections 2021-01-06T22:04:32.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75affe700 1 --2- 172.21.15.203:0/102584027 >> v2:172.21.15.91:3300/0 conn(0x7ff75414ad90 0x7ff75414b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.690 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75affe700 1 --2- 172.21.15.203:0/102584027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7540c0fa0 0x7ff7540bccb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.691 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75affe700 1 -- 172.21.15.203:0/102584027 >> 172.21.15.203:0/102584027 conn(0x7ff7540667b0 msgr2=0x7ff7540c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:32.691 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75affe700 1 -- 172.21.15.203:0/102584027 shutdown_connections 2021-01-06T22:04:32.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75affe700 1 -- 172.21.15.203:0/102584027 wait complete. 2021-01-06T22:04:32.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75affe700 1 Processor -- start 2021-01-06T22:04:32.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75affe700 1 -- start start 2021-01-06T22:04:32.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75affe700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7540c0fa0 0x7ff7540b7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:32.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75affe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff75414ad90 0x7ff7540b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:32.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75affe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7541513f0 0x7ff754150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:32.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75affe700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75415e660 con 0x7ff7540c0fa0 2021-01-06T22:04:32.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75affe700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff754151080 con 0x7ff7541513f0 2021-01-06T22:04:32.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75affe700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff754150de0 con 0x7ff75414ad90 2021-01-06T22:04:32.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75a7fd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7541513f0 0x7ff754150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:32.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75a7fd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7541513f0 0x7ff754150470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49754/0 (socket says 172.21.15.203:49754) 2021-01-06T22:04:32.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75a7fd700 1 -- 172.21.15.203:0/2710528838 learned_addr learned my addr 172.21.15.203:0/2710528838 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:32.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff7597fb700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff75414ad90 0x7ff7540b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:32.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff759ffc700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7540c0fa0 0x7ff7540b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:32.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75a7fd700 1 -- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff75414ad90 msgr2=0x7ff7540b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:32.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75a7fd700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff75414ad90 0x7ff7540b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75a7fd700 1 -- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7540c0fa0 msgr2=0x7ff7540b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:32.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75a7fd700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7540c0fa0 0x7ff7540b7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75a7fd700 1 -- 172.21.15.203:0/2710528838 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75415e8c0 con 0x7ff7541513f0 2021-01-06T22:04:32.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff7597fb700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff75414ad90 0x7ff7540b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:04:32.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff759ffc700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7540c0fa0 0x7ff7540b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:04:32.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.683+0000 7ff75a7fd700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7541513f0 0x7ff754150470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7ff74c00ebf0 tx=0x7ff74c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:32.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff741ffb700 1 -- 172.21.15.203:0/2710528838 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c00c8e0 con 0x7ff7541513f0 2021-01-06T22:04:32.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff741ffb700 1 -- 172.21.15.203:0/2710528838 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74c00ca40 con 0x7ff7541513f0 2021-01-06T22:04:32.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff75affe700 1 -- 172.21.15.203:0/2710528838 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7540b1aa0 con 0x7ff7541513f0 2021-01-06T22:04:32.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff741ffb700 1 -- 172.21.15.203:0/2710528838 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c00cde0 con 0x7ff7541513f0 2021-01-06T22:04:32.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff75affe700 1 -- 172.21.15.203:0/2710528838 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7540b21e0 con 0x7ff7541513f0 2021-01-06T22:04:32.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff75a7fd700 1 -- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7541513f0 msgr2=0x7ff754150470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2021-01-06T22:04:32.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff75a7fd700 1 -- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7541513f0 msgr2=0x7ff754150470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2021-01-06T22:04:32.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff75a7fd700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7541513f0 0x7ff754150470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7ff74c00ebf0 tx=0x7ff74c008040).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:04:32.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff75a7fd700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7541513f0 0x7ff754150470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7ff74c00ebf0 tx=0x7ff74c008040).stop 2021-01-06T22:04:32.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff741ffb700 1 -- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7541513f0 msgr2=0x7ff754150470 unknown :-1 s=STATE_CLOSED l=1).mark_down 2021-01-06T22:04:32.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff741ffb700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7541513f0 0x7ff754150470 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff741ffb700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7340034e0 0x7ff7340058d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:32.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff741ffb700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff75414ad90 0x7ff734007f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:32.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff741ffb700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7540c0fa0 0x7ff73400a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:32.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff741ffb700 1 -- 172.21.15.203:0/2710528838 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73400a930 con 0x7ff7540c0fa0 2021-01-06T22:04:32.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff741ffb700 1 -- 172.21.15.203:0/2710528838 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73400aa70 con 0x7ff7340034e0 2021-01-06T22:04:32.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff741ffb700 1 -- 172.21.15.203:0/2710528838 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73400abb0 con 0x7ff75414ad90 2021-01-06T22:04:32.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff7597fb700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff75414ad90 0x7ff734007f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:32.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff759ffc700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7340034e0 0x7ff7340058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:32.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff75a7fd700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7540c0fa0 0x7ff73400a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:32.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff759ffc700 1 -- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff75414ad90 msgr2=0x7ff734007f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:32.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff759ffc700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff75414ad90 0x7ff734007f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff759ffc700 1 -- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7540c0fa0 msgr2=0x7ff73400a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:32.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff759ffc700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7540c0fa0 0x7ff73400a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff759ffc700 1 -- 172.21.15.203:0/2710528838 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) v3 -- 0x7ff73400aea0 con 0x7ff7340034e0 2021-01-06T22:04:32.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff759ffc700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7340034e0 0x7ff7340058d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7ff75000d780 tx=0x7ff750009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:32.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff75affe700 1 -- 172.21.15.203:0/2710528838 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7540bb750 con 0x7ff7340034e0 2021-01-06T22:04:32.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff741ffb700 1 -- 172.21.15.203:0/2710528838 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750007020 con 0x7ff7340034e0 2021-01-06T22:04:32.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff741ffb700 1 -- 172.21.15.203:0/2710528838 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7500071e0 con 0x7ff7340034e0 2021-01-06T22:04:32.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.687+0000 7ff741ffb700 1 -- 172.21.15.203:0/2710528838 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff75000f500 con 0x7ff7340034e0 2021-01-06T22:04:32.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.691+0000 7ff741ffb700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff73405df70 0x7ff734060430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:32.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.691+0000 7ff741ffb700 1 -- 172.21.15.203:0/2710528838 <== mon.1 v2:172.21.15.203:3300/0 4 ==== osd_map(24..24 src has 1..24) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff75006d760 con 0x7ff7340034e0 2021-01-06T22:04:32.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.691+0000 7ff7597fb700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff73405df70 0x7ff734060430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:32.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.691+0000 7ff7597fb700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff73405df70 0x7ff734060430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7ff7541519d0 tx=0x7ff744016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:32.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.695+0000 7ff741ffb700 1 -- 172.21.15.203:0/2710528838 <== mon.1 v2:172.21.15.203:3300/0 5 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff750042fb0 con 0x7ff7340034e0 2021-01-06T22:04:32.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.895+0000 7ff75affe700 1 -- 172.21.15.203:0/2710528838 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7540bacb0 con 0x7ff7340034e0 2021-01-06T22:04:32.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.895+0000 7ff741ffb700 1 -- 172.21.15.203:0/2710528838 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+3222 (secure 0 0 0) 0x7ff7540bacb0 con 0x7ff7340034e0 2021-01-06T22:04:32.897 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:32.898 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:04:32.460169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":2,"incarnation":13,"state":"up:resolve","state_seq":15,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:04:32.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.899+0000 7ff75affe700 1 -- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff73405df70 msgr2=0x7ff734060430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:32.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.899+0000 7ff75affe700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff73405df70 0x7ff734060430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7ff7541519d0 tx=0x7ff744016000).stop 2021-01-06T22:04:32.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.899+0000 7ff75affe700 1 -- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7340034e0 msgr2=0x7ff7340058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:32.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.899+0000 7ff75affe700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7340034e0 0x7ff7340058d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7ff75000d780 tx=0x7ff750009040).stop 2021-01-06T22:04:32.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.899+0000 7ff75affe700 1 -- 172.21.15.203:0/2710528838 shutdown_connections 2021-01-06T22:04:32.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.899+0000 7ff75affe700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff73405df70 0x7ff734060430 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.899+0000 7ff75affe700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7540c0fa0 0x7ff73400a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.899+0000 7ff75affe700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff75414ad90 0x7ff734007f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.899+0000 7ff75affe700 1 --2- 172.21.15.203:0/2710528838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7340034e0 0x7ff7340058d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:32.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.899+0000 7ff75affe700 1 -- 172.21.15.203:0/2710528838 >> 172.21.15.203:0/2710528838 conn(0x7ff7540667b0 msgr2=0x7ff75414a210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:32.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.899+0000 7ff75affe700 1 -- 172.21.15.203:0/2710528838 shutdown_connections 2021-01-06T22:04:32.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:32.899+0000 7ff75affe700 1 -- 172.21.15.203:0/2710528838 wait complete. 2021-01-06T22:04:32.904 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 14 2021-01-06T22:04:32.920 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 14, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 14, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4254": { "addr": "172.21.15.91:6836/3250440616", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6834", "nonce": 3250440616, "type": "v2" }, { "addr": "172.21.15.91:6836", "nonce": 3250440616, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4254, "incarnation": 13, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:resolve", "state_seq": 15 }, "gid_4273": { "addr": "172.21.15.203:6837/3277561222", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6836", "nonce": 3277561222, "type": "v2" }, { "addr": "172.21.15.203:6837", "nonce": 3277561222, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4273, "incarnation": 9, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 3 }, "gid_4274": { "addr": "172.21.15.91:6837/3888667247", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6835", "nonce": 3888667247, "type": "v2" }, { "addr": "172.21.15.91:6837", "nonce": 3888667247, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4274, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 2 } }, "last_failure": 0, "last_failure_osd_epoch": 24, "max_file_size": 1099511627776, "max_mds": 3, "metadata_pool": 3, "modified": "2021-01-06T22:04:32.460169+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4274, "mds_1": 4273, "mds_2": 4254 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2021-01-06T22:04:34.922 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:04:35.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.691+0000 7f811481a700 1 Processor -- start 2021-01-06T22:04:35.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.691+0000 7f811481a700 1 -- start start 2021-01-06T22:04:35.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f811481a700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81100be350 0x7f81100657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:35.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f811481a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f811014ba00 0x7f811014be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:35.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f811481a700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811014c370 con 0x7f81100be730 2021-01-06T22:04:35.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f811481a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811014ee80 con 0x7f811014ba00 2021-01-06T22:04:35.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f811481a700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811014f120 con 0x7f81100be350 2021-01-06T22:04:35.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f810ffff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f811014ba00 0x7f811014be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:35.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f810ffff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f811014ba00 0x7f811014be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49766/0 (socket says 172.21.15.203:49766) 2021-01-06T22:04:35.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f810ffff700 1 -- 172.21.15.203:0/4245454243 learned_addr learned my addr 172.21.15.203:0/4245454243 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:35.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f810f7fe700 1 --2- 172.21.15.203:0/4245454243 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81100be350 0x7f81100657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:35.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f810ffff700 1 -- 172.21.15.203:0/4245454243 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81100be350 msgr2=0x7f81100657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:35.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f810ffff700 1 --2- 172.21.15.203:0/4245454243 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81100be350 0x7f81100657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:35.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f810ffff700 1 -- 172.21.15.203:0/4245454243 >> v1:172.21.15.91:6789/0 conn(0x7f81100be730 legacy=0x7f81100ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:35.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f810ffff700 1 -- 172.21.15.203:0/4245454243 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811014f3c0 con 0x7f811014ba00 2021-01-06T22:04:35.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f810ffff700 1 --2- 172.21.15.203:0/4245454243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f811014ba00 0x7f811014be80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f8108009d50 tx=0x7f8108009290).ready entity=mon.1 client_cookie=27fbd1436ae0b453 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:35.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f810e7fc700 1 -- 172.21.15.203:0/4245454243 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810800bbc0 con 0x7f811014ba00 2021-01-06T22:04:35.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f810e7fc700 1 -- 172.21.15.203:0/4245454243 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810800bd20 con 0x7f811014ba00 2021-01-06T22:04:35.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.695+0000 7f810e7fc700 1 -- 172.21.15.203:0/4245454243 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108011610 con 0x7f811014ba00 2021-01-06T22:04:35.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.699+0000 7f811481a700 1 -- 172.21.15.203:0/4245454243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f811014ba00 msgr2=0x7f811014be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:35.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.699+0000 7f811481a700 1 --2- 172.21.15.203:0/4245454243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f811014ba00 0x7f811014be80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f8108009d50 tx=0x7f8108009290).stop 2021-01-06T22:04:35.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.699+0000 7f811481a700 1 -- 172.21.15.203:0/4245454243 shutdown_connections 2021-01-06T22:04:35.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.699+0000 7f811481a700 1 --2- 172.21.15.203:0/4245454243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f811014ba00 0x7f811014be80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:35.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.699+0000 7f811481a700 1 --2- 172.21.15.203:0/4245454243 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81100be350 0x7f81100657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:35.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.699+0000 7f811481a700 1 -- 172.21.15.203:0/4245454243 >> 172.21.15.203:0/4245454243 conn(0x7f8110067e20 msgr2=0x7f8110068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:35.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.699+0000 7f811481a700 1 -- 172.21.15.203:0/4245454243 shutdown_connections 2021-01-06T22:04:35.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.699+0000 7f811481a700 1 -- 172.21.15.203:0/4245454243 wait complete. 2021-01-06T22:04:35.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.699+0000 7f811481a700 1 Processor -- start 2021-01-06T22:04:35.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.699+0000 7f811481a700 1 -- start start 2021-01-06T22:04:35.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.699+0000 7f811481a700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81100be350 0x7f811015cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:35.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.699+0000 7f811481a700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81100be730 0x7f81101416b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:35.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.699+0000 7f811481a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f811014ba00 0x7f81101478f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:35.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.699+0000 7f811481a700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81101605e0 con 0x7f81100be350 2021-01-06T22:04:35.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.699+0000 7f811481a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81100b8220 con 0x7f811014ba00 2021-01-06T22:04:35.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.699+0000 7f811481a700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8110160810 con 0x7f81100be730 2021-01-06T22:04:35.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f810ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f811014ba00 0x7f81101478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:35.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f810ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f811014ba00 0x7f81101478f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49770/0 (socket says 172.21.15.203:49770) 2021-01-06T22:04:35.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f810ffff700 1 -- 172.21.15.203:0/4036130928 learned_addr learned my addr 172.21.15.203:0/4036130928 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:35.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f810f7fe700 1 --2- 172.21.15.203:0/4036130928 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81100be350 0x7f811015cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:35.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f810effd700 1 --2- 172.21.15.203:0/4036130928 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81100be730 0x7f81101416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:35.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f810ffff700 1 -- 172.21.15.203:0/4036130928 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81100be730 msgr2=0x7f81101416b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:35.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f810ffff700 1 --2- 172.21.15.203:0/4036130928 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81100be730 0x7f81101416b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:35.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f810ffff700 1 -- 172.21.15.203:0/4036130928 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81100be350 msgr2=0x7f811015cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:35.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f810ffff700 1 --2- 172.21.15.203:0/4036130928 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81100be350 0x7f811015cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:35.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f810ffff700 1 -- 172.21.15.203:0/4036130928 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8110160ab0 con 0x7f811014ba00 2021-01-06T22:04:35.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f810effd700 1 --2- 172.21.15.203:0/4036130928 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81100be730 0x7f81101416b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:04:35.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f810ffff700 1 --2- 172.21.15.203:0/4036130928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f811014ba00 0x7f81101478f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f81080092d0 tx=0x7f8108017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:35.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f80f77fe700 1 -- 172.21.15.203:0/4036130928 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108011e70 con 0x7f811014ba00 2021-01-06T22:04:35.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f80f77fe700 1 -- 172.21.15.203:0/4036130928 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810800be90 con 0x7f811014ba00 2021-01-06T22:04:35.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f811481a700 1 -- 172.21.15.203:0/4036130928 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8110160d50 con 0x7f811014ba00 2021-01-06T22:04:35.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f80f77fe700 1 -- 172.21.15.203:0/4036130928 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108010d60 con 0x7f811014ba00 2021-01-06T22:04:35.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f811481a700 1 -- 172.21.15.203:0/4036130928 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811014e180 con 0x7f811014ba00 2021-01-06T22:04:35.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f811481a700 1 -- 172.21.15.203:0/4036130928 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811005d810 con 0x7f811014ba00 2021-01-06T22:04:35.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.703+0000 7f80f77fe700 1 -- 172.21.15.203:0/4036130928 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f810800f070 con 0x7f811014ba00 2021-01-06T22:04:35.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.707+0000 7f80f77fe700 1 --2- 172.21.15.203:0/4036130928 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f80f00563d0 0x7f80f0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:35.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.707+0000 7f80f77fe700 1 -- 172.21.15.203:0/4036130928 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f81080713d0 con 0x7f811014ba00 2021-01-06T22:04:35.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.707+0000 7f810f7fe700 1 --2- 172.21.15.203:0/4036130928 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f80f00563d0 0x7f80f0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:35.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.707+0000 7f810f7fe700 1 --2- 172.21.15.203:0/4036130928 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f80f00563d0 0x7f80f0058890 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f810000bb50 tx=0x7f8100006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:35.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.711+0000 7f80f77fe700 1 -- 172.21.15.203:0/4036130928 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f8108046c00 con 0x7f811014ba00 2021-01-06T22:04:35.912 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.911+0000 7f811481a700 1 -- 172.21.15.203:0/4036130928 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8110160ee0 con 0x7f811014ba00 2021-01-06T22:04:35.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.911+0000 7f80f77fe700 1 -- 172.21.15.203:0/4036130928 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+3221 (secure 0 0 0) 0x7f810804a220 con 0x7f811014ba00 2021-01-06T22:04:35.913 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:35.914 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:04:35.487774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":2,"incarnation":13,"state":"up:active","state_seq":18,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:04:35.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.915+0000 7f811481a700 1 -- 172.21.15.203:0/4036130928 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f80f00563d0 msgr2=0x7f80f0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:35.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.915+0000 7f811481a700 1 --2- 172.21.15.203:0/4036130928 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f80f00563d0 0x7f80f0058890 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f810000bb50 tx=0x7f8100006de0).stop 2021-01-06T22:04:35.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.915+0000 7f811481a700 1 -- 172.21.15.203:0/4036130928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f811014ba00 msgr2=0x7f81101478f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:35.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.915+0000 7f811481a700 1 --2- 172.21.15.203:0/4036130928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f811014ba00 0x7f81101478f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f81080092d0 tx=0x7f8108017000).stop 2021-01-06T22:04:35.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.915+0000 7f811481a700 1 -- 172.21.15.203:0/4036130928 shutdown_connections 2021-01-06T22:04:35.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.915+0000 7f811481a700 1 --2- 172.21.15.203:0/4036130928 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f80f00563d0 0x7f80f0058890 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:35.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.915+0000 7f811481a700 1 --2- 172.21.15.203:0/4036130928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f811014ba00 0x7f81101478f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:35.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.915+0000 7f811481a700 1 --2- 172.21.15.203:0/4036130928 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81100be350 0x7f811015cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:35.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.915+0000 7f811481a700 1 --2- 172.21.15.203:0/4036130928 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81100be730 0x7f81101416b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:35.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.915+0000 7f811481a700 1 -- 172.21.15.203:0/4036130928 >> 172.21.15.203:0/4036130928 conn(0x7f8110067e20 msgr2=0x7f81100ba610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:35.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.915+0000 7f811481a700 1 -- 172.21.15.203:0/4036130928 shutdown_connections 2021-01-06T22:04:35.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:35.915+0000 7f811481a700 1 -- 172.21.15.203:0/4036130928 wait complete. 2021-01-06T22:04:35.918 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 17 2021-01-06T22:04:35.928 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=2, replacing gid=4270 2021-01-06T22:04:35.928 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 4 secs before reviving mds.c 2021-01-06T22:04:39.928 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2021-01-06T22:04:39.929 INFO:tasks.ceph.mds.c:Restarting daemon 2021-01-06T22:04:39.929 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-01-06T22:04:39.931 INFO:tasks.ceph.mds.c:Started 2021-01-06T22:04:39.932 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:04:40.002 INFO:tasks.ceph.mds.c.smithi203.stdout:starting mds.c at 2021-01-06T22:04:40.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.059+0000 7f7e9e322700 1 Processor -- start 2021-01-06T22:04:40.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.059+0000 7f7e9e322700 1 -- start start 2021-01-06T22:04:40.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9e322700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7e981513f0 0x7f7e980c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:40.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9e322700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7e9814ad90 0x7f7e9814b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:40.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9e322700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e980bacb0 con 0x7f7e980bccb0 2021-01-06T22:04:40.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9e322700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e980bd160 con 0x7f7e9814ad90 2021-01-06T22:04:40.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9e322700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9814b680 con 0x7f7e981513f0 2021-01-06T22:04:40.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9d320700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7e981513f0 0x7f7e980c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:40.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9d320700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7e981513f0 0x7f7e980c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49788/0 (socket says 172.21.15.203:49788) 2021-01-06T22:04:40.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9d320700 1 -- 172.21.15.203:0/114336966 learned_addr learned my addr 172.21.15.203:0/114336966 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:40.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9db21700 1 --2- 172.21.15.203:0/114336966 >> v2:172.21.15.91:3300/0 conn(0x7f7e9814ad90 0x7f7e9814b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:40.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9d320700 1 -- 172.21.15.203:0/114336966 >> v2:172.21.15.91:3300/0 conn(0x7f7e9814ad90 msgr2=0x7f7e9814b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:04:40.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9d320700 1 --2- 172.21.15.203:0/114336966 >> v2:172.21.15.91:3300/0 conn(0x7f7e9814ad90 0x7f7e9814b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:40.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9d320700 1 -- 172.21.15.203:0/114336966 >> v1:172.21.15.91:6789/0 conn(0x7f7e980bccb0 legacy=0x7f7e981517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:40.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9d320700 1 -- 172.21.15.203:0/114336966 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98152000 con 0x7f7e981513f0 2021-01-06T22:04:40.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9d320700 1 --2- 172.21.15.203:0/114336966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e981513f0 0x7f7e980c0fa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f7e98061ee0 tx=0x7f7e94006ea0).ready entity=mon.1 client_cookie=1942ccd5fc21136c server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:40.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e87fff700 1 -- 172.21.15.203:0/114336966 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e940098d0 con 0x7f7e981513f0 2021-01-06T22:04:40.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e87fff700 1 -- 172.21.15.203:0/114336966 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e94009a30 con 0x7f7e981513f0 2021-01-06T22:04:40.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e87fff700 1 -- 172.21.15.203:0/114336966 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94009db0 con 0x7f7e981513f0 2021-01-06T22:04:40.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9e322700 1 -- 172.21.15.203:0/114336966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e981513f0 msgr2=0x7f7e980c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:40.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9e322700 1 --2- 172.21.15.203:0/114336966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e981513f0 0x7f7e980c0fa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f7e98061ee0 tx=0x7f7e94006ea0).stop 2021-01-06T22:04:40.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9e322700 1 -- 172.21.15.203:0/114336966 shutdown_connections 2021-01-06T22:04:40.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9e322700 1 --2- 172.21.15.203:0/114336966 >> v2:172.21.15.91:3300/0 conn(0x7f7e9814ad90 0x7f7e9814b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:40.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9e322700 1 --2- 172.21.15.203:0/114336966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e981513f0 0x7f7e980c0fa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:40.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9e322700 1 -- 172.21.15.203:0/114336966 >> 172.21.15.203:0/114336966 conn(0x7f7e980667b0 msgr2=0x7f7e980c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:40.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9e322700 1 -- 172.21.15.203:0/114336966 shutdown_connections 2021-01-06T22:04:40.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9e322700 1 -- 172.21.15.203:0/114336966 wait complete. 2021-01-06T22:04:40.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9e322700 1 Processor -- start 2021-01-06T22:04:40.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.063+0000 7f7e9e322700 1 -- start start 2021-01-06T22:04:40.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9e322700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e980bccb0 0x7f7e98149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:40.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9e322700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e9814ad90 0x7f7e98060320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:40.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9e322700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e981513f0 0x7f7e98060810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:40.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9e322700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e98060d00 con 0x7f7e980bccb0 2021-01-06T22:04:40.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9e322700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9814dcf0 con 0x7f7e981513f0 2021-01-06T22:04:40.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9e322700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9814df50 con 0x7f7e9814ad90 2021-01-06T22:04:40.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9db21700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e981513f0 0x7f7e98060810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:40.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9db21700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e981513f0 0x7f7e98060810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49798/0 (socket says 172.21.15.203:49798) 2021-01-06T22:04:40.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9db21700 1 -- 172.21.15.203:0/596199957 learned_addr learned my addr 172.21.15.203:0/596199957 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:04:40.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9d320700 1 --2- 172.21.15.203:0/596199957 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e980bccb0 0x7f7e98149e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:40.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9cb1f700 1 --2- 172.21.15.203:0/596199957 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e9814ad90 0x7f7e98060320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:40.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9db21700 1 -- 172.21.15.203:0/596199957 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e9814ad90 msgr2=0x7f7e98060320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:40.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9db21700 1 --2- 172.21.15.203:0/596199957 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e9814ad90 0x7f7e98060320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:40.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9db21700 1 -- 172.21.15.203:0/596199957 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e980bccb0 msgr2=0x7f7e98149e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:40.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9db21700 1 --2- 172.21.15.203:0/596199957 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e980bccb0 0x7f7e98149e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:40.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9db21700 1 -- 172.21.15.203:0/596199957 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9814e1f0 con 0x7f7e981513f0 2021-01-06T22:04:40.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9cb1f700 1 --2- 172.21.15.203:0/596199957 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e9814ad90 0x7f7e98060320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:04:40.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9db21700 1 --2- 172.21.15.203:0/596199957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e981513f0 0x7f7e98060810 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f7e9000ebf0 tx=0x7f7e90008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:40.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e857fa700 1 -- 172.21.15.203:0/596199957 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9000c880 con 0x7f7e981513f0 2021-01-06T22:04:40.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e857fa700 1 -- 172.21.15.203:0/596199957 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9000c9e0 con 0x7f7e981513f0 2021-01-06T22:04:40.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9e322700 1 -- 172.21.15.203:0/596199957 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9814c920 con 0x7f7e981513f0 2021-01-06T22:04:40.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e857fa700 1 -- 172.21.15.203:0/596199957 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9000cd80 con 0x7f7e981513f0 2021-01-06T22:04:40.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9e322700 1 -- 172.21.15.203:0/596199957 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9814ce60 con 0x7f7e981513f0 2021-01-06T22:04:40.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9e322700 1 -- 172.21.15.203:0/596199957 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9814d020 con 0x7f7e981513f0 2021-01-06T22:04:40.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e857fa700 1 -- 172.21.15.203:0/596199957 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f7e90013640 con 0x7f7e981513f0 2021-01-06T22:04:40.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e857fa700 1 --2- 172.21.15.203:0/596199957 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e780563d0 0x7f7e78058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:04:40.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e857fa700 1 -- 172.21.15.203:0/596199957 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7e9006f940 con 0x7f7e981513f0 2021-01-06T22:04:40.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.067+0000 7f7e9d320700 1 --2- 172.21.15.203:0/596199957 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e780563d0 0x7f7e78058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:04:40.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.071+0000 7f7e9d320700 1 --2- 172.21.15.203:0/596199957 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e780563d0 0x7f7e78058890 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f7e9400b190 tx=0x7f7e94016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:04:40.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.075+0000 7f7e857fa700 1 -- 172.21.15.203:0/596199957 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f7e90045110 con 0x7f7e981513f0 2021-01-06T22:04:40.291 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.291+0000 7f7e9e322700 1 -- 172.21.15.203:0/596199957 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e980450a0 con 0x7f7e981513f0 2021-01-06T22:04:40.292 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.291+0000 7f7e857fa700 1 -- 172.21.15.203:0/596199957 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3577 (secure 0 0 0) 0x7f7e90048730 con 0x7f7e981513f0 2021-01-06T22:04:40.293 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:04:40.293 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4867,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:04:35.487774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":2,"incarnation":13,"state":"up:active","state_seq":18,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:04:40.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.295+0000 7f7e9e322700 1 -- 172.21.15.203:0/596199957 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e780563d0 msgr2=0x7f7e78058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:40.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.295+0000 7f7e9e322700 1 --2- 172.21.15.203:0/596199957 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e780563d0 0x7f7e78058890 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f7e9400b190 tx=0x7f7e94016000).stop 2021-01-06T22:04:40.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.295+0000 7f7e9e322700 1 -- 172.21.15.203:0/596199957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e981513f0 msgr2=0x7f7e98060810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:04:40.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.295+0000 7f7e9e322700 1 --2- 172.21.15.203:0/596199957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e981513f0 0x7f7e98060810 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f7e9000ebf0 tx=0x7f7e90008040).stop 2021-01-06T22:04:40.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.295+0000 7f7e9e322700 1 -- 172.21.15.203:0/596199957 shutdown_connections 2021-01-06T22:04:40.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.295+0000 7f7e9e322700 1 --2- 172.21.15.203:0/596199957 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e780563d0 0x7f7e78058890 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:40.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.295+0000 7f7e9e322700 1 --2- 172.21.15.203:0/596199957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e981513f0 0x7f7e98060810 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:40.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.295+0000 7f7e9e322700 1 --2- 172.21.15.203:0/596199957 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e980bccb0 0x7f7e98149e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:40.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.295+0000 7f7e9e322700 1 --2- 172.21.15.203:0/596199957 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e9814ad90 0x7f7e98060320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:04:40.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.295+0000 7f7e9e322700 1 -- 172.21.15.203:0/596199957 >> 172.21.15.203:0/596199957 conn(0x7f7e980667b0 msgr2=0x7f7e98152310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:04:40.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.295+0000 7f7e9e322700 1 -- 172.21.15.203:0/596199957 shutdown_connections 2021-01-06T22:04:40.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:04:40.295+0000 7f7e9e322700 1 -- 172.21.15.203:0/596199957 wait complete. 2021-01-06T22:04:40.297 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 18 2021-01-06T22:04:40.307 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2021-01-06T22:04:40.307 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 57 secs before thrashing 2021-01-06T22:04:41.125 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:04:41.128 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:05:08.975 INFO:tasks.workunit.client.0.smithi203.stderr:Note: checking out 'd0ed162b51928c50f20cee111f8292828eda755e'. 2021-01-06T22:05:08.975 INFO:tasks.workunit.client.0.smithi203.stderr: 2021-01-06T22:05:08.976 INFO:tasks.workunit.client.0.smithi203.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2021-01-06T22:05:08.976 INFO:tasks.workunit.client.0.smithi203.stderr:changes and commit them, and you can discard any commits you make in this 2021-01-06T22:05:08.976 INFO:tasks.workunit.client.0.smithi203.stderr:state without impacting any branches by performing another checkout. 2021-01-06T22:05:08.977 INFO:tasks.workunit.client.0.smithi203.stderr: 2021-01-06T22:05:08.977 INFO:tasks.workunit.client.0.smithi203.stderr:If you want to create a new branch to retain commits you create, you may 2021-01-06T22:05:08.977 INFO:tasks.workunit.client.0.smithi203.stderr:do so (now or later) by using -b with the checkout command again. Example: 2021-01-06T22:05:08.977 INFO:tasks.workunit.client.0.smithi203.stderr: 2021-01-06T22:05:08.977 INFO:tasks.workunit.client.0.smithi203.stderr: git checkout -b 2021-01-06T22:05:08.978 INFO:tasks.workunit.client.0.smithi203.stderr: 2021-01-06T22:05:08.978 INFO:tasks.workunit.client.0.smithi203.stderr:HEAD is now at d0ed162b51 Merge PR #38463 into wip-pdonnell-testing-20210105.221014 2021-01-06T22:05:08.979 DEBUG:teuthology.orchestra.run.smithi203:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2021-01-06T22:05:08.986 INFO:tasks.workunit.client.0.smithi203.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2021-01-06T22:05:08.988 INFO:tasks.workunit.client.0.smithi203.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2021-01-06T22:05:08.988 INFO:tasks.workunit.client.0.smithi203.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2021-01-06T22:05:09.708 INFO:tasks.workunit.client.0.smithi203.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2021-01-06T22:05:09.824 INFO:tasks.workunit.client.0.smithi203.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2021-01-06T22:05:09.879 INFO:tasks.workunit.client.0.smithi203.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2021-01-06T22:05:09.882 INFO:tasks.workunit.client.0.smithi203.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2021-01-06T22:05:09.882 INFO:tasks.workunit.client.0.smithi203.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2021-01-06T22:05:09.934 INFO:tasks.workunit.client.0.smithi203.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2021-01-06T22:05:09.938 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2021-01-06T22:05:09.938 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2021-01-06T22:05:09.988 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2021-01-06T22:05:09.991 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2021-01-06T22:05:09.992 DEBUG:teuthology.orchestra.run.smithi203:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d0ed162b51928c50f20cee111f8292828eda755e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2021-01-06T22:05:10.230 INFO:tasks.workunit.client.0.smithi203.stderr:--2021-01-06 22:05:10-- http://download.ceph.com/qa/ffsb.tar.bz2 2021-01-06T22:05:10.247 INFO:tasks.workunit.client.0.smithi203.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2021-01-06T22:05:10.273 INFO:tasks.workunit.client.0.smithi203.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2021-01-06T22:05:10.294 INFO:tasks.workunit.client.0.smithi203.stderr:HTTP request sent, awaiting response... 200 OK 2021-01-06T22:05:10.294 INFO:tasks.workunit.client.0.smithi203.stderr:Length: 128931 (126K) [application/octet-stream] 2021-01-06T22:05:10.297 INFO:tasks.workunit.client.0.smithi203.stderr:Saving to: 'ffsb.tar.bz2' 2021-01-06T22:05:11.033 INFO:tasks.workunit.client.0.smithi203.stderr: 2021-01-06T22:05:11.035 INFO:tasks.workunit.client.0.smithi203.stderr: 0K .......... .......... .......... .......... .......... 39% 67.8K 1s 2021-01-06T22:05:11.058 INFO:tasks.workunit.client.0.smithi203.stderr: 50K .......... .......... .......... .......... .......... 79% 2.18M 0s 2021-01-06T22:05:11.102 INFO:tasks.workunit.client.0.smithi203.stderr: 100K .......... .......... ..... 100% 590K=0.8s 2021-01-06T22:05:11.103 INFO:tasks.workunit.client.0.smithi203.stderr: 2021-01-06T22:05:11.103 INFO:tasks.workunit.client.0.smithi203.stderr:2021-01-06 22:05:11 (157 KB/s) - 'ffsb.tar.bz2' saved [128931/128931] 2021-01-06T22:05:11.103 INFO:tasks.workunit.client.0.smithi203.stderr: 2021-01-06T22:05:11.134 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/ 2021-01-06T22:05:11.137 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/ffsb_fs.c 2021-01-06T22:05:11.143 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/INSTALL 2021-01-06T22:05:11.148 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/fileops.c 2021-01-06T22:05:11.152 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/missing 2021-01-06T22:05:11.156 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/config.h.in 2021-01-06T22:05:11.160 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/config.guess 2021-01-06T22:05:11.165 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/fh.h 2021-01-06T22:05:11.169 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/COPYING 2021-01-06T22:05:11.173 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/util.h 2021-01-06T22:05:11.177 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/main.c 2021-01-06T22:05:11.180 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/ChangeLog 2021-01-06T22:05:11.183 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/rand.c 2021-01-06T22:05:11.186 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/ffsb_fs.h 2021-01-06T22:05:11.191 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/install-sh 2021-01-06T22:05:11.195 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/rwlock.h 2021-01-06T22:05:11.198 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/parser.h 2021-01-06T22:05:11.202 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/filelist.h 2021-01-06T22:05:11.205 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/LICENSE 2021-01-06T22:05:11.209 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/Makefile.in 2021-01-06T22:05:11.214 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/filelist.c 2021-01-06T22:05:11.218 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/configure 2021-01-06T22:05:11.229 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:05:11.231 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/README 2021-01-06T22:05:11.232 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:05:11.238 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/cirlist.c 2021-01-06T22:05:11.241 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/metaops.c 2021-01-06T22:05:11.244 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/registration-description 2021-01-06T22:05:11.249 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/list.c 2021-01-06T22:05:11.252 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/list.h 2021-01-06T22:05:11.257 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/rand.h 2021-01-06T22:05:11.260 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/parser.c 2021-01-06T22:05:11.264 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/ffsb_tg.c 2021-01-06T22:05:11.268 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/cirlist.h 2021-01-06T22:05:11.273 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/AUTHORS 2021-01-06T22:05:11.277 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/aclocal.m4 2021-01-06T22:05:11.282 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/configure.in 2021-01-06T22:05:11.285 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/examples/ 2021-01-06T22:05:11.288 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/examples/profile_everything 2021-01-06T22:05:11.292 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/depcomp 2021-01-06T22:05:11.297 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/ffsb_stats.c 2021-01-06T22:05:11.301 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/util.c 2021-01-06T22:05:11.304 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/Makefile.am 2021-01-06T22:05:11.308 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/ffsb_thread.c 2021-01-06T22:05:11.312 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/rbt.c 2021-01-06T22:05:11.317 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/rbt.h 2021-01-06T22:05:11.321 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/ffsb_tg.h 2021-01-06T22:05:11.326 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/stamp-h.in 2021-01-06T22:05:11.332 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/config.sub 2021-01-06T22:05:11.337 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/ffsb_thread.h 2021-01-06T22:05:11.341 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/fileops.h 2021-01-06T22:05:11.345 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/ffsb_op.h 2021-01-06T22:05:11.348 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/public-description 2021-01-06T22:05:11.351 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/mkinstalldirs 2021-01-06T22:05:11.354 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/ffsb.h 2021-01-06T22:05:11.358 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/metaops.h 2021-01-06T22:05:11.361 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/ffsb_op.c 2021-01-06T22:05:11.366 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/fh.c 2021-01-06T22:05:11.370 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/NEWS 2021-01-06T22:05:11.373 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/rwlock.c 2021-01-06T22:05:11.377 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/ffsb_stats.h 2021-01-06T22:05:11.381 INFO:tasks.workunit.client.0.smithi203.stdout:ffsb-6.0-rc2/ffsb_fc.c 2021-01-06T22:05:11.407 INFO:tasks.workunit.client.0.smithi203.stdout:patching file parser.c 2021-01-06T22:05:12.754 INFO:tasks.workunit.client.0.smithi203.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2021-01-06T22:05:13.776 INFO:tasks.workunit.client.0.smithi203.stdout:checking whether build environment is sane... yes 2021-01-06T22:05:13.807 INFO:tasks.workunit.client.0.smithi203.stdout:checking for a thread-safe mkdir -p... /bin/mkdir -p 2021-01-06T22:05:13.809 INFO:tasks.workunit.client.0.smithi203.stdout:checking for gawk... gawk 2021-01-06T22:05:13.940 INFO:tasks.workunit.client.0.smithi203.stdout:checking whether make sets $(MAKE)... yes 2021-01-06T22:05:14.051 INFO:tasks.workunit.client.0.smithi203.stdout:checking build system type... x86_64-unknown-linux-gnu 2021-01-06T22:05:14.052 INFO:tasks.workunit.client.0.smithi203.stdout:checking host system type... x86_64-unknown-linux-gnu 2021-01-06T22:05:14.102 INFO:tasks.workunit.client.0.smithi203.stdout:checking for style of include used by make... GNU 2021-01-06T22:05:14.114 INFO:tasks.workunit.client.0.smithi203.stdout:checking for gcc... gcc 2021-01-06T22:05:14.309 INFO:tasks.workunit.client.0.smithi203.stdout:checking for C compiler default output file name... a.out 2021-01-06T22:05:14.317 INFO:tasks.workunit.client.0.smithi203.stdout:checking whether the C compiler works... yes 2021-01-06T22:05:14.325 INFO:tasks.workunit.client.0.smithi203.stdout:checking whether we are cross compiling... no 2021-01-06T22:05:14.433 INFO:tasks.workunit.client.0.smithi203.stdout:checking for suffix of executables... 2021-01-06T22:05:14.526 INFO:tasks.workunit.client.0.smithi203.stdout:checking for suffix of object files... o 2021-01-06T22:05:14.612 INFO:tasks.workunit.client.0.smithi203.stdout:checking whether we are using the GNU C compiler... yes 2021-01-06T22:05:15.215 INFO:tasks.workunit.client.0.smithi203.stdout:checking whether gcc accepts -g... yes 2021-01-06T22:05:15.345 INFO:tasks.workunit.client.0.smithi203.stdout:checking for gcc option to accept ISO C89... none needed 2021-01-06T22:05:16.638 INFO:tasks.workunit.client.0.smithi203.stdout:checking dependency style of gcc... gcc3 2021-01-06T22:05:16.783 INFO:tasks.workunit.client.0.smithi203.stdout:checking how to run the C preprocessor... gcc -E 2021-01-06T22:05:16.908 INFO:tasks.workunit.client.0.smithi203.stdout:checking for grep that handles long lines and -e... /bin/grep 2021-01-06T22:05:16.912 INFO:tasks.workunit.client.0.smithi203.stdout:checking for egrep... /bin/grep -E 2021-01-06T22:05:16.939 INFO:tasks.workunit.client.0.smithi203.stdout:checking for AIX... no 2021-01-06T22:05:16.987 INFO:tasks.workunit.client.0.smithi203.stdout:checking for gcc... (cached) gcc 2021-01-06T22:05:17.018 INFO:tasks.workunit.client.0.smithi203.stdout:checking whether we are using the GNU C compiler... (cached) yes 2021-01-06T22:05:17.020 INFO:tasks.workunit.client.0.smithi203.stdout:checking whether gcc accepts -g... (cached) yes 2021-01-06T22:05:17.022 INFO:tasks.workunit.client.0.smithi203.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2021-01-06T22:05:17.024 INFO:tasks.workunit.client.0.smithi203.stdout:checking dependency style of gcc... (cached) gcc3 2021-01-06T22:05:17.161 INFO:tasks.workunit.client.0.smithi203.stdout:checking for main in -lm... yes 2021-01-06T22:05:17.319 INFO:tasks.workunit.client.0.smithi203.stdout:checking for main in -lpthread... yes 2021-01-06T22:05:18.132 INFO:tasks.workunit.client.0.smithi203.stdout:checking for ANSI C header files... yes 2021-01-06T22:05:18.250 INFO:tasks.workunit.client.0.smithi203.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2021-01-06T22:05:18.427 INFO:tasks.workunit.client.0.smithi203.stdout:checking for sys/types.h... yes 2021-01-06T22:05:18.561 INFO:tasks.workunit.client.0.smithi203.stdout:checking for sys/stat.h... yes 2021-01-06T22:05:18.700 INFO:tasks.workunit.client.0.smithi203.stdout:checking for stdlib.h... yes 2021-01-06T22:05:18.835 INFO:tasks.workunit.client.0.smithi203.stdout:checking for string.h... yes 2021-01-06T22:05:18.975 INFO:tasks.workunit.client.0.smithi203.stdout:checking for memory.h... yes 2021-01-06T22:05:19.110 INFO:tasks.workunit.client.0.smithi203.stdout:checking for strings.h... yes 2021-01-06T22:05:19.260 INFO:tasks.workunit.client.0.smithi203.stdout:checking for inttypes.h... yes 2021-01-06T22:05:19.408 INFO:tasks.workunit.client.0.smithi203.stdout:checking for stdint.h... yes 2021-01-06T22:05:19.556 INFO:tasks.workunit.client.0.smithi203.stdout:checking for unistd.h... yes 2021-01-06T22:05:19.713 INFO:tasks.workunit.client.0.smithi203.stdout:checking pthread.h usability... yes 2021-01-06T22:05:19.780 INFO:tasks.workunit.client.0.smithi203.stdout:checking pthread.h presence... yes 2021-01-06T22:05:19.783 INFO:tasks.workunit.client.0.smithi203.stdout:checking for pthread.h... yes 2021-01-06T22:05:19.932 INFO:tasks.workunit.client.0.smithi203.stdout:checking fcntl.h usability... yes 2021-01-06T22:05:19.997 INFO:tasks.workunit.client.0.smithi203.stdout:checking fcntl.h presence... yes 2021-01-06T22:05:19.998 INFO:tasks.workunit.client.0.smithi203.stdout:checking for fcntl.h... yes 2021-01-06T22:05:20.474 INFO:tasks.workunit.client.0.smithi203.stdout:checking limits.h usability... yes 2021-01-06T22:05:20.532 INFO:tasks.workunit.client.0.smithi203.stdout:checking limits.h presence... yes 2021-01-06T22:05:20.534 INFO:tasks.workunit.client.0.smithi203.stdout:checking for limits.h... yes 2021-01-06T22:05:20.583 INFO:tasks.workunit.client.0.smithi203.stdout:checking for stdint.h... (cached) yes 2021-01-06T22:05:20.719 INFO:tasks.workunit.client.0.smithi203.stdout:checking sys/time.h usability... yes 2021-01-06T22:05:20.778 INFO:tasks.workunit.client.0.smithi203.stdout:checking sys/time.h presence... yes 2021-01-06T22:05:20.779 INFO:tasks.workunit.client.0.smithi203.stdout:checking for sys/time.h... yes 2021-01-06T22:05:21.743 INFO:tasks.workunit.client.0.smithi203.stdout:checking for unistd.h... (cached) yes 2021-01-06T22:05:21.873 INFO:tasks.workunit.client.0.smithi203.stdout:checking sys/vfs.h usability... yes 2021-01-06T22:05:21.925 INFO:tasks.workunit.client.0.smithi203.stdout:checking sys/vfs.h presence... yes 2021-01-06T22:05:21.927 INFO:tasks.workunit.client.0.smithi203.stdout:checking for sys/vfs.h... yes 2021-01-06T22:05:22.085 INFO:tasks.workunit.client.0.smithi203.stdout:checking sys/limits.h usability... no 2021-01-06T22:05:22.155 INFO:tasks.workunit.client.0.smithi203.stdout:checking sys/limits.h presence... no 2021-01-06T22:05:22.158 INFO:tasks.workunit.client.0.smithi203.stdout:checking for sys/limits.h... no 2021-01-06T22:05:22.324 INFO:tasks.workunit.client.0.smithi203.stdout:checking for an ANSI C-conforming const... yes 2021-01-06T22:05:22.416 INFO:tasks.workunit.client.0.smithi203.stdout:checking whether time.h and sys/time.h may both be included... yes 2021-01-06T22:05:22.422 INFO:tasks.workunit.client.0.smithi203.stdout:checking for function prototypes... yes 2021-01-06T22:05:22.534 INFO:tasks.workunit.client.0.smithi203.stdout:checking whether setvbuf arguments are reversed... no 2021-01-06T22:05:22.711 INFO:tasks.workunit.client.0.smithi203.stdout:checking for system... yes 2021-01-06T22:05:23.015 INFO:tasks.workunit.client.0.smithi203.stdout:checking for gettimeofday... yes 2021-01-06T22:05:23.062 INFO:tasks.workunit.client.0.smithi203.stdout:checking for mkdir... yes 2021-01-06T22:05:23.231 INFO:tasks.workunit.client.0.smithi203.stdout:checking for strerror... yes 2021-01-06T22:05:23.412 INFO:tasks.workunit.client.0.smithi203.stdout:checking for open64... yes 2021-01-06T22:05:24.205 INFO:tasks.workunit.client.0.smithi203.stdout:checking for stat64... yes 2021-01-06T22:05:24.374 INFO:tasks.workunit.client.0.smithi203.stdout:checking for fseeko64... yes 2021-01-06T22:05:24.551 INFO:tasks.workunit.client.0.smithi203.stdout:checking for lrand48_r... yes 2021-01-06T22:05:25.270 INFO:tasks.workunit.client.0.smithi203.stdout:checking for srand48_r... yes 2021-01-06T22:05:25.339 INFO:tasks.workunit.client.0.smithi203.stdout:configure: creating ./config.status 2021-01-06T22:05:25.948 INFO:tasks.workunit.client.0.smithi203.stdout:config.status: creating Makefile 2021-01-06T22:05:25.992 INFO:tasks.workunit.client.0.smithi203.stdout:config.status: creating config.h 2021-01-06T22:05:26.051 INFO:tasks.workunit.client.0.smithi203.stdout:config.status: executing depfiles commands 2021-01-06T22:05:26.231 INFO:tasks.workunit.client.0.smithi203.stdout:config.status: executing default commands 2021-01-06T22:05:26.374 INFO:tasks.workunit.client.0.smithi203.stdout:/* config.h. Generated from config.h.in by configure. */ 2021-01-06T22:05:26.374 INFO:tasks.workunit.client.0.smithi203.stdout:/* config.h.in. Generated from configure.in by autoheader. */ 2021-01-06T22:05:26.374 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.375 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the header file. */ 2021-01-06T22:05:26.375 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_FCNTL_H 1 2021-01-06T22:05:26.375 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.376 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the `fseeko64' function. */ 2021-01-06T22:05:26.376 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_FSEEKO64 1 2021-01-06T22:05:26.376 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.376 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the `gettimeofday' function. */ 2021-01-06T22:05:26.376 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_GETTIMEOFDAY 1 2021-01-06T22:05:26.377 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.377 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the header file. */ 2021-01-06T22:05:26.377 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_INTTYPES_H 1 2021-01-06T22:05:26.377 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.378 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the `m' library (-lm). */ 2021-01-06T22:05:26.378 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_LIBM 1 2021-01-06T22:05:26.379 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.379 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the `pthread' library (-lpthread). */ 2021-01-06T22:05:26.379 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_LIBPTHREAD 1 2021-01-06T22:05:26.379 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.380 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the header file. */ 2021-01-06T22:05:26.380 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_LIMITS_H 1 2021-01-06T22:05:26.380 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.380 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the `lrand48_r' function. */ 2021-01-06T22:05:26.381 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_LRAND48_R 1 2021-01-06T22:05:26.381 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.381 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the header file. */ 2021-01-06T22:05:26.381 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_MEMORY_H 1 2021-01-06T22:05:26.382 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.382 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the `mkdir' function. */ 2021-01-06T22:05:26.382 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_MKDIR 1 2021-01-06T22:05:26.383 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.383 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the `open64' function. */ 2021-01-06T22:05:26.383 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_OPEN64 1 2021-01-06T22:05:26.383 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.383 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the header file. */ 2021-01-06T22:05:26.384 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_PTHREAD_H 1 2021-01-06T22:05:26.384 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.384 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the `srand48_r' function. */ 2021-01-06T22:05:26.384 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_SRAND48_R 1 2021-01-06T22:05:26.385 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.385 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the `stat64' function. */ 2021-01-06T22:05:26.385 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_STAT64 1 2021-01-06T22:05:26.385 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.386 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the header file. */ 2021-01-06T22:05:26.386 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_STDINT_H 1 2021-01-06T22:05:26.386 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.386 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the header file. */ 2021-01-06T22:05:26.387 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_STDLIB_H 1 2021-01-06T22:05:26.387 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.387 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the `strerror' function. */ 2021-01-06T22:05:26.388 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_STRERROR 1 2021-01-06T22:05:26.388 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.388 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the header file. */ 2021-01-06T22:05:26.388 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_STRINGS_H 1 2021-01-06T22:05:26.388 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.389 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the header file. */ 2021-01-06T22:05:26.389 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_STRING_H 1 2021-01-06T22:05:26.389 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.390 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the `system' function. */ 2021-01-06T22:05:26.390 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_SYSTEM 1 2021-01-06T22:05:26.390 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.390 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the header file. */ 2021-01-06T22:05:26.391 INFO:tasks.workunit.client.0.smithi203.stdout:/* #undef HAVE_SYS_LIMITS_H */ 2021-01-06T22:05:26.391 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.391 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the header file. */ 2021-01-06T22:05:26.392 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_SYS_STAT_H 1 2021-01-06T22:05:26.392 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.392 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the header file. */ 2021-01-06T22:05:26.392 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_SYS_TIME_H 1 2021-01-06T22:05:26.392 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.393 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the header file. */ 2021-01-06T22:05:26.393 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_SYS_TYPES_H 1 2021-01-06T22:05:26.393 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.393 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the header file. */ 2021-01-06T22:05:26.394 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_SYS_VFS_H 1 2021-01-06T22:05:26.394 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.394 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have that is POSIX.1 compatible. */ 2021-01-06T22:05:26.394 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_SYS_WAIT_H 1 2021-01-06T22:05:26.395 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.395 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the header file. */ 2021-01-06T22:05:26.395 INFO:tasks.workunit.client.0.smithi203.stdout:#define HAVE_UNISTD_H 1 2021-01-06T22:05:26.395 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.395 INFO:tasks.workunit.client.0.smithi203.stdout:/* Name of package */ 2021-01-06T22:05:26.396 INFO:tasks.workunit.client.0.smithi203.stdout:#define PACKAGE "ffsb" 2021-01-06T22:05:26.396 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.396 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to the address where bug reports for this package should be sent. */ 2021-01-06T22:05:26.396 INFO:tasks.workunit.client.0.smithi203.stdout:#define PACKAGE_BUGREPORT "" 2021-01-06T22:05:26.397 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.397 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to the full name of this package. */ 2021-01-06T22:05:26.397 INFO:tasks.workunit.client.0.smithi203.stdout:#define PACKAGE_NAME "" 2021-01-06T22:05:26.397 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.398 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to the full name and version of this package. */ 2021-01-06T22:05:26.398 INFO:tasks.workunit.client.0.smithi203.stdout:#define PACKAGE_STRING "" 2021-01-06T22:05:26.398 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.398 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to the one symbol short name of this package. */ 2021-01-06T22:05:26.399 INFO:tasks.workunit.client.0.smithi203.stdout:#define PACKAGE_TARNAME "" 2021-01-06T22:05:26.399 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.399 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to the version of this package. */ 2021-01-06T22:05:26.400 INFO:tasks.workunit.client.0.smithi203.stdout:#define PACKAGE_VERSION "" 2021-01-06T22:05:26.400 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.400 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if the C compiler supports function prototypes. */ 2021-01-06T22:05:26.400 INFO:tasks.workunit.client.0.smithi203.stdout:#define PROTOTYPES 1 2021-01-06T22:05:26.400 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.401 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if the `setvbuf' function takes the buffering type as its 2021-01-06T22:05:26.401 INFO:tasks.workunit.client.0.smithi203.stdout: second argument and the buffer pointer as the third, as on System V before 2021-01-06T22:05:26.401 INFO:tasks.workunit.client.0.smithi203.stdout: release 3. */ 2021-01-06T22:05:26.402 INFO:tasks.workunit.client.0.smithi203.stdout:/* #undef SETVBUF_REVERSED */ 2021-01-06T22:05:26.402 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.403 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you have the ANSI C header files. */ 2021-01-06T22:05:26.403 INFO:tasks.workunit.client.0.smithi203.stdout:#define STDC_HEADERS 1 2021-01-06T22:05:26.403 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.403 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if you can safely include both and . */ 2021-01-06T22:05:26.404 INFO:tasks.workunit.client.0.smithi203.stdout:#define TIME_WITH_SYS_TIME 1 2021-01-06T22:05:26.404 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.404 INFO:tasks.workunit.client.0.smithi203.stdout:/* Version number of package */ 2021-01-06T22:05:26.404 INFO:tasks.workunit.client.0.smithi203.stdout:#define VERSION "6.0-RC2" 2021-01-06T22:05:26.405 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.405 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to 1 if on AIX 3. 2021-01-06T22:05:26.406 INFO:tasks.workunit.client.0.smithi203.stdout: System headers sometimes define this. 2021-01-06T22:05:26.406 INFO:tasks.workunit.client.0.smithi203.stdout: We just want to avoid a redefinition error message. */ 2021-01-06T22:05:26.406 INFO:tasks.workunit.client.0.smithi203.stdout:#ifndef _ALL_SOURCE 2021-01-06T22:05:26.406 INFO:tasks.workunit.client.0.smithi203.stdout:/* # undef _ALL_SOURCE */ 2021-01-06T22:05:26.406 INFO:tasks.workunit.client.0.smithi203.stdout:#endif 2021-01-06T22:05:26.408 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.408 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define like PROTOTYPES; this can be used by system headers. */ 2021-01-06T22:05:26.408 INFO:tasks.workunit.client.0.smithi203.stdout:#define __PROTOTYPES 1 2021-01-06T22:05:26.408 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:26.409 INFO:tasks.workunit.client.0.smithi203.stdout:/* Define to empty if `const' does not conform to ANSI C. */ 2021-01-06T22:05:26.409 INFO:tasks.workunit.client.0.smithi203.stdout:/* #undef const */ 2021-01-06T22:05:26.455 INFO:tasks.workunit.client.0.smithi203.stdout:make all-am 2021-01-06T22:05:26.512 INFO:tasks.workunit.client.0.smithi203.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb-6.0-rc2' 2021-01-06T22:05:26.513 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2021-01-06T22:05:26.611 INFO:tasks.workunit.client.0.smithi203.stderr:fileops.c: In function 'ffsb_readfile': 2021-01-06T22:05:26.612 INFO:tasks.workunit.client.0.smithi203.stderr:fileops.c:138:35: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t {aka long unsigned int}' [-Wformat=] 2021-01-06T22:05:26.612 INFO:tasks.workunit.client.0.smithi203.stderr: printf("Error: read size %llu bytes too big " 2021-01-06T22:05:26.612 INFO:tasks.workunit.client.0.smithi203.stderr: ~~~^ 2021-01-06T22:05:26.613 INFO:tasks.workunit.client.0.smithi203.stderr: %lu 2021-01-06T22:05:26.614 INFO:tasks.workunit.client.0.smithi203.stderr:fileops.c:138:14: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 5 has type 'uint64_t {aka long unsigned int}' [-Wformat=] 2021-01-06T22:05:26.614 INFO:tasks.workunit.client.0.smithi203.stderr: printf("Error: read size %llu bytes too big " 2021-01-06T22:05:26.614 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2021-01-06T22:05:26.614 INFO:tasks.workunit.client.0.smithi203.stderr:fileops.c:140:29: note: format string is defined here 2021-01-06T22:05:26.615 INFO:tasks.workunit.client.0.smithi203.stderr: " for file of size %llu bytes\n" 2021-01-06T22:05:26.615 INFO:tasks.workunit.client.0.smithi203.stderr: ~~~^ 2021-01-06T22:05:26.615 INFO:tasks.workunit.client.0.smithi203.stderr: %lu 2021-01-06T22:05:26.615 INFO:tasks.workunit.client.0.smithi203.stderr:fileops.c:144:14: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t {aka long unsigned int}' [-Wformat=] 2021-01-06T22:05:26.615 INFO:tasks.workunit.client.0.smithi203.stderr: printf("minimum file size must be at least " 2021-01-06T22:05:26.616 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2021-01-06T22:05:26.617 INFO:tasks.workunit.client.0.smithi203.stderr:fileops.c:145:12: note: format string is defined here 2021-01-06T22:05:26.618 INFO:tasks.workunit.client.0.smithi203.stderr: " %llu bytes\n", minfilesize); 2021-01-06T22:05:26.619 INFO:tasks.workunit.client.0.smithi203.stderr: ~~~^ 2021-01-06T22:05:26.619 INFO:tasks.workunit.client.0.smithi203.stderr: %lu 2021-01-06T22:05:26.620 INFO:tasks.workunit.client.0.smithi203.stderr:fileops.c: In function 'ffsb_stat': 2021-01-06T22:05:26.620 INFO:tasks.workunit.client.0.smithi203.stderr:fileops.c:542:2: warning: implicit declaration of function 'fhstat'; did you mean 'fstat'? [-Wimplicit-function-declaration] 2021-01-06T22:05:26.620 INFO:tasks.workunit.client.0.smithi203.stderr: fhstat(curfile->name, ft, fs); 2021-01-06T22:05:26.620 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~ 2021-01-06T22:05:26.621 INFO:tasks.workunit.client.0.smithi203.stderr: fstat 2021-01-06T22:05:26.748 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2021-01-06T22:05:26.831 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2021-01-06T22:05:27.013 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2021-01-06T22:05:27.057 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2021-01-06T22:05:27.272 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/main.Tpo .deps/main.Po 2021-01-06T22:05:27.581 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2021-01-06T22:05:27.691 INFO:tasks.workunit.client.0.smithi203.stderr:fh.c: In function 'fhwrite': 2021-01-06T22:05:27.692 INFO:tasks.workunit.client.0.smithi203.stderr:fh.c:180:18: warning: format '%d' expects argument of type 'int', but argument 2 has type 'ssize_t {aka long int}' [-Wformat=] 2021-01-06T22:05:27.692 INFO:tasks.workunit.client.0.smithi203.stderr: printf("Wrote %d instead of %d bytes.\n" 2021-01-06T22:05:27.692 INFO:tasks.workunit.client.0.smithi203.stderr: ~^ 2021-01-06T22:05:27.693 INFO:tasks.workunit.client.0.smithi203.stderr: %ld 2021-01-06T22:05:27.693 INFO:tasks.workunit.client.0.smithi203.stderr:fh.c: In function 'fhseek': 2021-01-06T22:05:27.693 INFO:tasks.workunit.client.0.smithi203.stderr:fh.c:212:41: warning: format '%lld' expects argument of type 'long long int', but argument 3 has type 'uint64_t {aka long unsigned int}' [-Wformat=] 2021-01-06T22:05:27.693 INFO:tasks.workunit.client.0.smithi203.stderr: fprintf(stderr, "tried to seek to %lld\n", offset); 2021-01-06T22:05:27.693 INFO:tasks.workunit.client.0.smithi203.stderr: ~~~^ 2021-01-06T22:05:27.694 INFO:tasks.workunit.client.0.smithi203.stderr: %ld 2021-01-06T22:05:27.699 INFO:tasks.workunit.client.0.smithi203.stderr:fh.c:214:20: warning: format '%lld' expects argument of type 'long long int', but argument 3 has type 'uint64_t {aka long unsigned int}' [-Wformat=] 2021-01-06T22:05:27.700 INFO:tasks.workunit.client.0.smithi203.stderr: fprintf(stderr, "tried to seek from current " 2021-01-06T22:05:27.700 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2021-01-06T22:05:27.701 INFO:tasks.workunit.client.0.smithi203.stderr:fh.c:215:21: note: format string is defined here 2021-01-06T22:05:27.701 INFO:tasks.workunit.client.0.smithi203.stderr: "position to %lld\n", offset); 2021-01-06T22:05:27.701 INFO:tasks.workunit.client.0.smithi203.stderr: ~~~^ 2021-01-06T22:05:27.702 INFO:tasks.workunit.client.0.smithi203.stderr: %ld 2021-01-06T22:05:27.798 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2021-01-06T22:05:27.826 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2021-01-06T22:05:28.060 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2021-01-06T22:05:28.090 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2021-01-06T22:05:28.243 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2021-01-06T22:05:28.273 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2021-01-06T22:05:28.541 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2021-01-06T22:05:28.570 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2021-01-06T22:05:28.716 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2021-01-06T22:05:28.740 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2021-01-06T22:05:28.993 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2021-01-06T22:05:29.020 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2021-01-06T22:05:29.119 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_tg.c: In function 'tg_print_config_helper': 2021-01-06T22:05:29.119 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_tg.c:274:35: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t {aka long unsigned int}' [-Wformat=] 2021-01-06T22:05:29.120 INFO:tasks.workunit.client.0.smithi203.stderr: printf("\t read_size = %llu\t(%s)\n", tg->read_size, 2021-01-06T22:05:29.120 INFO:tasks.workunit.client.0.smithi203.stderr: ~~~^ ~~~~~~~~~~~~~ 2021-01-06T22:05:29.120 INFO:tasks.workunit.client.0.smithi203.stderr: %lu 2021-01-06T22:05:29.121 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_tg.c:283:35: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t {aka long unsigned int}' [-Wformat=] 2021-01-06T22:05:29.121 INFO:tasks.workunit.client.0.smithi203.stderr: printf("\t write_size = %llu\t(%s)\n", tg->write_size, 2021-01-06T22:05:29.121 INFO:tasks.workunit.client.0.smithi203.stderr: ~~~^ ~~~~~~~~~~~~~~ 2021-01-06T22:05:29.122 INFO:tasks.workunit.client.0.smithi203.stderr: %lu 2021-01-06T22:05:29.250 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2021-01-06T22:05:30.118 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2021-01-06T22:05:30.231 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_fs.c: In function 'verify_file': 2021-01-06T22:05:30.231 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_fs.c:217:19: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t {aka long unsigned int}' [-Wformat=] 2021-01-06T22:05:30.232 INFO:tasks.workunit.client.0.smithi203.stderr: printf("size %llu bytes for file %s is invalid\n", 2021-01-06T22:05:30.232 INFO:tasks.workunit.client.0.smithi203.stderr: ~~~^ 2021-01-06T22:05:30.232 INFO:tasks.workunit.client.0.smithi203.stderr: %lu 2021-01-06T22:05:30.233 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_fs.c: In function 'fs_print_config': 2021-01-06T22:05:30.233 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_fs.c:595:22: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t {aka long unsigned int}' [-Wformat=] 2021-01-06T22:05:30.234 INFO:tasks.workunit.client.0.smithi203.stderr: printf("\t\t %12llu (%6s) -> %u (%.2f\%)\n", 2021-01-06T22:05:30.234 INFO:tasks.workunit.client.0.smithi203.stderr: ~~~~~^ 2021-01-06T22:05:30.234 INFO:tasks.workunit.client.0.smithi203.stderr: %12lu 2021-01-06T22:05:30.234 INFO:tasks.workunit.client.0.smithi203.stderr: fs->size_weights[i].size, 2021-01-06T22:05:30.234 INFO:tasks.workunit.client.0.smithi203.stderr: ~~~~~~~~~~~~~~~~~~~~~~~~ 2021-01-06T22:05:30.235 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_fs.c:595:43: warning: unknown conversion type character ')' in format [-Wformat=] 2021-01-06T22:05:30.235 INFO:tasks.workunit.client.0.smithi203.stderr: printf("\t\t %12llu (%6s) -> %u (%.2f\%)\n", 2021-01-06T22:05:30.235 INFO:tasks.workunit.client.0.smithi203.stderr: ^ 2021-01-06T22:05:30.235 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_fs.c:603:36: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t {aka long unsigned int}' [-Wformat=] 2021-01-06T22:05:30.236 INFO:tasks.workunit.client.0.smithi203.stderr: printf("\t min file size = %llu\t(%s)\n", fs->minfilesize, 2021-01-06T22:05:30.236 INFO:tasks.workunit.client.0.smithi203.stderr: ~~~^ ~~~~~~~~~~~~~~~ 2021-01-06T22:05:30.236 INFO:tasks.workunit.client.0.smithi203.stderr: %lu 2021-01-06T22:05:30.237 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_fs.c:605:36: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t {aka long unsigned int}' [-Wformat=] 2021-01-06T22:05:30.237 INFO:tasks.workunit.client.0.smithi203.stderr: printf("\t max file size = %llu\t(%s)\n", fs->maxfilesize, 2021-01-06T22:05:30.237 INFO:tasks.workunit.client.0.smithi203.stderr: ~~~^ ~~~~~~~~~~~~~~~ 2021-01-06T22:05:30.237 INFO:tasks.workunit.client.0.smithi203.stderr: %lu 2021-01-06T22:05:30.238 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_fs.c:616:41: warning: unknown conversion type character '\x0a' in format [-Wformat=] 2021-01-06T22:05:30.238 INFO:tasks.workunit.client.0.smithi203.stderr: printf("\t current utilization = %.2f\%\n", getfsutil(fs->basedir)*100); 2021-01-06T22:05:30.238 INFO:tasks.workunit.client.0.smithi203.stderr: ^~ 2021-01-06T22:05:30.238 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_fs.c:618:42: warning: unknown conversion type character '\x0a' in format [-Wformat=] 2021-01-06T22:05:30.238 INFO:tasks.workunit.client.0.smithi203.stderr: printf("\t desired utilization = %.2lf%\n", fs->desired_fsutil * 100); 2021-01-06T22:05:30.239 INFO:tasks.workunit.client.0.smithi203.stderr: ^~ 2021-01-06T22:05:30.239 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_fs.c: In function 'fs_needs_stats': 2021-01-06T22:05:30.239 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_fs.c:627:24: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2021-01-06T22:05:30.239 INFO:tasks.workunit.client.0.smithi203.stderr: return (fs != NULL) ? (int)fs->fsd.config : 0; 2021-01-06T22:05:30.239 INFO:tasks.workunit.client.0.smithi203.stderr: ^ 2021-01-06T22:05:30.386 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2021-01-06T22:05:30.412 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2021-01-06T22:05:30.597 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2021-01-06T22:05:30.626 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2021-01-06T22:05:30.732 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_op.c: In function 'print_op_results': 2021-01-06T22:05:30.732 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_op.c:83:23: warning: embedded '\0' in format [-Wformat-contains-nul] 2021-01-06T22:05:30.733 INFO:tasks.workunit.client.0.smithi203.stderr: sprintf(buf, "%s/sec\0", buf); 2021-01-06T22:05:30.733 INFO:tasks.workunit.client.0.smithi203.stderr: ^~ 2021-01-06T22:05:30.733 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_op.c:86:19: warning: embedded '\0' in format [-Wformat-contains-nul] 2021-01-06T22:05:30.733 INFO:tasks.workunit.client.0.smithi203.stderr: sprintf(buf, "NA\0"); 2021-01-06T22:05:30.734 INFO:tasks.workunit.client.0.smithi203.stderr: ^~ 2021-01-06T22:05:30.734 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_op.c: In function 'print_results': 2021-01-06T22:05:30.734 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_op.c:112:61: warning: unknown conversion type character 'T' in format [-Wformat=] 2021-01-06T22:05:30.734 INFO:tasks.workunit.client.0.smithi203.stderr: printf(" Op Name Transactions\t Trans/sec\t% Trans\t % Op Weight\t Throughput\n"); 2021-01-06T22:05:30.734 INFO:tasks.workunit.client.0.smithi203.stderr: ^ 2021-01-06T22:05:30.735 INFO:tasks.workunit.client.0.smithi203.stderr:ffsb_op.c:112:74: warning: unknown conversion type character 'O' in format [-Wformat=] 2021-01-06T22:05:30.735 INFO:tasks.workunit.client.0.smithi203.stderr: printf(" Op Name Transactions\t Trans/sec\t% Trans\t % Op Weight\t Throughput\n"); 2021-01-06T22:05:30.735 INFO:tasks.workunit.client.0.smithi203.stderr: ^ 2021-01-06T22:05:30.826 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2021-01-06T22:05:30.847 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2021-01-06T22:05:30.950 INFO:tasks.workunit.client.0.smithi203.stderr:util.c: In function 'ffsb_bench_gettimeofday': 2021-01-06T22:05:30.951 INFO:tasks.workunit.client.0.smithi203.stderr:util.c:333:46: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t {aka long unsigned int}' [-Wformat=] 2021-01-06T22:05:30.951 INFO:tasks.workunit.client.0.smithi203.stderr: printf("average time for gettimeofday(): %llu nsec\n", average); 2021-01-06T22:05:30.951 INFO:tasks.workunit.client.0.smithi203.stderr: ~~~^ 2021-01-06T22:05:30.952 INFO:tasks.workunit.client.0.smithi203.stderr: %lu 2021-01-06T22:05:30.952 INFO:tasks.workunit.client.0.smithi203.stderr:util.c: In function 'ffsb_bench_getpid': 2021-01-06T22:05:30.952 INFO:tasks.workunit.client.0.smithi203.stderr:util.c:350:40: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t {aka long unsigned int}' [-Wformat=] 2021-01-06T22:05:30.952 INFO:tasks.workunit.client.0.smithi203.stderr: printf("average time for getpid(): %llu nsec\n", average); 2021-01-06T22:05:30.952 INFO:tasks.workunit.client.0.smithi203.stderr: ~~~^ 2021-01-06T22:05:30.953 INFO:tasks.workunit.client.0.smithi203.stderr: %lu 2021-01-06T22:05:31.066 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/util.Tpo .deps/util.Po 2021-01-06T22:05:31.092 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2021-01-06T22:05:31.198 INFO:tasks.workunit.client.0.smithi203.stderr:In file included from parser.h:22:0, 2021-01-06T22:05:31.199 INFO:tasks.workunit.client.0.smithi203.stderr: from parser.c:25: 2021-01-06T22:05:31.199 INFO:tasks.workunit.client.0.smithi203.stderr:parser.c: In function 'init_filesys': 2021-01-06T22:05:31.201 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2021-01-06T22:05:31.202 INFO:tasks.workunit.client.0.smithi203.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2021-01-06T22:05:31.202 INFO:tasks.workunit.client.0.smithi203.stderr: ^ 2021-01-06T22:05:31.202 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2021-01-06T22:05:31.202 INFO:tasks.workunit.client.0.smithi203.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2021-01-06T22:05:31.203 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~ 2021-01-06T22:05:31.203 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2021-01-06T22:05:31.203 INFO:tasks.workunit.client.0.smithi203.stderr: container_of(ptr, type, member) 2021-01-06T22:05:31.203 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~ 2021-01-06T22:05:31.204 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:39:13: note: in expansion of macro 'list_entry' 2021-01-06T22:05:31.204 INFO:tasks.workunit.client.0.smithi203.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2021-01-06T22:05:31.204 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~ 2021-01-06T22:05:31.204 INFO:tasks.workunit.client.0.smithi203.stderr:parser.c:934:3: note: in expansion of macro 'list_for_each_entry' 2021-01-06T22:05:31.204 INFO:tasks.workunit.client.0.smithi203.stderr: list_for_each_entry(tmp_list, &list_head->list, list) 2021-01-06T22:05:31.205 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~~~~~~~~ 2021-01-06T22:05:31.205 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2021-01-06T22:05:31.205 INFO:tasks.workunit.client.0.smithi203.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2021-01-06T22:05:31.205 INFO:tasks.workunit.client.0.smithi203.stderr: ^ 2021-01-06T22:05:31.206 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2021-01-06T22:05:31.206 INFO:tasks.workunit.client.0.smithi203.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2021-01-06T22:05:31.206 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~ 2021-01-06T22:05:31.206 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2021-01-06T22:05:31.206 INFO:tasks.workunit.client.0.smithi203.stderr: container_of(ptr, type, member) 2021-01-06T22:05:31.207 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~ 2021-01-06T22:05:31.207 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:41:13: note: in expansion of macro 'list_entry' 2021-01-06T22:05:31.207 INFO:tasks.workunit.client.0.smithi203.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2021-01-06T22:05:31.207 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~ 2021-01-06T22:05:31.207 INFO:tasks.workunit.client.0.smithi203.stderr:parser.c:934:3: note: in expansion of macro 'list_for_each_entry' 2021-01-06T22:05:31.208 INFO:tasks.workunit.client.0.smithi203.stderr: list_for_each_entry(tmp_list, &list_head->list, list) 2021-01-06T22:05:31.208 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~~~~~~~~ 2021-01-06T22:05:31.208 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2021-01-06T22:05:31.208 INFO:tasks.workunit.client.0.smithi203.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2021-01-06T22:05:31.209 INFO:tasks.workunit.client.0.smithi203.stderr: ^ 2021-01-06T22:05:31.209 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2021-01-06T22:05:31.209 INFO:tasks.workunit.client.0.smithi203.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2021-01-06T22:05:31.209 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~ 2021-01-06T22:05:31.209 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2021-01-06T22:05:31.210 INFO:tasks.workunit.client.0.smithi203.stderr: container_of(ptr, type, member) 2021-01-06T22:05:31.210 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~ 2021-01-06T22:05:31.210 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:39:13: note: in expansion of macro 'list_entry' 2021-01-06T22:05:31.210 INFO:tasks.workunit.client.0.smithi203.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2021-01-06T22:05:31.211 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~ 2021-01-06T22:05:31.211 INFO:tasks.workunit.client.0.smithi203.stderr:parser.c:941:3: note: in expansion of macro 'list_for_each_entry' 2021-01-06T22:05:31.211 INFO:tasks.workunit.client.0.smithi203.stderr: list_for_each_entry(tmp_list, &list_head->list, list) { 2021-01-06T22:05:31.211 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~~~~~~~~ 2021-01-06T22:05:31.212 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2021-01-06T22:05:31.212 INFO:tasks.workunit.client.0.smithi203.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2021-01-06T22:05:31.212 INFO:tasks.workunit.client.0.smithi203.stderr: ^ 2021-01-06T22:05:31.213 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2021-01-06T22:05:31.213 INFO:tasks.workunit.client.0.smithi203.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2021-01-06T22:05:31.213 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~ 2021-01-06T22:05:31.213 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2021-01-06T22:05:31.213 INFO:tasks.workunit.client.0.smithi203.stderr: container_of(ptr, type, member) 2021-01-06T22:05:31.214 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~ 2021-01-06T22:05:31.214 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:41:13: note: in expansion of macro 'list_entry' 2021-01-06T22:05:31.214 INFO:tasks.workunit.client.0.smithi203.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2021-01-06T22:05:31.214 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~ 2021-01-06T22:05:31.215 INFO:tasks.workunit.client.0.smithi203.stderr:parser.c:941:3: note: in expansion of macro 'list_for_each_entry' 2021-01-06T22:05:31.215 INFO:tasks.workunit.client.0.smithi203.stderr: list_for_each_entry(tmp_list, &list_head->list, list) { 2021-01-06T22:05:31.215 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~~~~~~~~ 2021-01-06T22:05:31.215 INFO:tasks.workunit.client.0.smithi203.stderr:parser.c: In function 'init_tg_stats': 2021-01-06T22:05:31.215 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2021-01-06T22:05:31.216 INFO:tasks.workunit.client.0.smithi203.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2021-01-06T22:05:31.216 INFO:tasks.workunit.client.0.smithi203.stderr: ^ 2021-01-06T22:05:31.216 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2021-01-06T22:05:31.217 INFO:tasks.workunit.client.0.smithi203.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2021-01-06T22:05:31.217 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~ 2021-01-06T22:05:31.217 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2021-01-06T22:05:31.217 INFO:tasks.workunit.client.0.smithi203.stderr: container_of(ptr, type, member) 2021-01-06T22:05:31.218 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~ 2021-01-06T22:05:31.218 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:39:13: note: in expansion of macro 'list_entry' 2021-01-06T22:05:31.218 INFO:tasks.workunit.client.0.smithi203.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2021-01-06T22:05:31.218 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~ 2021-01-06T22:05:31.219 INFO:tasks.workunit.client.0.smithi203.stderr:parser.c:971:6: note: in expansion of macro 'list_for_each_entry' 2021-01-06T22:05:31.219 INFO:tasks.workunit.client.0.smithi203.stderr: list_for_each_entry(tmp_list, 2021-01-06T22:05:31.219 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~~~~~~~~ 2021-01-06T22:05:31.219 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2021-01-06T22:05:31.223 INFO:tasks.workunit.client.0.smithi203.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2021-01-06T22:05:31.224 INFO:tasks.workunit.client.0.smithi203.stderr: ^ 2021-01-06T22:05:31.224 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2021-01-06T22:05:31.224 INFO:tasks.workunit.client.0.smithi203.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2021-01-06T22:05:31.224 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~ 2021-01-06T22:05:31.225 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2021-01-06T22:05:31.225 INFO:tasks.workunit.client.0.smithi203.stderr: container_of(ptr, type, member) 2021-01-06T22:05:31.225 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~ 2021-01-06T22:05:31.225 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:41:13: note: in expansion of macro 'list_entry' 2021-01-06T22:05:31.226 INFO:tasks.workunit.client.0.smithi203.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2021-01-06T22:05:31.226 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~ 2021-01-06T22:05:31.226 INFO:tasks.workunit.client.0.smithi203.stderr:parser.c:971:6: note: in expansion of macro 'list_for_each_entry' 2021-01-06T22:05:31.226 INFO:tasks.workunit.client.0.smithi203.stderr: list_for_each_entry(tmp_list, 2021-01-06T22:05:31.226 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~~~~~~~~ 2021-01-06T22:05:31.227 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2021-01-06T22:05:31.227 INFO:tasks.workunit.client.0.smithi203.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2021-01-06T22:05:31.227 INFO:tasks.workunit.client.0.smithi203.stderr: ^ 2021-01-06T22:05:31.227 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2021-01-06T22:05:31.227 INFO:tasks.workunit.client.0.smithi203.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2021-01-06T22:05:31.228 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~ 2021-01-06T22:05:31.228 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2021-01-06T22:05:31.228 INFO:tasks.workunit.client.0.smithi203.stderr: container_of(ptr, type, member) 2021-01-06T22:05:31.228 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~ 2021-01-06T22:05:31.228 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:39:13: note: in expansion of macro 'list_entry' 2021-01-06T22:05:31.229 INFO:tasks.workunit.client.0.smithi203.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2021-01-06T22:05:31.229 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~ 2021-01-06T22:05:31.229 INFO:tasks.workunit.client.0.smithi203.stderr:parser.c:980:6: note: in expansion of macro 'list_for_each_entry' 2021-01-06T22:05:31.229 INFO:tasks.workunit.client.0.smithi203.stderr: list_for_each_entry(tmp_list, 2021-01-06T22:05:31.229 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~~~~~~~~ 2021-01-06T22:05:31.230 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2021-01-06T22:05:31.230 INFO:tasks.workunit.client.0.smithi203.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2021-01-06T22:05:31.230 INFO:tasks.workunit.client.0.smithi203.stderr: ^ 2021-01-06T22:05:31.230 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2021-01-06T22:05:31.230 INFO:tasks.workunit.client.0.smithi203.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2021-01-06T22:05:31.231 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~ 2021-01-06T22:05:31.231 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2021-01-06T22:05:31.231 INFO:tasks.workunit.client.0.smithi203.stderr: container_of(ptr, type, member) 2021-01-06T22:05:31.231 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~ 2021-01-06T22:05:31.231 INFO:tasks.workunit.client.0.smithi203.stderr:list.h:41:13: note: in expansion of macro 'list_entry' 2021-01-06T22:05:31.232 INFO:tasks.workunit.client.0.smithi203.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2021-01-06T22:05:31.232 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~ 2021-01-06T22:05:31.232 INFO:tasks.workunit.client.0.smithi203.stderr:parser.c:980:6: note: in expansion of macro 'list_for_each_entry' 2021-01-06T22:05:31.232 INFO:tasks.workunit.client.0.smithi203.stderr: list_for_each_entry(tmp_list, 2021-01-06T22:05:31.232 INFO:tasks.workunit.client.0.smithi203.stderr: ^~~~~~~~~~~~~~~~~~~ 2021-01-06T22:05:31.586 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2021-01-06T22:05:31.615 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2021-01-06T22:05:31.796 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2021-01-06T22:05:31.840 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2021-01-06T22:05:32.522 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2021-01-06T22:05:32.560 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2021-01-06T22:05:32.748 INFO:tasks.workunit.client.0.smithi203.stdout:mv -f .deps/list.Tpo .deps/list.Po 2021-01-06T22:05:32.797 INFO:tasks.workunit.client.0.smithi203.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2021-01-06T22:05:33.104 INFO:tasks.workunit.client.0.smithi203.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb-6.0-rc2' 2021-01-06T22:05:33.137 INFO:tasks.workunit.client.0.smithi203.stdout:FFSB version 6.0-RC2 started 2021-01-06T22:05:33.137 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:33.145 INFO:tasks.workunit.client.0.smithi203.stdout:benchmark time = 300 2021-01-06T22:05:33.145 INFO:tasks.workunit.client.0.smithi203.stdout:ThreadGroup 0 2021-01-06T22:05:33.145 INFO:tasks.workunit.client.0.smithi203.stdout:================ 2021-01-06T22:05:33.145 INFO:tasks.workunit.client.0.smithi203.stdout: num_threads = 32 2021-01-06T22:05:33.146 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:33.146 INFO:tasks.workunit.client.0.smithi203.stdout: read_random = off 2021-01-06T22:05:33.146 INFO:tasks.workunit.client.0.smithi203.stdout: read_size = 0 (0B) 2021-01-06T22:05:33.146 INFO:tasks.workunit.client.0.smithi203.stdout: read_blocksize = 0 (0B) 2021-01-06T22:05:33.146 INFO:tasks.workunit.client.0.smithi203.stdout: read_skip = off 2021-01-06T22:05:33.147 INFO:tasks.workunit.client.0.smithi203.stdout: read_skipsize = 0 (0B) 2021-01-06T22:05:33.147 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:33.147 INFO:tasks.workunit.client.0.smithi203.stdout: write_random = on 2021-01-06T22:05:33.147 INFO:tasks.workunit.client.0.smithi203.stdout: write_size = 5242880 (5MB) 2021-01-06T22:05:33.148 INFO:tasks.workunit.client.0.smithi203.stdout: fsync_file = 0 2021-01-06T22:05:33.148 INFO:tasks.workunit.client.0.smithi203.stdout: write_blocksize = 4096 (4KB) 2021-01-06T22:05:33.148 INFO:tasks.workunit.client.0.smithi203.stdout: wait time = 0 2021-01-06T22:05:33.148 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:33.148 INFO:tasks.workunit.client.0.smithi203.stdout: op weights 2021-01-06T22:05:33.149 INFO:tasks.workunit.client.0.smithi203.stdout: read = 0 (0.00%) 2021-01-06T22:05:33.149 INFO:tasks.workunit.client.0.smithi203.stdout: readall = 0 (0.00%) 2021-01-06T22:05:33.149 INFO:tasks.workunit.client.0.smithi203.stdout: write = 1 (100.00%) 2021-01-06T22:05:33.149 INFO:tasks.workunit.client.0.smithi203.stdout: create = 0 (0.00%) 2021-01-06T22:05:33.149 INFO:tasks.workunit.client.0.smithi203.stdout: append = 0 (0.00%) 2021-01-06T22:05:33.149 INFO:tasks.workunit.client.0.smithi203.stdout: delete = 0 (0.00%) 2021-01-06T22:05:33.150 INFO:tasks.workunit.client.0.smithi203.stdout: metaop = 0 (0.00%) 2021-01-06T22:05:33.150 INFO:tasks.workunit.client.0.smithi203.stdout: createdir = 0 (0.00%) 2021-01-06T22:05:33.150 INFO:tasks.workunit.client.0.smithi203.stdout: stat = 0 (0.00%) 2021-01-06T22:05:33.150 INFO:tasks.workunit.client.0.smithi203.stdout: writeall = 0 (0.00%) 2021-01-06T22:05:33.151 INFO:tasks.workunit.client.0.smithi203.stdout: writeall_fsync = 0 (0.00%) 2021-01-06T22:05:33.151 INFO:tasks.workunit.client.0.smithi203.stdout: open_close = 0 (0.00%) 2021-01-06T22:05:33.151 INFO:tasks.workunit.client.0.smithi203.stdout: write_fsync = 0 (0.00%) 2021-01-06T22:05:33.151 INFO:tasks.workunit.client.0.smithi203.stdout: create_fsync = 0 (0.00%) 2021-01-06T22:05:33.151 INFO:tasks.workunit.client.0.smithi203.stdout: append_fsync = 0 (0.00%) 2021-01-06T22:05:33.152 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:33.152 INFO:tasks.workunit.client.0.smithi203.stdout:FileSystem . 2021-01-06T22:05:33.152 INFO:tasks.workunit.client.0.smithi203.stdout:========== 2021-01-06T22:05:33.152 INFO:tasks.workunit.client.0.smithi203.stdout: num_dirs = 0 2021-01-06T22:05:33.153 INFO:tasks.workunit.client.0.smithi203.stdout: starting files = 128 2021-01-06T22:05:33.153 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:33.153 INFO:tasks.workunit.client.0.smithi203.stdout: min file size = 104857600 (100MB) 2021-01-06T22:05:33.153 INFO:tasks.workunit.client.0.smithi203.stdout: max file size = 104857600 (100MB) 2021-01-06T22:05:33.153 INFO:tasks.workunit.client.0.smithi203.stdout: directio = off 2021-01-06T22:05:33.154 INFO:tasks.workunit.client.0.smithi203.stdout: alignedio = on 2021-01-06T22:05:33.154 INFO:tasks.workunit.client.0.smithi203.stdout: bufferedio = off 2021-01-06T22:05:33.154 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:33.154 INFO:tasks.workunit.client.0.smithi203.stdout: aging is off 2021-01-06T22:05:33.155 INFO:tasks.workunit.client.0.smithi203.stdout: current utilization = 0.00% 2021-01-06T22:05:33.155 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:05:33.156 INFO:tasks.workunit.client.0.smithi203.stdout:checking existing fs: . 2021-01-06T22:05:33.156 INFO:tasks.workunit.client.0.smithi203.stdout:recreating new fileset 2021-01-06T22:05:33.156 INFO:tasks.workunit.client.0.smithi203.stderr:opendir: No such file or directory 2021-01-06T22:05:37.308 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:05:37.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.431+0000 7f59e436e700 1 Processor -- start 2021-01-06T22:05:37.432 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.431+0000 7f59e436e700 1 -- start start 2021-01-06T22:05:37.433 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e436e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f59dc0c08a0 0x7f59dc147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:37.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e436e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59dc0bb860 con 0x7f59dc1481b0 2021-01-06T22:05:37.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e436e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59dc14b940 con 0x7f59dc0c04c0 2021-01-06T22:05:37.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e436e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59dc0673e0 con 0x7f59dc0c08a0 2021-01-06T22:05:37.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e210a700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f59dc0c04c0 0x7f59dc066cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48494/0 (socket says 172.21.15.203:48494) 2021-01-06T22:05:37.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e210a700 1 -- 172.21.15.203:0/298680304 learned_addr learned my addr 172.21.15.203:0/298680304 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:05:37.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e1909700 1 --2- 172.21.15.203:0/298680304 >> v2:172.21.15.91:3300/0 conn(0x7f59dc0c08a0 0x7f59dc147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:37.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e1108700 1 -- 172.21.15.203:0/298680304 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344671624 0 0) 0x7f59dc14b940 con 0x7f59dc0c04c0 2021-01-06T22:05:37.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e1108700 1 -- 172.21.15.203:0/298680304 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59c4003230 con 0x7f59dc0c04c0 2021-01-06T22:05:37.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e1909700 1 -- 172.21.15.203:0/298680304 >> v1:172.21.15.203:6789/0 conn(0x7f59dc0c04c0 legacy=0x7f59dc066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:37.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e1909700 1 -- 172.21.15.203:0/298680304 >> v1:172.21.15.91:6789/0 conn(0x7f59dc1481b0 legacy=0x7f59dc14ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:37.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e1909700 1 -- 172.21.15.203:0/298680304 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc0bba90 con 0x7f59dc0c08a0 2021-01-06T22:05:37.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e1909700 1 --2- 172.21.15.203:0/298680304 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59dc0c08a0 0x7f59dc147cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f59d0009c90 tx=0x7f59d00091d0).ready entity=mon.0 client_cookie=62e60ca04ef733f0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:37.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e1108700 1 -- 172.21.15.203:0/298680304 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d000ba70 con 0x7f59dc0c08a0 2021-01-06T22:05:37.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e1108700 1 -- 172.21.15.203:0/298680304 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d000bbd0 con 0x7f59dc0c08a0 2021-01-06T22:05:37.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e1108700 1 -- 172.21.15.203:0/298680304 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d0011400 con 0x7f59dc0c08a0 2021-01-06T22:05:37.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e436e700 1 -- 172.21.15.203:0/298680304 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59dc0c08a0 msgr2=0x7f59dc147cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:37.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e436e700 1 --2- 172.21.15.203:0/298680304 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59dc0c08a0 0x7f59dc147cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f59d0009c90 tx=0x7f59d00091d0).stop 2021-01-06T22:05:37.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e436e700 1 -- 172.21.15.203:0/298680304 shutdown_connections 2021-01-06T22:05:37.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e436e700 1 --2- 172.21.15.203:0/298680304 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59dc0c08a0 0x7f59dc147cc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:37.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e436e700 1 -- 172.21.15.203:0/298680304 >> 172.21.15.203:0/298680304 conn(0x7f59dc056cd0 msgr2=0x7f59dc0626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:37.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.435+0000 7f59e436e700 1 -- 172.21.15.203:0/298680304 shutdown_connections 2021-01-06T22:05:37.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.439+0000 7f59e436e700 1 -- 172.21.15.203:0/298680304 wait complete. 2021-01-06T22:05:37.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.439+0000 7f59e436e700 1 Processor -- start 2021-01-06T22:05:37.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.439+0000 7f59e436e700 1 -- start start 2021-01-06T22:05:37.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e436e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59dc0c04c0 0x7f59dc0b8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:37.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e436e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59dc1481b0 0x7f59dc14d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:37.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e436e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59dc13ff70 0x7f59dc14daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:37.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e436e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc141a50 con 0x7f59dc13ff70 2021-01-06T22:05:37.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e436e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc141cf0 con 0x7f59dc1481b0 2021-01-06T22:05:37.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e436e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc15d7d0 con 0x7f59dc0c04c0 2021-01-06T22:05:37.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e210a700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59dc0c04c0 0x7f59dc0b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:37.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e290b700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59dc13ff70 0x7f59dc14daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:37.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e1909700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59dc1481b0 0x7f59dc14d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:37.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e210a700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59dc0c04c0 0x7f59dc0b8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:54988/0 (socket says 172.21.15.203:54988) 2021-01-06T22:05:37.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e290b700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59dc13ff70 0x7f59dc14daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:57756/0 (socket says 172.21.15.203:57756) 2021-01-06T22:05:37.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e210a700 1 -- 172.21.15.203:0/3921347722 learned_addr learned my addr 172.21.15.203:0/3921347722 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:37.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e210a700 1 -- 172.21.15.203:0/3921347722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59dc1481b0 msgr2=0x7f59dc14d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:37.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e210a700 1 --2- 172.21.15.203:0/3921347722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59dc1481b0 0x7f59dc14d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:37.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e210a700 1 -- 172.21.15.203:0/3921347722 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59dc13ff70 msgr2=0x7f59dc14daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:37.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e210a700 1 --2- 172.21.15.203:0/3921347722 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59dc13ff70 0x7f59dc14daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:37.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e210a700 1 -- 172.21.15.203:0/3921347722 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc15da70 con 0x7f59dc0c04c0 2021-01-06T22:05:37.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e290b700 1 --2- 172.21.15.203:0/3921347722 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59dc13ff70 0x7f59dc14daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:05:37.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e1909700 1 --2- 172.21.15.203:0/3921347722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59dc1481b0 0x7f59dc14d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:05:37.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e210a700 1 --2- 172.21.15.203:0/3921347722 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59dc0c04c0 0x7f59dc0b8620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f59d400c6d0 tx=0x7f59d4007ac0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:37.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59ce7fc700 1 -- 172.21.15.203:0/3921347722 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40072e0 con 0x7f59dc0c04c0 2021-01-06T22:05:37.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59ce7fc700 1 -- 172.21.15.203:0/3921347722 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d4007440 con 0x7f59dc0c04c0 2021-01-06T22:05:37.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e436e700 1 -- 172.21.15.203:0/3921347722 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59dc15dd10 con 0x7f59dc0c04c0 2021-01-06T22:05:37.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59ce7fc700 1 -- 172.21.15.203:0/3921347722 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40057f0 con 0x7f59dc0c04c0 2021-01-06T22:05:37.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.443+0000 7f59e436e700 1 -- 172.21.15.203:0/3921347722 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59dc14df90 con 0x7f59dc0c04c0 2021-01-06T22:05:37.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.447+0000 7f59e436e700 1 -- 172.21.15.203:0/3921347722 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59dc05cfe0 con 0x7f59dc0c04c0 2021-01-06T22:05:37.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.447+0000 7f59ce7fc700 1 -- 172.21.15.203:0/3921347722 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f59d401a060 con 0x7f59dc0c04c0 2021-01-06T22:05:37.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.447+0000 7f59ce7fc700 1 --2- 172.21.15.203:0/3921347722 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f59bc0563d0 0x7f59bc058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:37.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.447+0000 7f59ce7fc700 1 -- 172.21.15.203:0/3921347722 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f59d406ed00 con 0x7f59dc0c04c0 2021-01-06T22:05:37.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.447+0000 7f59e1909700 1 --2- 172.21.15.203:0/3921347722 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f59bc0563d0 0x7f59bc058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:37.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.447+0000 7f59e1909700 1 --2- 172.21.15.203:0/3921347722 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f59bc0563d0 0x7f59bc058890 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f59dc0c1520 tx=0x7f59d0017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:37.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.459+0000 7f59ce7fc700 1 -- 172.21.15.203:0/3921347722 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f59d4072030 con 0x7f59dc0c04c0 2021-01-06T22:05:37.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.683+0000 7f59e436e700 1 -- 172.21.15.203:0/3921347722 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59dc0576d0 con 0x7f59dc0c04c0 2021-01-06T22:05:37.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.683+0000 7f59ce7fc700 1 -- 172.21.15.203:0/3921347722 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3578 (secure 0 0 0) 0x7f59d4018070 con 0x7f59dc0c04c0 2021-01-06T22:05:37.683 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:05:37.683 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4867,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":18}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:05:31.368778+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":2,"incarnation":13,"state":"up:active","state_seq":18,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:05:37.685 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.687+0000 7f59e436e700 1 -- 172.21.15.203:0/3921347722 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f59bc0563d0 msgr2=0x7f59bc058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:37.686 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.687+0000 7f59e436e700 1 --2- 172.21.15.203:0/3921347722 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f59bc0563d0 0x7f59bc058890 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f59dc0c1520 tx=0x7f59d0017000).stop 2021-01-06T22:05:37.686 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.687+0000 7f59e436e700 1 -- 172.21.15.203:0/3921347722 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59dc0c04c0 msgr2=0x7f59dc0b8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:37.686 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.687+0000 7f59e436e700 1 --2- 172.21.15.203:0/3921347722 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59dc0c04c0 0x7f59dc0b8620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f59d400c6d0 tx=0x7f59d4007ac0).stop 2021-01-06T22:05:37.686 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.687+0000 7f59e436e700 1 -- 172.21.15.203:0/3921347722 shutdown_connections 2021-01-06T22:05:37.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.687+0000 7f59e436e700 1 --2- 172.21.15.203:0/3921347722 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f59bc0563d0 0x7f59bc058890 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:37.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.687+0000 7f59e436e700 1 --2- 172.21.15.203:0/3921347722 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59dc13ff70 0x7f59dc14daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:37.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.687+0000 7f59e436e700 1 --2- 172.21.15.203:0/3921347722 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59dc0c04c0 0x7f59dc0b8620 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:37.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.687+0000 7f59e436e700 1 --2- 172.21.15.203:0/3921347722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59dc1481b0 0x7f59dc14d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:37.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.687+0000 7f59e436e700 1 -- 172.21.15.203:0/3921347722 >> 172.21.15.203:0/3921347722 conn(0x7f59dc056cd0 msgr2=0x7f59dc0602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:37.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.687+0000 7f59e436e700 1 -- 172.21.15.203:0/3921347722 shutdown_connections 2021-01-06T22:05:37.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.687+0000 7f59e436e700 1 -- 172.21.15.203:0/3921347722 wait complete. 2021-01-06T22:05:37.688 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 19 2021-01-06T22:05:37.699 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=2) 2021-01-06T22:05:37.700 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-01-06T22:05:37.700 INFO:teuthology.orchestra.run:waiting for 300 2021-01-06T22:05:37.714 INFO:tasks.ceph.mds.b:Stopped 2021-01-06T22:05:37.714 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:05:37.835 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.835+0000 7ff09759e700 1 Processor -- start 2021-01-06T22:05:37.835 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.835+0000 7ff09759e700 1 -- start start 2021-01-06T22:05:37.837 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09759e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0980c03f0 0x7ff0980aa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:37.837 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09759e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff0980c13c0 0x7ff098150110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:37.837 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09759e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0980a9ca0 con 0x7ff0980c0fe0 2021-01-06T22:05:37.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09759e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff098066a50 con 0x7ff0980c13c0 2021-01-06T22:05:37.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09759e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff098066cf0 con 0x7ff0980c03f0 2021-01-06T22:05:37.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff096d9d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff0980c13c0 0x7ff098150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:37.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09659c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7ff0980c0fe0 0x7ff098060d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48508/0 (socket says 172.21.15.203:48508) 2021-01-06T22:05:37.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09659c700 1 -- 172.21.15.203:0/3730727626 learned_addr learned my addr 172.21.15.203:0/3730727626 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:05:37.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff095d9b700 1 --2- 172.21.15.203:0/3730727626 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0980c03f0 0x7ff0980aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:37.842 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff096d9d700 1 -- 172.21.15.203:0/3730727626 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0980c03f0 msgr2=0x7ff0980aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:05:37.842 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff096d9d700 1 --2- 172.21.15.203:0/3730727626 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0980c03f0 0x7ff0980aa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:37.843 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff096d9d700 1 -- 172.21.15.203:0/3730727626 >> v1:172.21.15.203:6789/0 conn(0x7ff0980c0fe0 legacy=0x7ff098060d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:37.843 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09559a700 1 -- 172.21.15.203:0/3730727626 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003830332 0 0) 0x7ff0980a9ca0 con 0x7ff0980c0fe0 2021-01-06T22:05:37.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff096d9d700 1 -- 172.21.15.203:0/3730727626 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff098068460 con 0x7ff0980c13c0 2021-01-06T22:05:37.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff096d9d700 1 --2- 172.21.15.203:0/3730727626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0980c13c0 0x7ff098150110 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7ff090009c90 tx=0x7ff0900091d0).ready entity=mon.1 client_cookie=33a71c8d71eb61af server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:37.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09559a700 1 -- 172.21.15.203:0/3730727626 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09000bbc0 con 0x7ff0980c13c0 2021-01-06T22:05:37.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09559a700 1 -- 172.21.15.203:0/3730727626 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09000bd20 con 0x7ff0980c13c0 2021-01-06T22:05:37.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09559a700 1 -- 172.21.15.203:0/3730727626 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090011610 con 0x7ff0980c13c0 2021-01-06T22:05:37.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09759e700 1 -- 172.21.15.203:0/3730727626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0980c13c0 msgr2=0x7ff098150110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:37.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09759e700 1 --2- 172.21.15.203:0/3730727626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0980c13c0 0x7ff098150110 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7ff090009c90 tx=0x7ff0900091d0).stop 2021-01-06T22:05:37.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09759e700 1 -- 172.21.15.203:0/3730727626 shutdown_connections 2021-01-06T22:05:37.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09759e700 1 --2- 172.21.15.203:0/3730727626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0980c13c0 0x7ff098150110 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:37.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09759e700 1 --2- 172.21.15.203:0/3730727626 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0980c03f0 0x7ff0980aa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:37.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09759e700 1 -- 172.21.15.203:0/3730727626 >> 172.21.15.203:0/3730727626 conn(0x7ff098061530 msgr2=0x7ff098061910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:37.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09759e700 1 -- 172.21.15.203:0/3730727626 shutdown_connections 2021-01-06T22:05:37.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09759e700 1 -- 172.21.15.203:0/3730727626 wait complete. 2021-01-06T22:05:37.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09759e700 1 Processor -- start 2021-01-06T22:05:37.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.839+0000 7ff09759e700 1 -- start start 2021-01-06T22:05:37.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff09759e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0980c03f0 0x7ff0981611a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:37.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff09759e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0980c0fe0 0x7ff09815e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:37.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff09759e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0980c13c0 0x7ff09814f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:37.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff09759e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff098161690 con 0x7ff0980c0fe0 2021-01-06T22:05:37.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff09759e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0980a3230 con 0x7ff0980c13c0 2021-01-06T22:05:37.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff09759e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0980a34d0 con 0x7ff0980c03f0 2021-01-06T22:05:37.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff096d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0980c13c0 0x7ff09814f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:37.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff096d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0980c13c0 0x7ff09814f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49924/0 (socket says 172.21.15.203:49924) 2021-01-06T22:05:37.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff096d9d700 1 -- 172.21.15.203:0/1189669171 learned_addr learned my addr 172.21.15.203:0/1189669171 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:37.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff09659c700 1 --2- 172.21.15.203:0/1189669171 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0980c03f0 0x7ff0981611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:37.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff095d9b700 1 --2- 172.21.15.203:0/1189669171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0980c0fe0 0x7ff09815e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:37.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff096d9d700 1 -- 172.21.15.203:0/1189669171 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0980c03f0 msgr2=0x7ff0981611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:37.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff096d9d700 1 --2- 172.21.15.203:0/1189669171 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0980c03f0 0x7ff0981611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:37.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff096d9d700 1 -- 172.21.15.203:0/1189669171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0980c0fe0 msgr2=0x7ff09815e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:37.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff096d9d700 1 --2- 172.21.15.203:0/1189669171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0980c0fe0 0x7ff09815e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:37.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff096d9d700 1 -- 172.21.15.203:0/1189669171 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09815d240 con 0x7ff0980c13c0 2021-01-06T22:05:37.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff096d9d700 1 --2- 172.21.15.203:0/1189669171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0980c13c0 0x7ff09814f170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7ff090011000 tx=0x7ff090017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:37.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff0867fc700 1 -- 172.21.15.203:0/1189669171 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090011e60 con 0x7ff0980c13c0 2021-01-06T22:05:37.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff09759e700 1 -- 172.21.15.203:0/1189669171 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09815d4e0 con 0x7ff0980c13c0 2021-01-06T22:05:37.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff0867fc700 1 -- 172.21.15.203:0/1189669171 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09000be90 con 0x7ff0980c13c0 2021-01-06T22:05:37.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff0867fc700 1 -- 172.21.15.203:0/1189669171 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090010d40 con 0x7ff0980c13c0 2021-01-06T22:05:37.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.843+0000 7ff09759e700 1 -- 172.21.15.203:0/1189669171 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09815da20 con 0x7ff0980c13c0 2021-01-06T22:05:37.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.847+0000 7ff0867fc700 1 -- 172.21.15.203:0/1189669171 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff09002d070 con 0x7ff0980c13c0 2021-01-06T22:05:37.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.847+0000 7ff09759e700 1 -- 172.21.15.203:0/1189669171 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09815dbe0 con 0x7ff0980c13c0 2021-01-06T22:05:37.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.847+0000 7ff0867fc700 1 --2- 172.21.15.203:0/1189669171 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0740563d0 0x7ff074058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:37.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.847+0000 7ff0867fc700 1 -- 172.21.15.203:0/1189669171 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff090071ee0 con 0x7ff0980c13c0 2021-01-06T22:05:37.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.847+0000 7ff09659c700 1 --2- 172.21.15.203:0/1189669171 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0740563d0 0x7ff074058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:37.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.847+0000 7ff09659c700 1 --2- 172.21.15.203:0/1189669171 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0740563d0 0x7ff074058890 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7ff088006770 tx=0x7ff088007c10).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:37.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:37.851+0000 7ff0867fc700 1 -- 172.21.15.203:0/1189669171 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff090047710 con 0x7ff0980c13c0 2021-01-06T22:05:38.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:38.103+0000 7ff09759e700 1 -- 172.21.15.203:0/1189669171 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff09805ca60 con 0x7ff0980c13c0 2021-01-06T22:05:38.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:38.103+0000 7ff0867fc700 1 -- 172.21.15.203:0/1189669171 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3578 (secure 0 0 0) 0x7ff090021070 con 0x7ff0980c13c0 2021-01-06T22:05:38.101 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:05:38.102 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4867,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":18}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:05:31.368778+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":2,"incarnation":13,"state":"up:active","state_seq":18,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:05:38.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:38.103+0000 7ff09759e700 1 -- 172.21.15.203:0/1189669171 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0740563d0 msgr2=0x7ff074058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:38.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:38.103+0000 7ff09759e700 1 --2- 172.21.15.203:0/1189669171 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0740563d0 0x7ff074058890 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7ff088006770 tx=0x7ff088007c10).stop 2021-01-06T22:05:38.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:38.103+0000 7ff09759e700 1 -- 172.21.15.203:0/1189669171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0980c13c0 msgr2=0x7ff09814f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:38.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:38.103+0000 7ff09759e700 1 --2- 172.21.15.203:0/1189669171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0980c13c0 0x7ff09814f170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7ff090011000 tx=0x7ff090017000).stop 2021-01-06T22:05:38.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:38.103+0000 7ff09759e700 1 -- 172.21.15.203:0/1189669171 shutdown_connections 2021-01-06T22:05:38.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:38.103+0000 7ff09759e700 1 --2- 172.21.15.203:0/1189669171 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0740563d0 0x7ff074058890 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:38.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:38.103+0000 7ff09759e700 1 --2- 172.21.15.203:0/1189669171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0980c13c0 0x7ff09814f170 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:38.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:38.103+0000 7ff09759e700 1 --2- 172.21.15.203:0/1189669171 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0980c03f0 0x7ff0981611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:38.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:38.103+0000 7ff09759e700 1 --2- 172.21.15.203:0/1189669171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0980c0fe0 0x7ff09815e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:38.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:38.103+0000 7ff09759e700 1 -- 172.21.15.203:0/1189669171 >> 172.21.15.203:0/1189669171 conn(0x7ff098061530 msgr2=0x7ff098061910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:38.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:38.103+0000 7ff09759e700 1 -- 172.21.15.203:0/1189669171 shutdown_connections 2021-01-06T22:05:38.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:38.103+0000 7ff09759e700 1 -- 172.21.15.203:0/1189669171 wait complete. 2021-01-06T22:05:38.105 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 19 2021-01-06T22:05:38.114 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2021-01-06T22:05:40.115 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:05:40.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.243+0000 7faab5a12700 1 Processor -- start 2021-01-06T22:05:40.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.243+0000 7faab5a12700 1 -- start start 2021-01-06T22:05:40.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faab5a12700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faab0067ab0 0x7faab00682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:40.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faab5a12700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faab00b4610 0x7faab00ba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:40.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faab5a12700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab014dc10 con 0x7faab00676d0 2021-01-06T22:05:40.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faab5a12700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faab00b8a50 con 0x7faab00b4610 2021-01-06T22:05:40.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faab5a12700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab00b8cf0 con 0x7faab0067ab0 2021-01-06T22:05:40.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faaa7fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faab0067ab0 0x7faab00682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:40.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faaaf7fe700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faab00b4610 0x7faab00ba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:40.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faaaf7fe700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faab00b4610 0x7faab00ba6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:57780/0 (socket says 172.21.15.203:57780) 2021-01-06T22:05:40.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faaaf7fe700 1 -- 172.21.15.203:0/1973077428 learned_addr learned my addr 172.21.15.203:0/1973077428 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:40.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faaa7fff700 1 -- 172.21.15.203:0/1973077428 >> v2:172.21.15.91:3300/0 conn(0x7faab00b4610 msgr2=0x7faab00ba6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:05:40.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faaa7fff700 1 --2- 172.21.15.203:0/1973077428 >> v2:172.21.15.91:3300/0 conn(0x7faab00b4610 0x7faab00ba6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:40.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faaa7fff700 1 -- 172.21.15.203:0/1973077428 >> v1:172.21.15.91:6789/0 conn(0x7faab00676d0 legacy=0x7faab014e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:40.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faaa7fff700 1 -- 172.21.15.203:0/1973077428 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab00b8f90 con 0x7faab0067ab0 2021-01-06T22:05:40.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faaaf7fe700 1 --2- 172.21.15.203:0/1973077428 >> v2:172.21.15.91:3300/0 conn(0x7faab00b4610 0x7faab00ba6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:05:40.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faaa7fff700 1 --2- 172.21.15.203:0/1973077428 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faab0067ab0 0x7faab00682c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7faa9800b980 tx=0x7faa98006de0).ready entity=mon.2 client_cookie=5aa090dddc8cddbb server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:40.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faaae7fc700 1 -- 172.21.15.203:0/1973077428 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98009840 con 0x7faab0067ab0 2021-01-06T22:05:40.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faaae7fc700 1 -- 172.21.15.203:0/1973077428 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa980099a0 con 0x7faab0067ab0 2021-01-06T22:05:40.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faaae7fc700 1 -- 172.21.15.203:0/1973077428 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98009d20 con 0x7faab0067ab0 2021-01-06T22:05:40.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faab5a12700 1 -- 172.21.15.203:0/1973077428 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faab0067ab0 msgr2=0x7faab00682c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:40.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faab5a12700 1 --2- 172.21.15.203:0/1973077428 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faab0067ab0 0x7faab00682c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7faa9800b980 tx=0x7faa98006de0).stop 2021-01-06T22:05:40.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faab5a12700 1 -- 172.21.15.203:0/1973077428 shutdown_connections 2021-01-06T22:05:40.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faab5a12700 1 --2- 172.21.15.203:0/1973077428 >> v2:172.21.15.91:3300/0 conn(0x7faab00b4610 0x7faab00ba6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:40.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faab5a12700 1 --2- 172.21.15.203:0/1973077428 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faab0067ab0 0x7faab00682c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:40.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faab5a12700 1 -- 172.21.15.203:0/1973077428 >> 172.21.15.203:0/1973077428 conn(0x7faab00c0cb0 msgr2=0x7faab00c1090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:40.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faab5a12700 1 -- 172.21.15.203:0/1973077428 shutdown_connections 2021-01-06T22:05:40.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.247+0000 7faab5a12700 1 -- 172.21.15.203:0/1973077428 wait complete. 2021-01-06T22:05:40.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faab5a12700 1 Processor -- start 2021-01-06T22:05:40.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faab5a12700 1 -- start start 2021-01-06T22:05:40.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faab5a12700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faab00676d0 0x7faab0147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:40.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faab5a12700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab0067ab0 0x7faab00bf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:40.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faab5a12700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faab00b4610 0x7faab013db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:40.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faab5a12700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab00bf4d0 con 0x7faab00676d0 2021-01-06T22:05:40.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faab5a12700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab00bf240 con 0x7faab0067ab0 2021-01-06T22:05:40.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faab5a12700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab0148fa0 con 0x7faab00b4610 2021-01-06T22:05:40.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faaa7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab0067ab0 0x7faab00bf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:40.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faaa7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab0067ab0 0x7faab00bf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49936/0 (socket says 172.21.15.203:49936) 2021-01-06T22:05:40.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faaa7fff700 1 -- 172.21.15.203:0/785136434 learned_addr learned my addr 172.21.15.203:0/785136434 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:40.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faaaeffd700 1 --2- 172.21.15.203:0/785136434 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faab00676d0 0x7faab0147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:40.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faaaf7fe700 1 --2- 172.21.15.203:0/785136434 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faab00b4610 0x7faab013db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:40.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faaa7fff700 1 -- 172.21.15.203:0/785136434 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faab00b4610 msgr2=0x7faab013db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:40.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faaa7fff700 1 --2- 172.21.15.203:0/785136434 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faab00b4610 0x7faab013db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:40.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faaa7fff700 1 -- 172.21.15.203:0/785136434 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faab00676d0 msgr2=0x7faab0147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:40.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faaa7fff700 1 --2- 172.21.15.203:0/785136434 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faab00676d0 0x7faab0147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:40.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faaa7fff700 1 -- 172.21.15.203:0/785136434 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab00befa0 con 0x7faab0067ab0 2021-01-06T22:05:40.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faaa7fff700 1 --2- 172.21.15.203:0/785136434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab0067ab0 0x7faab00bf8c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7faa98006e20 tx=0x7faa98016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:40.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faaa6ffd700 1 -- 172.21.15.203:0/785136434 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98012d40 con 0x7faab0067ab0 2021-01-06T22:05:40.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faaa6ffd700 1 -- 172.21.15.203:0/785136434 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9801c420 con 0x7faab0067ab0 2021-01-06T22:05:40.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faab5a12700 1 -- 172.21.15.203:0/785136434 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab0149200 con 0x7faab0067ab0 2021-01-06T22:05:40.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faab5a12700 1 -- 172.21.15.203:0/785136434 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab014b830 con 0x7faab0067ab0 2021-01-06T22:05:40.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.251+0000 7faaa6ffd700 1 -- 172.21.15.203:0/785136434 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98012ea0 con 0x7faab0067ab0 2021-01-06T22:05:40.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.255+0000 7faab5a12700 1 -- 172.21.15.203:0/785136434 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab005d600 con 0x7faab0067ab0 2021-01-06T22:05:40.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.255+0000 7faaa6ffd700 1 -- 172.21.15.203:0/785136434 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7faa9800f070 con 0x7faab0067ab0 2021-01-06T22:05:40.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.255+0000 7faaa6ffd700 1 --2- 172.21.15.203:0/785136434 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faa940563d0 0x7faa94058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:40.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.255+0000 7faaa6ffd700 1 -- 172.21.15.203:0/785136434 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6159+0+0 (secure 0 0 0) 0x7faa98070970 con 0x7faab0067ab0 2021-01-06T22:05:40.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.255+0000 7faaaeffd700 1 --2- 172.21.15.203:0/785136434 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faa940563d0 0x7faa94058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:40.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.255+0000 7faaaeffd700 1 --2- 172.21.15.203:0/785136434 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faa940563d0 0x7faa94058890 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7faaa00035f0 tx=0x7faaa0007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:40.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.259+0000 7faaa6ffd700 1 -- 172.21.15.203:0/785136434 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7faa980461a0 con 0x7faab0067ab0 2021-01-06T22:05:40.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.483+0000 7faab5a12700 1 -- 172.21.15.203:0/785136434 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faab00b4140 con 0x7faab0067ab0 2021-01-06T22:05:40.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.483+0000 7faaa6ffd700 1 -- 172.21.15.203:0/785136434 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3578 (secure 0 0 0) 0x7faa980497c0 con 0x7faab0067ab0 2021-01-06T22:05:40.483 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:05:40.484 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4867,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":18}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:05:31.368778+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":2,"incarnation":13,"state":"up:active","state_seq":18,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:05:40.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.487+0000 7faab5a12700 1 -- 172.21.15.203:0/785136434 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faa940563d0 msgr2=0x7faa94058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:40.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.487+0000 7faab5a12700 1 --2- 172.21.15.203:0/785136434 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faa940563d0 0x7faa94058890 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7faaa00035f0 tx=0x7faaa0007b80).stop 2021-01-06T22:05:40.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.487+0000 7faab5a12700 1 -- 172.21.15.203:0/785136434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab0067ab0 msgr2=0x7faab00bf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:40.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.487+0000 7faab5a12700 1 --2- 172.21.15.203:0/785136434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab0067ab0 0x7faab00bf8c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7faa98006e20 tx=0x7faa98016000).stop 2021-01-06T22:05:40.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.487+0000 7faab5a12700 1 -- 172.21.15.203:0/785136434 shutdown_connections 2021-01-06T22:05:40.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.487+0000 7faab5a12700 1 --2- 172.21.15.203:0/785136434 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faa940563d0 0x7faa94058890 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:40.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.487+0000 7faab5a12700 1 --2- 172.21.15.203:0/785136434 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faab00b4610 0x7faab013db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:40.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.487+0000 7faab5a12700 1 --2- 172.21.15.203:0/785136434 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faab00676d0 0x7faab0147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:40.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.487+0000 7faab5a12700 1 --2- 172.21.15.203:0/785136434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab0067ab0 0x7faab00bf8c0 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0x7faa98006e20 tx=0x7faa98016000).stop 2021-01-06T22:05:40.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.487+0000 7faab5a12700 1 -- 172.21.15.203:0/785136434 >> 172.21.15.203:0/785136434 conn(0x7faab00c0cb0 msgr2=0x7faab00bc140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:40.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.487+0000 7faab5a12700 1 -- 172.21.15.203:0/785136434 shutdown_connections 2021-01-06T22:05:40.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:40.487+0000 7faab5a12700 1 -- 172.21.15.203:0/785136434 wait complete. 2021-01-06T22:05:40.488 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 19 2021-01-06T22:05:40.498 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2021-01-06T22:05:41.337 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:05:41.340 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:05:42.501 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:05:42.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.635+0000 7fbeb2b88700 1 Processor -- start 2021-01-06T22:05:42.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.635+0000 7fbeb2b88700 1 -- start start 2021-01-06T22:05:42.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.635+0000 7fbeb2b88700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbeac1513f0 0x7fbeac0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:42.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.635+0000 7fbeb2b88700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbeac14ad90 0x7fbeac14b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:42.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.635+0000 7fbeb2b88700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeac0bacb0 con 0x7fbeac0bccb0 2021-01-06T22:05:42.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.635+0000 7fbeb2b88700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeac14b6f0 con 0x7fbeac14ad90 2021-01-06T22:05:42.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.635+0000 7fbeb2b88700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac0bd130 con 0x7fbeac1513f0 2021-01-06T22:05:42.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.635+0000 7fbeb2387700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbeac14ad90 0x7fbeac14b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:42.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.635+0000 7fbeb2387700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbeac14ad90 0x7fbeac14b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49958/0 (socket says 172.21.15.203:49958) 2021-01-06T22:05:42.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.635+0000 7fbeb2387700 1 -- 172.21.15.203:0/103026926 learned_addr learned my addr 172.21.15.203:0/103026926 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:42.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.635+0000 7fbeb1b86700 1 --2- 172.21.15.203:0/103026926 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbeac1513f0 0x7fbeac0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:42.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb2387700 1 -- 172.21.15.203:0/103026926 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbeac1513f0 msgr2=0x7fbeac0c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:05:42.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb2387700 1 --2- 172.21.15.203:0/103026926 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbeac1513f0 0x7fbeac0c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:42.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb2387700 1 -- 172.21.15.203:0/103026926 >> v1:172.21.15.91:6789/0 conn(0x7fbeac0bccb0 legacy=0x7fbeac1517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:42.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb2387700 1 -- 172.21.15.203:0/103026926 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac152000 con 0x7fbeac14ad90 2021-01-06T22:05:42.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb1b86700 1 --2- 172.21.15.203:0/103026926 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbeac1513f0 0x7fbeac0c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:05:42.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb2387700 1 --2- 172.21.15.203:0/103026926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeac14ad90 0x7fbeac14b200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fbea4009d80 tx=0x7fbea4009290).ready entity=mon.1 client_cookie=fc191bcab87ebe68 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:42.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb0b84700 1 -- 172.21.15.203:0/103026926 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea400bc40 con 0x7fbeac14ad90 2021-01-06T22:05:42.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb0b84700 1 -- 172.21.15.203:0/103026926 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea400bda0 con 0x7fbeac14ad90 2021-01-06T22:05:42.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb0b84700 1 -- 172.21.15.203:0/103026926 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4011610 con 0x7fbeac14ad90 2021-01-06T22:05:42.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb2b88700 1 -- 172.21.15.203:0/103026926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeac14ad90 msgr2=0x7fbeac14b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:42.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb2b88700 1 --2- 172.21.15.203:0/103026926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeac14ad90 0x7fbeac14b200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fbea4009d80 tx=0x7fbea4009290).stop 2021-01-06T22:05:42.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb2b88700 1 -- 172.21.15.203:0/103026926 shutdown_connections 2021-01-06T22:05:42.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb2b88700 1 --2- 172.21.15.203:0/103026926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeac14ad90 0x7fbeac14b200 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:42.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb2b88700 1 --2- 172.21.15.203:0/103026926 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbeac1513f0 0x7fbeac0c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:42.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb2b88700 1 -- 172.21.15.203:0/103026926 >> 172.21.15.203:0/103026926 conn(0x7fbeac0667b0 msgr2=0x7fbeac0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:42.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb2b88700 1 -- 172.21.15.203:0/103026926 shutdown_connections 2021-01-06T22:05:42.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb2b88700 1 -- 172.21.15.203:0/103026926 wait complete. 2021-01-06T22:05:42.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb2b88700 1 Processor -- start 2021-01-06T22:05:42.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.639+0000 7fbeb2b88700 1 -- start start 2021-01-06T22:05:42.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2b88700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbeac0bccb0 0x7fbeac142490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:42.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2b88700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbeac14ad90 0x7fbeac161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:42.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2b88700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeac1513f0 0x7fbeac149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:42.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2b88700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac151070 con 0x7fbeac14ad90 2021-01-06T22:05:42.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2b88700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac150dd0 con 0x7fbeac1513f0 2021-01-06T22:05:42.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2b88700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac164ea0 con 0x7fbeac0bccb0 2021-01-06T22:05:42.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2387700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeac1513f0 0x7fbeac149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:42.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2387700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeac1513f0 0x7fbeac149920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49964/0 (socket says 172.21.15.203:49964) 2021-01-06T22:05:42.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2387700 1 -- 172.21.15.203:0/2750741903 learned_addr learned my addr 172.21.15.203:0/2750741903 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:42.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb1b86700 1 --2- 172.21.15.203:0/2750741903 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbeac0bccb0 0x7fbeac142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:42.649 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb1385700 1 --2- 172.21.15.203:0/2750741903 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbeac14ad90 0x7fbeac161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:42.649 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2387700 1 -- 172.21.15.203:0/2750741903 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbeac0bccb0 msgr2=0x7fbeac142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:42.649 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2387700 1 --2- 172.21.15.203:0/2750741903 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbeac0bccb0 0x7fbeac142490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:42.649 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2387700 1 -- 172.21.15.203:0/2750741903 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbeac14ad90 msgr2=0x7fbeac161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:42.650 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2387700 1 --2- 172.21.15.203:0/2750741903 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbeac14ad90 0x7fbeac161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:42.650 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2387700 1 -- 172.21.15.203:0/2750741903 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac165140 con 0x7fbeac1513f0 2021-01-06T22:05:42.650 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2387700 1 --2- 172.21.15.203:0/2750741903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeac1513f0 0x7fbeac149920 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fbeac065e40 tx=0x7fbea4017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:42.650 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbe99ffb700 1 -- 172.21.15.203:0/2750741903 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea400bc40 con 0x7fbeac1513f0 2021-01-06T22:05:42.650 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbe99ffb700 1 -- 172.21.15.203:0/2750741903 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea4011e70 con 0x7fbeac1513f0 2021-01-06T22:05:42.651 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbe99ffb700 1 -- 172.21.15.203:0/2750741903 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4010c70 con 0x7fbeac1513f0 2021-01-06T22:05:42.651 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2b88700 1 -- 172.21.15.203:0/2750741903 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeac1653e0 con 0x7fbeac1513f0 2021-01-06T22:05:42.651 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2b88700 1 -- 172.21.15.203:0/2750741903 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeac165b20 con 0x7fbeac1513f0 2021-01-06T22:05:42.651 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.643+0000 7fbeb2b88700 1 -- 172.21.15.203:0/2750741903 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeac067d70 con 0x7fbeac1513f0 2021-01-06T22:05:42.652 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.647+0000 7fbe99ffb700 1 -- 172.21.15.203:0/2750741903 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fbea400f070 con 0x7fbeac1513f0 2021-01-06T22:05:42.652 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.647+0000 7fbe99ffb700 1 --2- 172.21.15.203:0/2750741903 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbe8c056420 0x7fbe8c0588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:42.652 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.647+0000 7fbe99ffb700 1 -- 172.21.15.203:0/2750741903 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fbea40714b0 con 0x7fbeac1513f0 2021-01-06T22:05:42.652 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.647+0000 7fbeb1b86700 1 --2- 172.21.15.203:0/2750741903 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbe8c056420 0x7fbe8c0588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:42.652 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.647+0000 7fbeb1b86700 1 --2- 172.21.15.203:0/2750741903 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbe8c056420 0x7fbe8c0588e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fbe9c00d9f0 tx=0x7fbe9c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:42.653 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.655+0000 7fbe99ffb700 1 -- 172.21.15.203:0/2750741903 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fbea4046d60 con 0x7fbeac1513f0 2021-01-06T22:05:42.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.871+0000 7fbeb2b88700 1 -- 172.21.15.203:0/2750741903 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbeac0450a0 con 0x7fbeac1513f0 2021-01-06T22:05:42.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.875+0000 7fbe99ffb700 1 -- 172.21.15.203:0/2750741903 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3578 (secure 0 0 0) 0x7fbea404a380 con 0x7fbeac1513f0 2021-01-06T22:05:42.875 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:05:42.876 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4867,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":18}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:05:31.368778+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":2,"incarnation":13,"state":"up:active","state_seq":18,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:05:42.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.875+0000 7fbeb2b88700 1 -- 172.21.15.203:0/2750741903 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbe8c056420 msgr2=0x7fbe8c0588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:42.878 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.875+0000 7fbeb2b88700 1 --2- 172.21.15.203:0/2750741903 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbe8c056420 0x7fbe8c0588e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fbe9c00d9f0 tx=0x7fbe9c008040).stop 2021-01-06T22:05:42.878 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.875+0000 7fbeb2b88700 1 -- 172.21.15.203:0/2750741903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeac1513f0 msgr2=0x7fbeac149920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:42.879 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.875+0000 7fbeb2b88700 1 --2- 172.21.15.203:0/2750741903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeac1513f0 0x7fbeac149920 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fbeac065e40 tx=0x7fbea4017000).stop 2021-01-06T22:05:42.879 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.875+0000 7fbeb2b88700 1 -- 172.21.15.203:0/2750741903 shutdown_connections 2021-01-06T22:05:42.879 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.875+0000 7fbeb2b88700 1 --2- 172.21.15.203:0/2750741903 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbe8c056420 0x7fbe8c0588e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:42.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.875+0000 7fbeb2b88700 1 --2- 172.21.15.203:0/2750741903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeac1513f0 0x7fbeac149920 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:42.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.875+0000 7fbeb2b88700 1 --2- 172.21.15.203:0/2750741903 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbeac0bccb0 0x7fbeac142490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:42.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.875+0000 7fbeb2b88700 1 --2- 172.21.15.203:0/2750741903 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbeac14ad90 0x7fbeac161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:42.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.875+0000 7fbeb2b88700 1 -- 172.21.15.203:0/2750741903 >> 172.21.15.203:0/2750741903 conn(0x7fbeac0667b0 msgr2=0x7fbeac0ba490 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:42.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.875+0000 7fbeb2b88700 1 -- 172.21.15.203:0/2750741903 shutdown_connections 2021-01-06T22:05:42.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:42.875+0000 7fbeb2b88700 1 -- 172.21.15.203:0/2750741903 wait complete. 2021-01-06T22:05:42.883 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 19 2021-01-06T22:05:42.889 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2021-01-06T22:05:44.890 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:05:45.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.019+0000 7f45c180b700 1 Processor -- start 2021-01-06T22:05:45.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.019+0000 7f45c180b700 1 -- start start 2021-01-06T22:05:45.026 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45c180b700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f45bc14eaa0 0x7f45bc063190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:45.026 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45c180b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f45bc05bf60 0x7f45bc063680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:45.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45c180b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45bc0b7720 con 0x7f45bc0bb7d0 2021-01-06T22:05:45.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45c180b700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45bc0bd320 con 0x7f45bc14eaa0 2021-01-06T22:05:45.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45c180b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45bc14f020 con 0x7f45bc05bf60 2021-01-06T22:05:45.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45bb7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f45bc05bf60 0x7f45bc063680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:45.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45bb7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f45bc05bf60 0x7f45bc063680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49982/0 (socket says 172.21.15.203:49982) 2021-01-06T22:05:45.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45baffd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f45bc0bb7d0 0x7f45bc0bbbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48572/0 (socket says 172.21.15.203:48572) 2021-01-06T22:05:45.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45bb7fe700 1 -- 172.21.15.203:0/267969008 learned_addr learned my addr 172.21.15.203:0/267969008 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:45.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45ba7fc700 1 --2- 172.21.15.203:0/267969008 >> v2:172.21.15.91:3300/0 conn(0x7f45bc14eaa0 0x7f45bc063190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:45.029 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45bb7fe700 1 -- 172.21.15.203:0/267969008 >> v2:172.21.15.91:3300/0 conn(0x7f45bc14eaa0 msgr2=0x7f45bc063190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:05:45.029 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45bb7fe700 1 --2- 172.21.15.203:0/267969008 >> v2:172.21.15.91:3300/0 conn(0x7f45bc14eaa0 0x7f45bc063190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:45.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45bb7fe700 1 -- 172.21.15.203:0/267969008 >> v1:172.21.15.203:6789/0 conn(0x7f45bc0bb7d0 legacy=0x7f45bc0bbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:45.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45bb7fe700 1 -- 172.21.15.203:0/267969008 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc0b79c0 con 0x7f45bc05bf60 2021-01-06T22:05:45.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45b9ffb700 1 -- 172.21.15.203:0/267969008 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4109292190 0 0) 0x7f45bc0b7720 con 0x7f45bc0bb7d0 2021-01-06T22:05:45.039 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45ba7fc700 1 --2- 172.21.15.203:0/267969008 >> v2:172.21.15.91:3300/0 conn(0x7f45bc14eaa0 0x7f45bc063190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:05:45.039 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45bb7fe700 1 --2- 172.21.15.203:0/267969008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45bc05bf60 0x7f45bc063680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f45bc063fd0 tx=0x7f45b4009290).ready entity=mon.1 client_cookie=95f37d3471767408 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:45.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45b9ffb700 1 -- 172.21.15.203:0/267969008 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400bbc0 con 0x7f45bc05bf60 2021-01-06T22:05:45.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45b9ffb700 1 -- 172.21.15.203:0/267969008 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b400bd20 con 0x7f45bc05bf60 2021-01-06T22:05:45.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45b9ffb700 1 -- 172.21.15.203:0/267969008 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4011610 con 0x7f45bc05bf60 2021-01-06T22:05:45.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45c180b700 1 -- 172.21.15.203:0/267969008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45bc05bf60 msgr2=0x7f45bc063680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:45.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45c180b700 1 --2- 172.21.15.203:0/267969008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45bc05bf60 0x7f45bc063680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f45bc063fd0 tx=0x7f45b4009290).stop 2021-01-06T22:05:45.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45c180b700 1 -- 172.21.15.203:0/267969008 shutdown_connections 2021-01-06T22:05:45.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45c180b700 1 --2- 172.21.15.203:0/267969008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45bc05bf60 0x7f45bc063680 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:45.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45c180b700 1 --2- 172.21.15.203:0/267969008 >> v2:172.21.15.91:3300/0 conn(0x7f45bc14eaa0 0x7f45bc063190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:45.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45c180b700 1 -- 172.21.15.203:0/267969008 >> 172.21.15.203:0/267969008 conn(0x7f45bc0c1760 msgr2=0x7f45bc0bd730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:45.042 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45c180b700 1 -- 172.21.15.203:0/267969008 shutdown_connections 2021-01-06T22:05:45.042 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.023+0000 7f45c180b700 1 -- 172.21.15.203:0/267969008 wait complete. 2021-01-06T22:05:45.044 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45c180b700 1 Processor -- start 2021-01-06T22:05:45.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45c180b700 1 -- start start 2021-01-06T22:05:45.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45c180b700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45bc05bf60 0x7f45bc14dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:45.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45c180b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45bc0bb7d0 0x7f45bc15e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:45.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45c180b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45bc14eaa0 0x7f45bc143210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:45.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45c180b700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc0b8eb0 con 0x7f45bc05bf60 2021-01-06T22:05:45.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45c180b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc0b90e0 con 0x7f45bc14eaa0 2021-01-06T22:05:45.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45c180b700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc0b9380 con 0x7f45bc0bb7d0 2021-01-06T22:05:45.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45bb7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45bc14eaa0 0x7f45bc143210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:45.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45baffd700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45bc05bf60 0x7f45bc14dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:45.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45bb7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45bc14eaa0 0x7f45bc143210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49988/0 (socket says 172.21.15.203:49988) 2021-01-06T22:05:45.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45bb7fe700 1 -- 172.21.15.203:0/2350402607 learned_addr learned my addr 172.21.15.203:0/2350402607 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:45.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45ba7fc700 1 --2- 172.21.15.203:0/2350402607 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45bc0bb7d0 0x7f45bc15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:45.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45bb7fe700 1 -- 172.21.15.203:0/2350402607 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45bc0bb7d0 msgr2=0x7f45bc15e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:45.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45bb7fe700 1 --2- 172.21.15.203:0/2350402607 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45bc0bb7d0 0x7f45bc15e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:45.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45bb7fe700 1 -- 172.21.15.203:0/2350402607 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45bc05bf60 msgr2=0x7f45bc14dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:45.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45bb7fe700 1 --2- 172.21.15.203:0/2350402607 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45bc05bf60 0x7f45bc14dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:45.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.027+0000 7f45bb7fe700 1 -- 172.21.15.203:0/2350402607 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc0b9620 con 0x7f45bc14eaa0 2021-01-06T22:05:45.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.031+0000 7f45bb7fe700 1 --2- 172.21.15.203:0/2350402607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45bc14eaa0 0x7f45bc143210 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f45bc14f160 tx=0x7f45b4017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:45.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.031+0000 7f45a2ffd700 1 -- 172.21.15.203:0/2350402607 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400be90 con 0x7f45bc14eaa0 2021-01-06T22:05:45.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.031+0000 7f45a2ffd700 1 -- 172.21.15.203:0/2350402607 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b4011e30 con 0x7f45bc14eaa0 2021-01-06T22:05:45.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.031+0000 7f45c180b700 1 -- 172.21.15.203:0/2350402607 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45bc0b9940 con 0x7f45bc14eaa0 2021-01-06T22:05:45.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.031+0000 7f45a2ffd700 1 -- 172.21.15.203:0/2350402607 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4010cd0 con 0x7f45bc14eaa0 2021-01-06T22:05:45.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.031+0000 7f45c180b700 1 -- 172.21.15.203:0/2350402607 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45bc0b9e80 con 0x7f45bc14eaa0 2021-01-06T22:05:45.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.031+0000 7f45c180b700 1 -- 172.21.15.203:0/2350402607 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45bc149400 con 0x7f45bc14eaa0 2021-01-06T22:05:45.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.031+0000 7f45a2ffd700 1 -- 172.21.15.203:0/2350402607 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f45b400f070 con 0x7f45bc14eaa0 2021-01-06T22:05:45.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.031+0000 7f45a2ffd700 1 --2- 172.21.15.203:0/2350402607 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f459c0563d0 0x7f459c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:45.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.031+0000 7f45a2ffd700 1 -- 172.21.15.203:0/2350402607 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f45b4071270 con 0x7f45bc14eaa0 2021-01-06T22:05:45.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.031+0000 7f45baffd700 1 --2- 172.21.15.203:0/2350402607 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f459c0563d0 0x7f459c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:45.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.031+0000 7f45baffd700 1 --2- 172.21.15.203:0/2350402607 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f459c0563d0 0x7f459c058890 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f45ac0011a0 tx=0x7f45ac007a10).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:45.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.039+0000 7f45a2ffd700 1 -- 172.21.15.203:0/2350402607 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f45bc149400 con 0x7f45bc14eaa0 2021-01-06T22:05:45.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.311+0000 7f45c180b700 1 -- 172.21.15.203:0/2350402607 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45bc149400 con 0x7f45bc14eaa0 2021-01-06T22:05:45.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.311+0000 7f45a2ffd700 1 -- 172.21.15.203:0/2350402607 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3578 (secure 0 0 0) 0x7f45bc149400 con 0x7f45bc14eaa0 2021-01-06T22:05:45.311 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:05:45.311 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4867,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":18}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:05:31.368778+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":2,"incarnation":13,"state":"up:active","state_seq":18,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:05:45.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.311+0000 7f45c180b700 1 -- 172.21.15.203:0/2350402607 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f459c0563d0 msgr2=0x7f459c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:45.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.311+0000 7f45c180b700 1 --2- 172.21.15.203:0/2350402607 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f459c0563d0 0x7f459c058890 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f45ac0011a0 tx=0x7f45ac007a10).stop 2021-01-06T22:05:45.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.311+0000 7f45c180b700 1 -- 172.21.15.203:0/2350402607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45bc14eaa0 msgr2=0x7f45bc143210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:45.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.311+0000 7f45c180b700 1 --2- 172.21.15.203:0/2350402607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45bc14eaa0 0x7f45bc143210 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f45bc14f160 tx=0x7f45b4017000).stop 2021-01-06T22:05:45.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.311+0000 7f45c180b700 1 -- 172.21.15.203:0/2350402607 shutdown_connections 2021-01-06T22:05:45.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.311+0000 7f45c180b700 1 --2- 172.21.15.203:0/2350402607 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f459c0563d0 0x7f459c058890 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:45.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.315+0000 7f45c180b700 1 --2- 172.21.15.203:0/2350402607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45bc14eaa0 0x7f45bc143210 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:45.317 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.315+0000 7f45c180b700 1 --2- 172.21.15.203:0/2350402607 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45bc05bf60 0x7f45bc14dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:45.317 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.315+0000 7f45c180b700 1 --2- 172.21.15.203:0/2350402607 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45bc0bb7d0 0x7f45bc15e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:45.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.315+0000 7f45c180b700 1 -- 172.21.15.203:0/2350402607 >> 172.21.15.203:0/2350402607 conn(0x7f45bc0c1760 msgr2=0x7f45bc145a30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:45.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.315+0000 7f45c180b700 1 -- 172.21.15.203:0/2350402607 shutdown_connections 2021-01-06T22:05:45.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:45.315+0000 7f45c180b700 1 -- 172.21.15.203:0/2350402607 wait complete. 2021-01-06T22:05:45.321 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 19 2021-01-06T22:05:45.329 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2021-01-06T22:05:47.329 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:05:47.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.467+0000 7f15cadc7700 1 Processor -- start 2021-01-06T22:05:47.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.467+0000 7f15cadc7700 1 -- start start 2021-01-06T22:05:47.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15cadc7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f15c414e310 0x7f15c41471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:47.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15cadc7700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15c4064760 0x7f15c41476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:47.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15cadc7700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c40c16b0 con 0x7f15c4064380 2021-01-06T22:05:47.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15cadc7700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c4147b90 con 0x7f15c414e310 2021-01-06T22:05:47.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15cadc7700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c40bb2d0 con 0x7f15c4064760 2021-01-06T22:05:47.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15c9dc5700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f15c4064380 0x7f15c40c0fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48586/0 (socket says 172.21.15.203:48586) 2021-01-06T22:05:47.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15c9dc5700 1 -- 172.21.15.203:0/362999018 learned_addr learned my addr 172.21.15.203:0/362999018 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:05:47.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15c95c4700 1 --2- 172.21.15.203:0/362999018 >> v2:172.21.15.203:3300/0 conn(0x7f15c414e310 0x7f15c41471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:47.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15ca5c6700 1 --2- 172.21.15.203:0/362999018 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15c4064760 0x7f15c41476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:47.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15c95c4700 1 -- 172.21.15.203:0/362999018 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15c4064760 msgr2=0x7f15c41476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:05:47.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15c95c4700 1 --2- 172.21.15.203:0/362999018 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15c4064760 0x7f15c41476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:47.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15c95c4700 1 -- 172.21.15.203:0/362999018 >> v1:172.21.15.203:6789/0 conn(0x7f15c4064380 legacy=0x7f15c40c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:47.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15c95c4700 1 -- 172.21.15.203:0/362999018 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c414a560 con 0x7f15c414e310 2021-01-06T22:05:47.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15c8dc3700 1 -- 172.21.15.203:0/362999018 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2411566136 0 0) 0x7f15c00010d0 con 0x7f15c4064380 2021-01-06T22:05:47.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15ca5c6700 1 --2- 172.21.15.203:0/362999018 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15c4064760 0x7f15c41476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:05:47.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15c95c4700 1 --2- 172.21.15.203:0/362999018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c414e310 0x7f15c41471b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f15b400cb30 tx=0x7f15b4006e70).ready entity=mon.1 client_cookie=a5cd8db59cf18e3a server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:47.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15c8dc3700 1 -- 172.21.15.203:0/362999018 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b400a8d0 con 0x7f15c414e310 2021-01-06T22:05:47.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15c8dc3700 1 -- 172.21.15.203:0/362999018 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b400aa30 con 0x7f15c414e310 2021-01-06T22:05:47.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15c8dc3700 1 -- 172.21.15.203:0/362999018 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b400adb0 con 0x7f15c414e310 2021-01-06T22:05:47.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15cadc7700 1 -- 172.21.15.203:0/362999018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c414e310 msgr2=0x7f15c41471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:47.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15cadc7700 1 --2- 172.21.15.203:0/362999018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c414e310 0x7f15c41471b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f15b400cb30 tx=0x7f15b4006e70).stop 2021-01-06T22:05:47.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15cadc7700 1 -- 172.21.15.203:0/362999018 shutdown_connections 2021-01-06T22:05:47.478 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15cadc7700 1 --2- 172.21.15.203:0/362999018 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15c4064760 0x7f15c41476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:47.478 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15cadc7700 1 --2- 172.21.15.203:0/362999018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c414e310 0x7f15c41471b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:47.478 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15cadc7700 1 -- 172.21.15.203:0/362999018 >> 172.21.15.203:0/362999018 conn(0x7f15c40667b0 msgr2=0x7f15c40c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:47.478 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15cadc7700 1 -- 172.21.15.203:0/362999018 shutdown_connections 2021-01-06T22:05:47.478 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15cadc7700 1 -- 172.21.15.203:0/362999018 wait complete. 2021-01-06T22:05:47.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15cadc7700 1 Processor -- start 2021-01-06T22:05:47.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.471+0000 7f15cadc7700 1 -- start start 2021-01-06T22:05:47.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15cadc7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c4064380 0x7f15c4149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:47.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15cadc7700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15c4064760 0x7f15c4159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:47.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15cadc7700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15c414e310 0x7f15c4159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:47.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15cadc7700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c40b6420 con 0x7f15c4064760 2021-01-06T22:05:47.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15cadc7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c40b5d20 con 0x7f15c4064380 2021-01-06T22:05:47.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15cadc7700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c40be980 con 0x7f15c414e310 2021-01-06T22:05:47.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15c9dc5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c4064380 0x7f15c4149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:47.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15c9dc5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c4064380 0x7f15c4149160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49998/0 (socket says 172.21.15.203:49998) 2021-01-06T22:05:47.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15c9dc5700 1 -- 172.21.15.203:0/2313304184 learned_addr learned my addr 172.21.15.203:0/2313304184 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:47.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15ca5c6700 1 --2- 172.21.15.203:0/2313304184 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15c414e310 0x7f15c4159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:47.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15c95c4700 1 --2- 172.21.15.203:0/2313304184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15c4064760 0x7f15c4159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:47.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15c9dc5700 1 -- 172.21.15.203:0/2313304184 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15c414e310 msgr2=0x7f15c4159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:47.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15c9dc5700 1 --2- 172.21.15.203:0/2313304184 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15c414e310 0x7f15c4159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:47.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15c9dc5700 1 -- 172.21.15.203:0/2313304184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15c4064760 msgr2=0x7f15c4159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:47.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15c9dc5700 1 --2- 172.21.15.203:0/2313304184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15c4064760 0x7f15c4159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:47.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15c9dc5700 1 -- 172.21.15.203:0/2313304184 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c41516c0 con 0x7f15c4064380 2021-01-06T22:05:47.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15c95c4700 1 --2- 172.21.15.203:0/2313304184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15c4064760 0x7f15c4159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:05:47.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15c9dc5700 1 --2- 172.21.15.203:0/2313304184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c4064380 0x7f15c4149160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f15c000c760 tx=0x7f15c0007910).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:47.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15b1ffb700 1 -- 172.21.15.203:0/2313304184 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c0007320 con 0x7f15c4064380 2021-01-06T22:05:47.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15b1ffb700 1 -- 172.21.15.203:0/2313304184 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c0007480 con 0x7f15c4064380 2021-01-06T22:05:47.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15b1ffb700 1 -- 172.21.15.203:0/2313304184 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c00057f0 con 0x7f15c4064380 2021-01-06T22:05:47.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15cadc7700 1 -- 172.21.15.203:0/2313304184 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c4151180 con 0x7f15c4064380 2021-01-06T22:05:47.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15cadc7700 1 -- 172.21.15.203:0/2313304184 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c40b5f80 con 0x7f15c4064380 2021-01-06T22:05:47.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15cadc7700 1 -- 172.21.15.203:0/2313304184 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c40450a0 con 0x7f15c4064380 2021-01-06T22:05:47.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.475+0000 7f15b1ffb700 1 -- 172.21.15.203:0/2313304184 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f15c0010030 con 0x7f15c4064380 2021-01-06T22:05:47.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.479+0000 7f15b1ffb700 1 --2- 172.21.15.203:0/2313304184 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f15a40563d0 0x7f15a4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:47.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.479+0000 7f15b1ffb700 1 -- 172.21.15.203:0/2313304184 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f15c006f110 con 0x7f15c4064380 2021-01-06T22:05:47.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.479+0000 7f15c95c4700 1 --2- 172.21.15.203:0/2313304184 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f15a40563d0 0x7f15a4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:47.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.479+0000 7f15c95c4700 1 --2- 172.21.15.203:0/2313304184 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f15a40563d0 0x7f15a4058890 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f15b4006b70 tx=0x7f15b4017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:47.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.483+0000 7f15b1ffb700 1 -- 172.21.15.203:0/2313304184 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f15c0044940 con 0x7f15c4064380 2021-01-06T22:05:47.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.691+0000 7f15cadc7700 1 -- 172.21.15.203:0/2313304184 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15c414a2a0 con 0x7f15c4064380 2021-01-06T22:05:47.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.691+0000 7f15b1ffb700 1 -- 172.21.15.203:0/2313304184 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3578 (secure 0 0 0) 0x7f15c0047f60 con 0x7f15c4064380 2021-01-06T22:05:47.690 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:05:47.690 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4867,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":18}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:05:31.368778+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":2,"incarnation":13,"state":"up:active","state_seq":18,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:05:47.691 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.691+0000 7f15cadc7700 1 -- 172.21.15.203:0/2313304184 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f15a40563d0 msgr2=0x7f15a4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:47.691 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.691+0000 7f15cadc7700 1 --2- 172.21.15.203:0/2313304184 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f15a40563d0 0x7f15a4058890 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f15b4006b70 tx=0x7f15b4017000).stop 2021-01-06T22:05:47.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.691+0000 7f15cadc7700 1 -- 172.21.15.203:0/2313304184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c4064380 msgr2=0x7f15c4149160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:47.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.691+0000 7f15cadc7700 1 --2- 172.21.15.203:0/2313304184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c4064380 0x7f15c4149160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f15c000c760 tx=0x7f15c0007910).stop 2021-01-06T22:05:47.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.691+0000 7f15cadc7700 1 -- 172.21.15.203:0/2313304184 shutdown_connections 2021-01-06T22:05:47.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.691+0000 7f15cadc7700 1 --2- 172.21.15.203:0/2313304184 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f15a40563d0 0x7f15a4058890 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:47.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.691+0000 7f15cadc7700 1 --2- 172.21.15.203:0/2313304184 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15c414e310 0x7f15c4159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:47.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.691+0000 7f15cadc7700 1 --2- 172.21.15.203:0/2313304184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c4064380 0x7f15c4149160 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:47.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.691+0000 7f15cadc7700 1 --2- 172.21.15.203:0/2313304184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15c4064760 0x7f15c4159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:47.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.691+0000 7f15cadc7700 1 -- 172.21.15.203:0/2313304184 >> 172.21.15.203:0/2313304184 conn(0x7f15c40667b0 msgr2=0x7f15c4146f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:47.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.691+0000 7f15cadc7700 1 -- 172.21.15.203:0/2313304184 shutdown_connections 2021-01-06T22:05:47.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:47.691+0000 7f15cadc7700 1 -- 172.21.15.203:0/2313304184 wait complete. 2021-01-06T22:05:47.694 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 19 2021-01-06T22:05:47.705 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2021-01-06T22:05:49.705 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:05:49.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.843+0000 7fc7e2c93700 1 Processor -- start 2021-01-06T22:05:49.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e2c93700 1 -- start start 2021-01-06T22:05:49.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e2c93700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7dc063010 0x7fc7dc14e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:49.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e2c93700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc7dc0be1b0 0x7fc7dc147790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:49.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e2c93700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7dc0be990 con 0x7fc7dc0633f0 2021-01-06T22:05:49.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e2c93700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7dc14ec60 con 0x7fc7dc0be1b0 2021-01-06T22:05:49.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e2c93700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc0be6f0 con 0x7fc7dc063010 2021-01-06T22:05:49.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e1c91700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7dc063010 0x7fc7dc14e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:49.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e2492700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc7dc0be1b0 0x7fc7dc147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:49.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e1c91700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7dc063010 0x7fc7dc14e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:55094/0 (socket says 172.21.15.203:55094) 2021-01-06T22:05:49.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e2492700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc7dc0be1b0 0x7fc7dc147790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:57864/0 (socket says 172.21.15.203:57864) 2021-01-06T22:05:49.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e1c91700 1 -- 172.21.15.203:0/2505074456 learned_addr learned my addr 172.21.15.203:0/2505074456 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:49.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e1c91700 1 -- 172.21.15.203:0/2505074456 >> v2:172.21.15.91:3300/0 conn(0x7fc7dc0be1b0 msgr2=0x7fc7dc147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:05:49.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e1c91700 1 --2- 172.21.15.203:0/2505074456 >> v2:172.21.15.91:3300/0 conn(0x7fc7dc0be1b0 0x7fc7dc147790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:49.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e1c91700 1 -- 172.21.15.203:0/2505074456 >> v1:172.21.15.91:6789/0 conn(0x7fc7dc0633f0 legacy=0x7fc7dc064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:49.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e1c91700 1 -- 172.21.15.203:0/2505074456 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7dc0c1050 con 0x7fc7dc063010 2021-01-06T22:05:49.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e1c91700 1 --2- 172.21.15.203:0/2505074456 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7dc063010 0x7fc7dc14e770 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fc7d800b9e0 tx=0x7fc7d8006ea0).ready entity=mon.2 client_cookie=9ac036df17a6a563 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:49.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e0c8f700 1 -- 172.21.15.203:0/2505074456 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80098a0 con 0x7fc7dc063010 2021-01-06T22:05:49.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e0c8f700 1 -- 172.21.15.203:0/2505074456 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d8009a00 con 0x7fc7dc063010 2021-01-06T22:05:49.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.847+0000 7fc7e0c8f700 1 -- 172.21.15.203:0/2505074456 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8009d80 con 0x7fc7dc063010 2021-01-06T22:05:49.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.851+0000 7fc7e2c93700 1 -- 172.21.15.203:0/2505074456 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7dc063010 msgr2=0x7fc7dc14e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:49.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.851+0000 7fc7e2c93700 1 --2- 172.21.15.203:0/2505074456 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7dc063010 0x7fc7dc14e770 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fc7d800b9e0 tx=0x7fc7d8006ea0).stop 2021-01-06T22:05:49.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.851+0000 7fc7e2c93700 1 -- 172.21.15.203:0/2505074456 shutdown_connections 2021-01-06T22:05:49.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.851+0000 7fc7e2c93700 1 --2- 172.21.15.203:0/2505074456 >> v2:172.21.15.91:3300/0 conn(0x7fc7dc0be1b0 0x7fc7dc147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:49.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.851+0000 7fc7e2c93700 1 --2- 172.21.15.203:0/2505074456 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7dc063010 0x7fc7dc14e770 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:49.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.851+0000 7fc7e2c93700 1 -- 172.21.15.203:0/2505074456 >> 172.21.15.203:0/2505074456 conn(0x7fc7dc067e20 msgr2=0x7fc7dc068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:49.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.851+0000 7fc7e2c93700 1 -- 172.21.15.203:0/2505074456 shutdown_connections 2021-01-06T22:05:49.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.851+0000 7fc7e2c93700 1 -- 172.21.15.203:0/2505074456 wait complete. 2021-01-06T22:05:49.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.851+0000 7fc7e2c93700 1 Processor -- start 2021-01-06T22:05:49.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.851+0000 7fc7e2c93700 1 -- start start 2021-01-06T22:05:49.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2c93700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7dc063010 0x7fc7dc15a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:49.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2c93700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7dc0633f0 0x7fc7dc14d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:49.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2c93700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7dc0be1b0 0x7fc7dc148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:49.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2c93700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc0c0cd0 con 0x7fc7dc0633f0 2021-01-06T22:05:49.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2c93700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc14a810 con 0x7fc7dc0be1b0 2021-01-06T22:05:49.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2c93700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc14aab0 con 0x7fc7dc063010 2021-01-06T22:05:49.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2492700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7dc0be1b0 0x7fc7dc148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:49.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2492700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7dc0be1b0 0x7fc7dc148810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50022/0 (socket says 172.21.15.203:50022) 2021-01-06T22:05:49.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2492700 1 -- 172.21.15.203:0/3099686698 learned_addr learned my addr 172.21.15.203:0/3099686698 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:49.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e1490700 1 --2- 172.21.15.203:0/3099686698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7dc0633f0 0x7fc7dc14d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:49.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e1c91700 1 --2- 172.21.15.203:0/3099686698 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7dc063010 0x7fc7dc15a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:49.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2492700 1 -- 172.21.15.203:0/3099686698 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7dc063010 msgr2=0x7fc7dc15a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:49.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2492700 1 --2- 172.21.15.203:0/3099686698 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7dc063010 0x7fc7dc15a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:49.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2492700 1 -- 172.21.15.203:0/3099686698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7dc0633f0 msgr2=0x7fc7dc14d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:49.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2492700 1 --2- 172.21.15.203:0/3099686698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7dc0633f0 0x7fc7dc14d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:49.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2492700 1 -- 172.21.15.203:0/3099686698 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7dc14cc90 con 0x7fc7dc0be1b0 2021-01-06T22:05:49.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2492700 1 --2- 172.21.15.203:0/3099686698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7dc0be1b0 0x7fc7dc148810 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fc7d400c200 tx=0x7fc7d40091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:49.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7c9ffb700 1 -- 172.21.15.203:0/3099686698 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d400ce60 con 0x7fc7dc0be1b0 2021-01-06T22:05:49.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7c9ffb700 1 -- 172.21.15.203:0/3099686698 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d4012030 con 0x7fc7dc0be1b0 2021-01-06T22:05:49.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7c9ffb700 1 -- 172.21.15.203:0/3099686698 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40123d0 con 0x7fc7dc0be1b0 2021-01-06T22:05:49.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2c93700 1 -- 172.21.15.203:0/3099686698 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7dc14cf30 con 0x7fc7dc0be1b0 2021-01-06T22:05:49.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2c93700 1 -- 172.21.15.203:0/3099686698 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7dc060a50 con 0x7fc7dc0be1b0 2021-01-06T22:05:49.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7c9ffb700 1 -- 172.21.15.203:0/3099686698 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fc7d40117e0 con 0x7fc7dc0be1b0 2021-01-06T22:05:49.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e2c93700 1 -- 172.21.15.203:0/3099686698 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7dc0450a0 con 0x7fc7dc0be1b0 2021-01-06T22:05:49.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7c9ffb700 1 --2- 172.21.15.203:0/3099686698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc7bc0563d0 0x7fc7bc058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:49.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7c9ffb700 1 -- 172.21.15.203:0/3099686698 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc7d4073980 con 0x7fc7dc0be1b0 2021-01-06T22:05:49.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.855+0000 7fc7e1c91700 1 --2- 172.21.15.203:0/3099686698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc7bc0563d0 0x7fc7bc058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:49.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.859+0000 7fc7e1c91700 1 --2- 172.21.15.203:0/3099686698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc7bc0563d0 0x7fc7bc058890 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fc7d8006ee0 tx=0x7fc7d8018000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:49.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:49.863+0000 7fc7c9ffb700 1 -- 172.21.15.203:0/3099686698 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fc7d40491b0 con 0x7fc7dc0be1b0 2021-01-06T22:05:50.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:50.091+0000 7fc7e2c93700 1 -- 172.21.15.203:0/3099686698 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7dc061e70 con 0x7fc7dc0be1b0 2021-01-06T22:05:50.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:50.095+0000 7fc7c9ffb700 1 -- 172.21.15.203:0/3099686698 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3578 (secure 0 0 0) 0x7fc7d4020030 con 0x7fc7dc0be1b0 2021-01-06T22:05:50.092 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:05:50.092 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4867,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":18}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:05:31.368778+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":2,"incarnation":13,"state":"up:active","state_seq":18,"addr":"172.21.15.91:6836/3250440616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3250440616},{"type":"v1","addr":"172.21.15.91:6836","nonce":3250440616}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:05:50.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:50.095+0000 7fc7e2c93700 1 -- 172.21.15.203:0/3099686698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc7bc0563d0 msgr2=0x7fc7bc058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:50.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:50.095+0000 7fc7e2c93700 1 --2- 172.21.15.203:0/3099686698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc7bc0563d0 0x7fc7bc058890 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fc7d8006ee0 tx=0x7fc7d8018000).stop 2021-01-06T22:05:50.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:50.095+0000 7fc7e2c93700 1 -- 172.21.15.203:0/3099686698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7dc0be1b0 msgr2=0x7fc7dc148810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:50.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:50.095+0000 7fc7e2c93700 1 --2- 172.21.15.203:0/3099686698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7dc0be1b0 0x7fc7dc148810 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fc7d400c200 tx=0x7fc7d40091d0).stop 2021-01-06T22:05:50.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:50.095+0000 7fc7e2c93700 1 -- 172.21.15.203:0/3099686698 shutdown_connections 2021-01-06T22:05:50.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:50.095+0000 7fc7e2c93700 1 --2- 172.21.15.203:0/3099686698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc7bc0563d0 0x7fc7bc058890 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:50.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:50.095+0000 7fc7e2c93700 1 --2- 172.21.15.203:0/3099686698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7dc0be1b0 0x7fc7dc148810 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:50.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:50.095+0000 7fc7e2c93700 1 --2- 172.21.15.203:0/3099686698 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc7dc063010 0x7fc7dc15a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:50.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:50.095+0000 7fc7e2c93700 1 --2- 172.21.15.203:0/3099686698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7dc0633f0 0x7fc7dc14d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:50.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:50.095+0000 7fc7e2c93700 1 -- 172.21.15.203:0/3099686698 >> 172.21.15.203:0/3099686698 conn(0x7fc7dc067e20 msgr2=0x7fc7dc0602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:50.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:50.095+0000 7fc7e2c93700 1 -- 172.21.15.203:0/3099686698 shutdown_connections 2021-01-06T22:05:50.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:50.095+0000 7fc7e2c93700 1 -- 172.21.15.203:0/3099686698 wait complete. 2021-01-06T22:05:50.098 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 19 2021-01-06T22:05:50.108 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2021-01-06T22:05:52.109 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:05:52.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.243+0000 7fdfd27fd700 1 Processor -- start 2021-01-06T22:05:52.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.243+0000 7fdfd27fd700 1 -- start start 2021-01-06T22:05:52.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.243+0000 7fdfd27fd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdfcc067b40 0x7fdfcc14e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:52.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.243+0000 7fdfd27fd700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdfcc067f20 0x7fdfcc1521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:52.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.243+0000 7fdfd27fd700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfcc0beb40 con 0x7fdfcc0b8f20 2021-01-06T22:05:52.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.243+0000 7fdfd27fd700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfcc0be600 con 0x7fdfcc067b40 2021-01-06T22:05:52.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.243+0000 7fdfd27fd700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc0be8a0 con 0x7fdfcc067f20 2021-01-06T22:05:52.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.243+0000 7fdfcbfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdfcc067b40 0x7fdfcc14e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:52.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.243+0000 7fdfcbfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdfcc067b40 0x7fdfcc14e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50038/0 (socket says 172.21.15.203:50038) 2021-01-06T22:05:52.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.243+0000 7fdfcbfff700 1 -- 172.21.15.203:0/3950078411 learned_addr learned my addr 172.21.15.203:0/3950078411 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:52.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.243+0000 7fdfcb7fe700 1 --2- 172.21.15.203:0/3950078411 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdfcc067f20 0x7fdfcc1521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:52.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.243+0000 7fdfcbfff700 1 -- 172.21.15.203:0/3950078411 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdfcc067f20 msgr2=0x7fdfcc1521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:05:52.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.243+0000 7fdfcbfff700 1 --2- 172.21.15.203:0/3950078411 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdfcc067f20 0x7fdfcc1521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:52.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.243+0000 7fdfcbfff700 1 -- 172.21.15.203:0/3950078411 >> v1:172.21.15.91:6789/0 conn(0x7fdfcc0b8f20 legacy=0x7fdfcc0bddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:52.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.243+0000 7fdfcbfff700 1 -- 172.21.15.203:0/3950078411 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc1517d0 con 0x7fdfcc067b40 2021-01-06T22:05:52.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.243+0000 7fdfcbfff700 1 --2- 172.21.15.203:0/3950078411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfcc067b40 0x7fdfcc14e160 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fdfc4009d50 tx=0x7fdfc4009290).ready entity=mon.1 client_cookie=1c7341a933eb1655 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:52.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfcaffd700 1 -- 172.21.15.203:0/3950078411 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400bd10 con 0x7fdfcc067b40 2021-01-06T22:05:52.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfcaffd700 1 -- 172.21.15.203:0/3950078411 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc400be70 con 0x7fdfcc067b40 2021-01-06T22:05:52.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfcaffd700 1 -- 172.21.15.203:0/3950078411 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4011610 con 0x7fdfcc067b40 2021-01-06T22:05:52.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd27fd700 1 -- 172.21.15.203:0/3950078411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfcc067b40 msgr2=0x7fdfcc14e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:52.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd27fd700 1 --2- 172.21.15.203:0/3950078411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfcc067b40 0x7fdfcc14e160 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fdfc4009d50 tx=0x7fdfc4009290).stop 2021-01-06T22:05:52.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd27fd700 1 -- 172.21.15.203:0/3950078411 shutdown_connections 2021-01-06T22:05:52.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd27fd700 1 --2- 172.21.15.203:0/3950078411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfcc067b40 0x7fdfcc14e160 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:52.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd27fd700 1 --2- 172.21.15.203:0/3950078411 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdfcc067f20 0x7fdfcc1521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:52.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd27fd700 1 -- 172.21.15.203:0/3950078411 >> 172.21.15.203:0/3950078411 conn(0x7fdfcc056cd0 msgr2=0x7fdfcc0570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:52.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd27fd700 1 -- 172.21.15.203:0/3950078411 shutdown_connections 2021-01-06T22:05:52.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd27fd700 1 -- 172.21.15.203:0/3950078411 wait complete. 2021-01-06T22:05:52.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd27fd700 1 Processor -- start 2021-01-06T22:05:52.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd27fd700 1 -- start start 2021-01-06T22:05:52.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd27fd700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdfcc067b40 0x7fdfcc0b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:52.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd27fd700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfcc067f20 0x7fdfcc0ba070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:52.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd27fd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfcc0b8f20 0x7fdfcc14d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:52.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd27fd700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc0ba560 con 0x7fdfcc067f20 2021-01-06T22:05:52.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd27fd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc0b9750 con 0x7fdfcc0b8f20 2021-01-06T22:05:52.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd27fd700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc0b1430 con 0x7fdfcc067b40 2021-01-06T22:05:52.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfcbfff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdfcc067b40 0x7fdfcc0b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:52.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfd0d9a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfcc0b8f20 0x7fdfcc14d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:52.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfcbfff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdfcc067b40 0x7fdfcc0b9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:55126/0 (socket says 172.21.15.203:55126) 2021-01-06T22:05:52.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfcbfff700 1 -- 172.21.15.203:0/2129486527 learned_addr learned my addr 172.21.15.203:0/2129486527 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:52.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.247+0000 7fdfcb7fe700 1 --2- 172.21.15.203:0/2129486527 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfcc067f20 0x7fdfcc0ba070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:52.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfd0d9a700 1 -- 172.21.15.203:0/2129486527 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdfcc067b40 msgr2=0x7fdfcc0b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:52.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfd0d9a700 1 --2- 172.21.15.203:0/2129486527 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdfcc067b40 0x7fdfcc0b9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:52.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfd0d9a700 1 -- 172.21.15.203:0/2129486527 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfcc067f20 msgr2=0x7fdfcc0ba070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:52.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfd0d9a700 1 --2- 172.21.15.203:0/2129486527 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfcc067f20 0x7fdfcc0ba070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:52.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfd0d9a700 1 -- 172.21.15.203:0/2129486527 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc15ecb0 con 0x7fdfcc0b8f20 2021-01-06T22:05:52.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfd0d9a700 1 --2- 172.21.15.203:0/2129486527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfcc0b8f20 0x7fdfcc14d030 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fdfbc00c4d0 tx=0x7fdfbc007b80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:52.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfc3fff700 1 -- 172.21.15.203:0/2129486527 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00f470 con 0x7fdfcc0b8f20 2021-01-06T22:05:52.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfc3fff700 1 -- 172.21.15.203:0/2129486527 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc00f5d0 con 0x7fdfcc0b8f20 2021-01-06T22:05:52.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfd27fd700 1 -- 172.21.15.203:0/2129486527 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfcc0b2a20 con 0x7fdfcc0b8f20 2021-01-06T22:05:52.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfc3fff700 1 -- 172.21.15.203:0/2129486527 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00f950 con 0x7fdfcc0b8f20 2021-01-06T22:05:52.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfd27fd700 1 -- 172.21.15.203:0/2129486527 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfcc0b2f60 con 0x7fdfcc0b8f20 2021-01-06T22:05:52.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfd27fd700 1 -- 172.21.15.203:0/2129486527 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfcc064e20 con 0x7fdfcc0b8f20 2021-01-06T22:05:52.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfc3fff700 1 -- 172.21.15.203:0/2129486527 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fdfbc019030 con 0x7fdfcc0b8f20 2021-01-06T22:05:52.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfc3fff700 1 --2- 172.21.15.203:0/2129486527 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdfb00563d0 0x7fdfb0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:52.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfc3fff700 1 -- 172.21.15.203:0/2129486527 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fdfbc06e790 con 0x7fdfcc0b8f20 2021-01-06T22:05:52.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfcbfff700 1 --2- 172.21.15.203:0/2129486527 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdfb00563d0 0x7fdfb0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:52.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.251+0000 7fdfcbfff700 1 --2- 172.21.15.203:0/2129486527 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdfb00563d0 0x7fdfb0058890 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fdfc4009500 tx=0x7fdfc4017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:52.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.259+0000 7fdfc3fff700 1 -- 172.21.15.203:0/2129486527 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fdfbc044010 con 0x7fdfcc0b8f20 2021-01-06T22:05:52.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.491+0000 7fdfd27fd700 1 -- 172.21.15.203:0/2129486527 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfcc0b9890 con 0x7fdfcc0b8f20 2021-01-06T22:05:52.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.491+0000 7fdfc3fff700 1 -- 172.21.15.203:0/2129486527 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+3226 (secure 0 0 0) 0x7fdfbc047630 con 0x7fdfcc0b8f20 2021-01-06T22:05:52.489 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:05:52.489 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:05:52.350635+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:resolve","state_seq":19,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:05:52.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.491+0000 7fdfd27fd700 1 -- 172.21.15.203:0/2129486527 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdfb00563d0 msgr2=0x7fdfb0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:52.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.491+0000 7fdfd27fd700 1 --2- 172.21.15.203:0/2129486527 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdfb00563d0 0x7fdfb0058890 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fdfc4009500 tx=0x7fdfc4017000).stop 2021-01-06T22:05:52.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.491+0000 7fdfd27fd700 1 -- 172.21.15.203:0/2129486527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfcc0b8f20 msgr2=0x7fdfcc14d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:52.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.491+0000 7fdfd27fd700 1 --2- 172.21.15.203:0/2129486527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfcc0b8f20 0x7fdfcc14d030 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fdfbc00c4d0 tx=0x7fdfbc007b80).stop 2021-01-06T22:05:52.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.491+0000 7fdfd27fd700 1 -- 172.21.15.203:0/2129486527 shutdown_connections 2021-01-06T22:05:52.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.491+0000 7fdfd27fd700 1 --2- 172.21.15.203:0/2129486527 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdfb00563d0 0x7fdfb0058890 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:52.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.491+0000 7fdfd27fd700 1 --2- 172.21.15.203:0/2129486527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfcc0b8f20 0x7fdfcc14d030 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:52.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.491+0000 7fdfd27fd700 1 --2- 172.21.15.203:0/2129486527 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdfcc067b40 0x7fdfcc0b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:52.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.491+0000 7fdfd27fd700 1 --2- 172.21.15.203:0/2129486527 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfcc067f20 0x7fdfcc0ba070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:52.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.491+0000 7fdfd27fd700 1 -- 172.21.15.203:0/2129486527 >> 172.21.15.203:0/2129486527 conn(0x7fdfcc056cd0 msgr2=0x7fdfcc065320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:52.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.491+0000 7fdfd27fd700 1 -- 172.21.15.203:0/2129486527 shutdown_connections 2021-01-06T22:05:52.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.491+0000 7fdfd27fd700 1 -- 172.21.15.203:0/2129486527 wait complete. 2021-01-06T22:05:52.495 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 21 2021-01-06T22:05:52.505 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2021-01-06T22:05:52.506 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2021-01-06T22:05:52.506 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:05:52.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72e19ad700 1 Processor -- start 2021-01-06T22:05:52.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72e19ad700 1 -- start start 2021-01-06T22:05:52.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72e19ad700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f72dc0c13c0 0x7f72dc0aa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:52.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72e19ad700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72dc0a9ca0 con 0x7f72dc0c0fe0 2021-01-06T22:05:52.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72e19ad700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72dc066a50 con 0x7f72dc0c03f0 2021-01-06T22:05:52.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72e19ad700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72dc066cf0 con 0x7f72dc0c13c0 2021-01-06T22:05:52.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72dbfff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f72dc0c03f0 0x7f72dc150110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48652/0 (socket says 172.21.15.203:48652) 2021-01-06T22:05:52.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72dbfff700 1 -- 172.21.15.203:0/3348375570 learned_addr learned my addr 172.21.15.203:0/3348375570 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:05:52.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72e11ac700 1 --2- 172.21.15.203:0/3348375570 >> v2:172.21.15.203:3300/0 conn(0x7f72dc0c13c0 0x7f72dc0aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:52.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72e11ac700 1 -- 172.21.15.203:0/3348375570 >> v1:172.21.15.203:6789/0 conn(0x7f72dc0c03f0 legacy=0x7f72dc150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:52.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72e11ac700 1 -- 172.21.15.203:0/3348375570 >> v1:172.21.15.91:6789/0 conn(0x7f72dc0c0fe0 legacy=0x7f72dc060d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:52.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72e11ac700 1 -- 172.21.15.203:0/3348375570 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc068460 con 0x7f72dc0c13c0 2021-01-06T22:05:52.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72e11ac700 1 --2- 172.21.15.203:0/3348375570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0c13c0 0x7f72dc0aa030 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f72d4009c90 tx=0x7f72d40091d0).ready entity=mon.1 client_cookie=c2dba1e337574cfb server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:52.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72db7fe700 1 -- 172.21.15.203:0/3348375570 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d400bbc0 con 0x7f72dc0c13c0 2021-01-06T22:05:52.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72db7fe700 1 -- 172.21.15.203:0/3348375570 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d400bd20 con 0x7f72dc0c13c0 2021-01-06T22:05:52.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72db7fe700 1 -- 172.21.15.203:0/3348375570 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d4011610 con 0x7f72dc0c13c0 2021-01-06T22:05:52.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72e19ad700 1 -- 172.21.15.203:0/3348375570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0c13c0 msgr2=0x7f72dc0aa030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:52.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72e19ad700 1 --2- 172.21.15.203:0/3348375570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0c13c0 0x7f72dc0aa030 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f72d4009c90 tx=0x7f72d40091d0).stop 2021-01-06T22:05:52.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72e19ad700 1 -- 172.21.15.203:0/3348375570 shutdown_connections 2021-01-06T22:05:52.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72e19ad700 1 --2- 172.21.15.203:0/3348375570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0c13c0 0x7f72dc0aa030 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:52.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.631+0000 7f72e19ad700 1 -- 172.21.15.203:0/3348375570 >> 172.21.15.203:0/3348375570 conn(0x7f72dc061530 msgr2=0x7f72dc061910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:52.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e19ad700 1 -- 172.21.15.203:0/3348375570 shutdown_connections 2021-01-06T22:05:52.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e19ad700 1 -- 172.21.15.203:0/3348375570 wait complete. 2021-01-06T22:05:52.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e19ad700 1 Processor -- start 2021-01-06T22:05:52.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e19ad700 1 -- start start 2021-01-06T22:05:52.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e19ad700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0c03f0 0x7f72dc152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:52.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e19ad700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72dc0c0fe0 0x7f72dc1539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:52.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e19ad700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72dc0c13c0 0x7f72dc144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:52.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e19ad700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc064800 con 0x7f72dc0c0fe0 2021-01-06T22:05:52.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e19ad700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc153310 con 0x7f72dc0c03f0 2021-01-06T22:05:52.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e19ad700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc153560 con 0x7f72dc0c13c0 2021-01-06T22:05:52.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e09ab700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0c03f0 0x7f72dc152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:52.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e09ab700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0c03f0 0x7f72dc152e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50062/0 (socket says 172.21.15.203:50062) 2021-01-06T22:05:52.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e09ab700 1 -- 172.21.15.203:0/285641239 learned_addr learned my addr 172.21.15.203:0/285641239 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:52.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72dbfff700 1 --2- 172.21.15.203:0/285641239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72dc0c0fe0 0x7f72dc1539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:52.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e11ac700 1 --2- 172.21.15.203:0/285641239 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72dc0c13c0 0x7f72dc144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:52.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e09ab700 1 -- 172.21.15.203:0/285641239 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72dc0c13c0 msgr2=0x7f72dc144bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:52.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e09ab700 1 --2- 172.21.15.203:0/285641239 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72dc0c13c0 0x7f72dc144bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:52.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e09ab700 1 -- 172.21.15.203:0/285641239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72dc0c0fe0 msgr2=0x7f72dc1539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:52.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e09ab700 1 --2- 172.21.15.203:0/285641239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72dc0c0fe0 0x7f72dc1539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:52.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e09ab700 1 -- 172.21.15.203:0/285641239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc148660 con 0x7f72dc0c03f0 2021-01-06T22:05:52.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e11ac700 1 --2- 172.21.15.203:0/285641239 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72dc0c13c0 0x7f72dc144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:05:52.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72e09ab700 1 --2- 172.21.15.203:0/285641239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0c03f0 0x7f72dc152e20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f72d000c710 tx=0x7f72d0007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:52.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.635+0000 7f72d8ff9700 1 -- 172.21.15.203:0/285641239 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0007020 con 0x7f72dc0c03f0 2021-01-06T22:05:52.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.639+0000 7f72d8ff9700 1 -- 172.21.15.203:0/285641239 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d0007180 con 0x7f72dc0c03f0 2021-01-06T22:05:52.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.639+0000 7f72e19ad700 1 -- 172.21.15.203:0/285641239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc148900 con 0x7f72dc0c03f0 2021-01-06T22:05:52.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.639+0000 7f72d8ff9700 1 -- 172.21.15.203:0/285641239 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000e560 con 0x7f72dc0c03f0 2021-01-06T22:05:52.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.639+0000 7f72e19ad700 1 -- 172.21.15.203:0/285641239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc1490e0 con 0x7f72dc0c03f0 2021-01-06T22:05:52.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.639+0000 7f72d8ff9700 1 -- 172.21.15.203:0/285641239 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f72d0010070 con 0x7f72dc0c03f0 2021-01-06T22:05:52.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.639+0000 7f72e19ad700 1 -- 172.21.15.203:0/285641239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc148f20 con 0x7f72dc0c03f0 2021-01-06T22:05:52.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.639+0000 7f72d8ff9700 1 --2- 172.21.15.203:0/285641239 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72bc056400 0x7f72bc0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:52.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.639+0000 7f72d8ff9700 1 -- 172.21.15.203:0/285641239 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f72d006ea20 con 0x7f72dc0c03f0 2021-01-06T22:05:52.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.639+0000 7f72dbfff700 1 --2- 172.21.15.203:0/285641239 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72bc056400 0x7f72bc0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:52.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.639+0000 7f72dbfff700 1 --2- 172.21.15.203:0/285641239 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72bc056400 0x7f72bc0588c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f72c80045d0 tx=0x7f72c8007b70).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:52.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.647+0000 7f72d8ff9700 1 -- 172.21.15.203:0/285641239 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f72d00441f0 con 0x7f72dc0c03f0 2021-01-06T22:05:52.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.859+0000 7f72e19ad700 1 -- 172.21.15.203:0/285641239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72dc0450a0 con 0x7f72dc0c03f0 2021-01-06T22:05:52.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.859+0000 7f72d8ff9700 1 -- 172.21.15.203:0/285641239 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+3226 (secure 0 0 0) 0x7f72d0047810 con 0x7f72dc0c03f0 2021-01-06T22:05:52.859 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:05:52.860 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:05:52.350635+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:resolve","state_seq":19,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:05:52.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.863+0000 7f72e19ad700 1 -- 172.21.15.203:0/285641239 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72bc056400 msgr2=0x7f72bc0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:52.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.863+0000 7f72e19ad700 1 --2- 172.21.15.203:0/285641239 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72bc056400 0x7f72bc0588c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f72c80045d0 tx=0x7f72c8007b70).stop 2021-01-06T22:05:52.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.863+0000 7f72e19ad700 1 -- 172.21.15.203:0/285641239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0c03f0 msgr2=0x7f72dc152e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:52.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.863+0000 7f72e19ad700 1 --2- 172.21.15.203:0/285641239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0c03f0 0x7f72dc152e20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f72d000c710 tx=0x7f72d0007c40).stop 2021-01-06T22:05:52.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.863+0000 7f72e19ad700 1 -- 172.21.15.203:0/285641239 shutdown_connections 2021-01-06T22:05:52.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.863+0000 7f72e19ad700 1 --2- 172.21.15.203:0/285641239 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72bc056400 0x7f72bc0588c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:52.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.863+0000 7f72e19ad700 1 --2- 172.21.15.203:0/285641239 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72dc0c13c0 0x7f72dc144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:52.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.863+0000 7f72e19ad700 1 --2- 172.21.15.203:0/285641239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0c03f0 0x7f72dc152e20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:52.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.863+0000 7f72e19ad700 1 --2- 172.21.15.203:0/285641239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72dc0c0fe0 0x7f72dc1539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:52.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.863+0000 7f72e19ad700 1 -- 172.21.15.203:0/285641239 >> 172.21.15.203:0/285641239 conn(0x7f72dc061530 msgr2=0x7f72dc147630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:52.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.863+0000 7f72e19ad700 1 -- 172.21.15.203:0/285641239 shutdown_connections 2021-01-06T22:05:52.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:52.863+0000 7f72e19ad700 1 -- 172.21.15.203:0/285641239 wait complete. 2021-01-06T22:05:52.864 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 21 2021-01-06T22:05:52.875 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 21, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 21, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4273": { "addr": "172.21.15.203:6837/3277561222", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6836", "nonce": 3277561222, "type": "v2" }, { "addr": "172.21.15.203:6837", "nonce": 3277561222, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4273, "incarnation": 9, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 3 }, "gid_4274": { "addr": "172.21.15.91:6837/3888667247", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6835", "nonce": 3888667247, "type": "v2" }, { "addr": "172.21.15.91:6837", "nonce": 3888667247, "type": "v1" } ] }, "export_targets": [ 1 ], "features": 4540138297136906239, "flags": 0, "gid": 4274, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 2 }, "gid_4867": { "addr": "172.21.15.203:6835/3382231276", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6834", "nonce": 3382231276, "type": "v2" }, { "addr": "172.21.15.203:6835", "nonce": 3382231276, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4867, "incarnation": 20, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:resolve", "state_seq": 19 } }, "last_failure": 0, "last_failure_osd_epoch": 25, "max_file_size": 1099511627776, "max_mds": 3, "metadata_pool": 3, "modified": "2021-01-06T22:05:52.350635+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4274, "mds_1": 4273, "mds_2": 4867 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2021-01-06T22:05:54.876 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:05:54.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62de036700 1 Processor -- start 2021-01-06T22:05:54.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62de036700 1 -- start start 2021-01-06T22:05:54.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62de036700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f62d80c44f0 0x7f62d80c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:54.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62de036700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f62d80c48d0 0x7f62d80c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:54.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62de036700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d80c0f50 con 0x7f62d8060fc0 2021-01-06T22:05:54.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62de036700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d80c1d60 con 0x7f62d80c48d0 2021-01-06T22:05:54.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62de036700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d8066cf0 con 0x7f62d80c44f0 2021-01-06T22:05:54.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62dd835700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f62d8060fc0 0x7f62d8150730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48668/0 (socket says 172.21.15.203:48668) 2021-01-06T22:05:55.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62dd835700 1 -- 172.21.15.203:0/440316240 learned_addr learned my addr 172.21.15.203:0/440316240 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:05:55.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62dd034700 1 --2- 172.21.15.203:0/440316240 >> v2:172.21.15.203:3300/0 conn(0x7f62d80c44f0 0x7f62d80c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:55.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62dc833700 1 --2- 172.21.15.203:0/440316240 >> v2:172.21.15.91:3300/0 conn(0x7f62d80c48d0 0x7f62d80c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:55.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62dd034700 1 -- 172.21.15.203:0/440316240 >> v2:172.21.15.91:3300/0 conn(0x7f62d80c48d0 msgr2=0x7f62d80c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:05:55.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62dd034700 1 --2- 172.21.15.203:0/440316240 >> v2:172.21.15.91:3300/0 conn(0x7f62d80c48d0 0x7f62d80c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:55.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62dd034700 1 -- 172.21.15.203:0/440316240 >> v1:172.21.15.203:6789/0 conn(0x7f62d8060fc0 legacy=0x7f62d8150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:55.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62dd034700 1 -- 172.21.15.203:0/440316240 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d8060750 con 0x7f62d80c44f0 2021-01-06T22:05:55.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62cffff700 1 -- 172.21.15.203:0/440316240 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946049004 0 0) 0x7f62d80c0f50 con 0x7f62d8060fc0 2021-01-06T22:05:55.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62dd034700 1 --2- 172.21.15.203:0/440316240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d80c44f0 0x7f62d80c1380 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f62d805c9f0 tx=0x7f62d4006de0).ready entity=mon.1 client_cookie=65c30de6ba80a8c1 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:55.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62cffff700 1 -- 172.21.15.203:0/440316240 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d40096c0 con 0x7f62d80c44f0 2021-01-06T22:05:55.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62cffff700 1 -- 172.21.15.203:0/440316240 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d4009820 con 0x7f62d80c44f0 2021-01-06T22:05:55.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62cffff700 1 -- 172.21.15.203:0/440316240 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4009ba0 con 0x7f62d80c44f0 2021-01-06T22:05:55.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62de036700 1 -- 172.21.15.203:0/440316240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d80c44f0 msgr2=0x7f62d80c1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:55.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62de036700 1 --2- 172.21.15.203:0/440316240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d80c44f0 0x7f62d80c1380 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f62d805c9f0 tx=0x7f62d4006de0).stop 2021-01-06T22:05:55.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62de036700 1 -- 172.21.15.203:0/440316240 shutdown_connections 2021-01-06T22:05:55.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62de036700 1 --2- 172.21.15.203:0/440316240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d80c44f0 0x7f62d80c1380 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:55.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62de036700 1 --2- 172.21.15.203:0/440316240 >> v2:172.21.15.91:3300/0 conn(0x7f62d80c48d0 0x7f62d80c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:55.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.995+0000 7f62de036700 1 -- 172.21.15.203:0/440316240 >> 172.21.15.203:0/440316240 conn(0x7f62d8056cd0 msgr2=0x7f62d8061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:55.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.999+0000 7f62de036700 1 -- 172.21.15.203:0/440316240 shutdown_connections 2021-01-06T22:05:55.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.999+0000 7f62de036700 1 -- 172.21.15.203:0/440316240 wait complete. 2021-01-06T22:05:55.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.999+0000 7f62de036700 1 Processor -- start 2021-01-06T22:05:55.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.999+0000 7f62de036700 1 -- start start 2021-01-06T22:05:55.008 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.999+0000 7f62de036700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d8060fc0 0x7f62d815bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:55.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.999+0000 7f62de036700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62d80c44f0 0x7f62d81612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:55.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.999+0000 7f62de036700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62d80c48d0 0x7f62d8161790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:55.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.999+0000 7f62de036700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d8153520 con 0x7f62d80c44f0 2021-01-06T22:05:55.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.999+0000 7f62de036700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d8153750 con 0x7f62d8060fc0 2021-01-06T22:05:55.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.999+0000 7f62de036700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d81539f0 con 0x7f62d80c48d0 2021-01-06T22:05:55.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.999+0000 7f62dd034700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d8060fc0 0x7f62d815bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:55.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.999+0000 7f62dd034700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d8060fc0 0x7f62d815bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50076/0 (socket says 172.21.15.203:50076) 2021-01-06T22:05:55.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:54.999+0000 7f62dd034700 1 -- 172.21.15.203:0/673821021 learned_addr learned my addr 172.21.15.203:0/673821021 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:55.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62dc833700 1 --2- 172.21.15.203:0/673821021 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62d80c44f0 0x7f62d81612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:55.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62dd835700 1 --2- 172.21.15.203:0/673821021 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62d80c48d0 0x7f62d8161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:55.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62dd034700 1 -- 172.21.15.203:0/673821021 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62d80c48d0 msgr2=0x7f62d8161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:55.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62dd034700 1 --2- 172.21.15.203:0/673821021 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62d80c48d0 0x7f62d8161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:55.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62dd034700 1 -- 172.21.15.203:0/673821021 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62d80c44f0 msgr2=0x7f62d81612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:55.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62dd034700 1 --2- 172.21.15.203:0/673821021 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62d80c44f0 0x7f62d81612a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:55.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62dd034700 1 -- 172.21.15.203:0/673821021 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d8153c90 con 0x7f62d8060fc0 2021-01-06T22:05:55.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62dd835700 1 --2- 172.21.15.203:0/673821021 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62d80c48d0 0x7f62d8161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:05:55.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62dd034700 1 --2- 172.21.15.203:0/673821021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d8060fc0 0x7f62d815bb90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f62d8066e30 tx=0x7f62d4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:55.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62cd7fa700 1 -- 172.21.15.203:0/673821021 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4012e50 con 0x7f62d8060fc0 2021-01-06T22:05:55.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62cd7fa700 1 -- 172.21.15.203:0/673821021 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d401c630 con 0x7f62d8060fc0 2021-01-06T22:05:55.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62de036700 1 -- 172.21.15.203:0/673821021 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d81518d0 con 0x7f62d8060fc0 2021-01-06T22:05:55.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62cd7fa700 1 -- 172.21.15.203:0/673821021 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d401c9d0 con 0x7f62d8060fc0 2021-01-06T22:05:55.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62de036700 1 -- 172.21.15.203:0/673821021 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d8152010 con 0x7f62d8060fc0 2021-01-06T22:05:55.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62de036700 1 -- 172.21.15.203:0/673821021 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d8150130 con 0x7f62d8060fc0 2021-01-06T22:05:55.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62cd7fa700 1 -- 172.21.15.203:0/673821021 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f62d400f070 con 0x7f62d8060fc0 2021-01-06T22:05:55.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62cd7fa700 1 --2- 172.21.15.203:0/673821021 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f62b80563d0 0x7f62b8058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:55.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62cd7fa700 1 -- 172.21.15.203:0/673821021 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f62d4070f60 con 0x7f62d8060fc0 2021-01-06T22:05:55.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.003+0000 7f62dc833700 1 --2- 172.21.15.203:0/673821021 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f62b80563d0 0x7f62b8058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:55.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.007+0000 7f62dc833700 1 --2- 172.21.15.203:0/673821021 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f62b80563d0 0x7f62b8058890 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f62c800caf0 tx=0x7f62c8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:55.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.011+0000 7f62cd7fa700 1 -- 172.21.15.203:0/673821021 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f62d8150130 con 0x7f62d8060fc0 2021-01-06T22:05:55.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.243+0000 7f62de036700 1 -- 172.21.15.203:0/673821021 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62d80aa3d0 con 0x7f62d8060fc0 2021-01-06T22:05:55.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.243+0000 7f62cd7fa700 1 -- 172.21.15.203:0/673821021 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+3225 (secure 0 0 0) 0x7f62d80aa3d0 con 0x7f62d8060fc0 2021-01-06T22:05:55.243 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:05:55.243 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:05:54.362143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:rejoin","state_seq":21,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:05:55.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.247+0000 7f62de036700 1 -- 172.21.15.203:0/673821021 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f62b80563d0 msgr2=0x7f62b8058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:55.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.247+0000 7f62de036700 1 --2- 172.21.15.203:0/673821021 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f62b80563d0 0x7f62b8058890 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f62c800caf0 tx=0x7f62c8008040).stop 2021-01-06T22:05:55.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.247+0000 7f62de036700 1 -- 172.21.15.203:0/673821021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d8060fc0 msgr2=0x7f62d815bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:55.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.247+0000 7f62de036700 1 --2- 172.21.15.203:0/673821021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d8060fc0 0x7f62d815bb90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f62d8066e30 tx=0x7f62d4016000).stop 2021-01-06T22:05:55.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.247+0000 7f62de036700 1 -- 172.21.15.203:0/673821021 shutdown_connections 2021-01-06T22:05:55.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.247+0000 7f62de036700 1 --2- 172.21.15.203:0/673821021 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f62b80563d0 0x7f62b8058890 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:55.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.247+0000 7f62de036700 1 --2- 172.21.15.203:0/673821021 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62d80c48d0 0x7f62d8161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:55.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.247+0000 7f62de036700 1 --2- 172.21.15.203:0/673821021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62d8060fc0 0x7f62d815bb90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:55.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.247+0000 7f62de036700 1 --2- 172.21.15.203:0/673821021 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62d80c44f0 0x7f62d81612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:55.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.247+0000 7f62de036700 1 -- 172.21.15.203:0/673821021 >> 172.21.15.203:0/673821021 conn(0x7f62d8056cd0 msgr2=0x7f62d814fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:55.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.247+0000 7f62de036700 1 -- 172.21.15.203:0/673821021 shutdown_connections 2021-01-06T22:05:55.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:55.247+0000 7f62de036700 1 -- 172.21.15.203:0/673821021 wait complete. 2021-01-06T22:05:55.249 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 23 2021-01-06T22:05:55.260 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:05:57.261 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:05:57.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.399+0000 7ff93f97d700 1 Processor -- start 2021-01-06T22:05:57.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.399+0000 7ff93f97d700 1 -- start start 2021-01-06T22:05:57.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.399+0000 7ff93f97d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff938064f50 0x7ff9380bad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:57.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.399+0000 7ff93f97d700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff9380bb240 0x7ff9380673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:57.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.399+0000 7ff93f97d700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9380653b0 con 0x7ff938064b70 2021-01-06T22:05:57.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.399+0000 7ff93f97d700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9380bed30 con 0x7ff9380bb240 2021-01-06T22:05:57.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.399+0000 7ff93f97d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93814b1a0 con 0x7ff938064f50 2021-01-06T22:05:57.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.399+0000 7ff93df1a700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff9380bb240 0x7ff9380673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:57.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.399+0000 7ff93df1a700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff9380bb240 0x7ff9380673a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:57938/0 (socket says 172.21.15.203:57938) 2021-01-06T22:05:57.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.399+0000 7ff93df1a700 1 -- 172.21.15.203:0/2239965503 learned_addr learned my addr 172.21.15.203:0/2239965503 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:57.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.399+0000 7ff93cf18700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff938064f50 0x7ff9380bad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:57.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.399+0000 7ff93df1a700 1 -- 172.21.15.203:0/2239965503 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff938064f50 msgr2=0x7ff9380bad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:05:57.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.399+0000 7ff93df1a700 1 --2- 172.21.15.203:0/2239965503 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff938064f50 0x7ff9380bad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:57.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.399+0000 7ff93df1a700 1 -- 172.21.15.203:0/2239965503 >> v1:172.21.15.91:6789/0 conn(0x7ff938064b70 legacy=0x7ff938144aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:57.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.399+0000 7ff93df1a700 1 -- 172.21.15.203:0/2239965503 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff938068230 con 0x7ff9380bb240 2021-01-06T22:05:57.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.399+0000 7ff93cf18700 1 --2- 172.21.15.203:0/2239965503 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff938064f50 0x7ff9380bad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:05:57.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93df1a700 1 --2- 172.21.15.203:0/2239965503 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9380bb240 0x7ff9380673a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7ff938065540 tx=0x7ff93000b530).ready entity=mon.0 client_cookie=194c3c2ad2d3c108 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:57.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff927fff700 1 -- 172.21.15.203:0/2239965503 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93000de40 con 0x7ff9380bb240 2021-01-06T22:05:57.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff927fff700 1 -- 172.21.15.203:0/2239965503 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff930010680 con 0x7ff9380bb240 2021-01-06T22:05:57.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff927fff700 1 -- 172.21.15.203:0/2239965503 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930010a00 con 0x7ff9380bb240 2021-01-06T22:05:57.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93f97d700 1 -- 172.21.15.203:0/2239965503 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9380bb240 msgr2=0x7ff9380673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:57.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93f97d700 1 --2- 172.21.15.203:0/2239965503 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9380bb240 0x7ff9380673a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7ff938065540 tx=0x7ff93000b530).stop 2021-01-06T22:05:57.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93f97d700 1 -- 172.21.15.203:0/2239965503 shutdown_connections 2021-01-06T22:05:57.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93f97d700 1 --2- 172.21.15.203:0/2239965503 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9380bb240 0x7ff9380673a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:57.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93f97d700 1 --2- 172.21.15.203:0/2239965503 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff938064f50 0x7ff9380bad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:57.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93f97d700 1 -- 172.21.15.203:0/2239965503 >> 172.21.15.203:0/2239965503 conn(0x7ff938056cd0 msgr2=0x7ff9380570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:57.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93f97d700 1 -- 172.21.15.203:0/2239965503 shutdown_connections 2021-01-06T22:05:57.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93f97d700 1 -- 172.21.15.203:0/2239965503 wait complete. 2021-01-06T22:05:57.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93f97d700 1 Processor -- start 2021-01-06T22:05:57.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93f97d700 1 -- start start 2021-01-06T22:05:57.408 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93f97d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff938064b70 0x7ff9381618f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:57.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93f97d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff938064f50 0x7ff9381575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:57.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93f97d700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9380bb240 0x7ff938157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:57.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93f97d700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff938161de0 con 0x7ff9380bb240 2021-01-06T22:05:57.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93f97d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9380b19f0 con 0x7ff938064f50 2021-01-06T22:05:57.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.403+0000 7ff93f97d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9380b14b0 con 0x7ff938064b70 2021-01-06T22:05:57.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff93cf18700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff938064f50 0x7ff9381575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:57.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff93df1a700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9380bb240 0x7ff938157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:57.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff93cf18700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff938064f50 0x7ff9381575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50096/0 (socket says 172.21.15.203:50096) 2021-01-06T22:05:57.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff93cf18700 1 -- 172.21.15.203:0/142684694 learned_addr learned my addr 172.21.15.203:0/142684694 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:05:57.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff93d719700 1 --2- 172.21.15.203:0/142684694 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff938064b70 0x7ff9381618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:57.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff93cf18700 1 -- 172.21.15.203:0/142684694 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff938064b70 msgr2=0x7ff9381618f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:57.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff93cf18700 1 --2- 172.21.15.203:0/142684694 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff938064b70 0x7ff9381618f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:57.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff93cf18700 1 -- 172.21.15.203:0/142684694 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9380bb240 msgr2=0x7ff938157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:57.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff93cf18700 1 --2- 172.21.15.203:0/142684694 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9380bb240 0x7ff938157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:57.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff93cf18700 1 -- 172.21.15.203:0/142684694 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9380b1750 con 0x7ff938064f50 2021-01-06T22:05:57.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff93cf18700 1 --2- 172.21.15.203:0/142684694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff938064f50 0x7ff9381575a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7ff92800ef80 tx=0x7ff928006de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:57.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff9257fa700 1 -- 172.21.15.203:0/142684694 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92800cb00 con 0x7ff938064f50 2021-01-06T22:05:57.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff9257fa700 1 -- 172.21.15.203:0/142684694 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92800cc60 con 0x7ff938064f50 2021-01-06T22:05:57.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff93f97d700 1 -- 172.21.15.203:0/142684694 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93811abe0 con 0x7ff938064f50 2021-01-06T22:05:57.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff9257fa700 1 -- 172.21.15.203:0/142684694 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9280123f0 con 0x7ff938064f50 2021-01-06T22:05:57.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff93f97d700 1 -- 172.21.15.203:0/142684694 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93811b120 con 0x7ff938064f50 2021-01-06T22:05:57.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff93f97d700 1 -- 172.21.15.203:0/142684694 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93805d600 con 0x7ff938064f50 2021-01-06T22:05:57.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.407+0000 7ff9257fa700 1 -- 172.21.15.203:0/142684694 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff9280127e0 con 0x7ff938064f50 2021-01-06T22:05:57.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.411+0000 7ff9257fa700 1 --2- 172.21.15.203:0/142684694 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff9180563d0 0x7ff918058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:05:57.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.411+0000 7ff9257fa700 1 -- 172.21.15.203:0/142684694 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7ff92806fb10 con 0x7ff938064f50 2021-01-06T22:05:57.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.411+0000 7ff93d719700 1 --2- 172.21.15.203:0/142684694 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff9180563d0 0x7ff918058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:05:57.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.411+0000 7ff93d719700 1 --2- 172.21.15.203:0/142684694 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff9180563d0 0x7ff918058890 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7ff934003790 tx=0x7ff934007af0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:05:57.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.415+0000 7ff9257fa700 1 -- 172.21.15.203:0/142684694 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff9280452e0 con 0x7ff938064f50 2021-01-06T22:05:57.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.635+0000 7ff93f97d700 1 -- 172.21.15.203:0/142684694 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff938061dc0 con 0x7ff938064f50 2021-01-06T22:05:57.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.639+0000 7ff9257fa700 1 -- 172.21.15.203:0/142684694 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+3225 (secure 0 0 0) 0x7ff928048900 con 0x7ff938064f50 2021-01-06T22:05:57.637 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:05:57.637 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:05:55.364759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:05:57.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.639+0000 7ff93f97d700 1 -- 172.21.15.203:0/142684694 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff9180563d0 msgr2=0x7ff918058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:57.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.639+0000 7ff93f97d700 1 --2- 172.21.15.203:0/142684694 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff9180563d0 0x7ff918058890 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7ff934003790 tx=0x7ff934007af0).stop 2021-01-06T22:05:57.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.639+0000 7ff93f97d700 1 -- 172.21.15.203:0/142684694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff938064f50 msgr2=0x7ff9381575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:05:57.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.639+0000 7ff93f97d700 1 --2- 172.21.15.203:0/142684694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff938064f50 0x7ff9381575a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7ff92800ef80 tx=0x7ff928006de0).stop 2021-01-06T22:05:57.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.639+0000 7ff93f97d700 1 -- 172.21.15.203:0/142684694 shutdown_connections 2021-01-06T22:05:57.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.639+0000 7ff93f97d700 1 --2- 172.21.15.203:0/142684694 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff9180563d0 0x7ff918058890 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:57.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.639+0000 7ff93f97d700 1 --2- 172.21.15.203:0/142684694 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9380bb240 0x7ff938157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:57.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.639+0000 7ff93f97d700 1 --2- 172.21.15.203:0/142684694 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff938064b70 0x7ff9381618f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:57.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.639+0000 7ff93f97d700 1 --2- 172.21.15.203:0/142684694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff938064f50 0x7ff9381575a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:05:57.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.639+0000 7ff93f97d700 1 -- 172.21.15.203:0/142684694 >> 172.21.15.203:0/142684694 conn(0x7ff938056cd0 msgr2=0x7ff938149370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:05:57.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.639+0000 7ff93f97d700 1 -- 172.21.15.203:0/142684694 shutdown_connections 2021-01-06T22:05:57.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:05:57.639+0000 7ff93f97d700 1 -- 172.21.15.203:0/142684694 wait complete. 2021-01-06T22:05:57.642 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 24 2021-01-06T22:05:57.652 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=2, replacing gid=4254 2021-01-06T22:05:57.652 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.b 2021-01-06T22:06:04.653 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2021-01-06T22:06:04.654 INFO:tasks.ceph.mds.b:Restarting daemon 2021-01-06T22:06:04.654 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-01-06T22:06:04.656 INFO:tasks.ceph.mds.b:Started 2021-01-06T22:06:04.657 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:06:04.740 INFO:tasks.ceph.mds.b.smithi091.stdout:starting mds.b at 2021-01-06T22:06:04.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.787+0000 7f3f1b59e700 1 Processor -- start 2021-01-06T22:06:04.787 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.787+0000 7f3f1b59e700 1 -- start start 2021-01-06T22:06:04.788 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1b59e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3f1c060da0 0x7f3f1c14ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:06:04.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1b59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1c0668d0 con 0x7f3f1c0bd6d0 2021-01-06T22:06:04.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1b59e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1c066b70 con 0x7f3f1c0602b0 2021-01-06T22:06:04.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1b59e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1c067a90 con 0x7f3f1c060da0 2021-01-06T22:06:04.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1a59c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f3f1c0602b0 0x7f3f1c14b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48706/0 (socket says 172.21.15.203:48706) 2021-01-06T22:06:04.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1a59c700 1 -- 172.21.15.203:0/2221243543 learned_addr learned my addr 172.21.15.203:0/2221243543 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:06:04.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1ad9d700 1 --2- 172.21.15.203:0/2221243543 >> v2:172.21.15.91:3300/0 conn(0x7f3f1c060da0 0x7f3f1c14ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:06:04.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 -- 172.21.15.203:0/2221243543 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2108610231 0 0) 0x7f3f1c066b70 con 0x7f3f1c0602b0 2021-01-06T22:06:04.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 -- 172.21.15.203:0/2221243543 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3efc003230 con 0x7f3f1c0602b0 2021-01-06T22:06:04.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 -- 172.21.15.203:0/2221243543 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3802546456 0 0) 0x7f3efc003230 con 0x7f3f1c0602b0 2021-01-06T22:06:04.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 -- 172.21.15.203:0/2221243543 >> v2:172.21.15.91:3300/0 conn(0x7f3f1c060da0 msgr2=0x7f3f1c14ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:06:04.796 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 --2- 172.21.15.203:0/2221243543 >> v2:172.21.15.91:3300/0 conn(0x7f3f1c060da0 0x7f3f1c14ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:04.796 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 -- 172.21.15.203:0/2221243543 >> v1:172.21.15.91:6789/0 conn(0x7f3f1c0bd6d0 legacy=0x7f3f1c060690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:06:04.796 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 -- 172.21.15.203:0/2221243543 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c066e10 con 0x7f3f1c0602b0 2021-01-06T22:06:04.796 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 -- 172.21.15.203:0/2221243543 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f3f0c0014e0 con 0x7f3f1c0602b0 2021-01-06T22:06:04.797 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 -- 172.21.15.203:0/2221243543 >> v1:172.21.15.203:6789/0 conn(0x7f3f1c0602b0 legacy=0x7f3f1c14b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:06:04.797 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 --2- 172.21.15.203:0/2221243543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3efc003720 0x7f3efc003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:06:04.797 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 --2- 172.21.15.203:0/2221243543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3efc0044d0 0x7f3efc004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:06:04.797 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 --2- 172.21.15.203:0/2221243543 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3efc004e40 0x7f3efc00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:06:04.798 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 -- 172.21.15.203:0/2221243543 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c067a90 con 0x7f3efc003720 2021-01-06T22:06:04.798 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 -- 172.21.15.203:0/2221243543 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc004250 con 0x7f3efc0044d0 2021-01-06T22:06:04.798 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 -- 172.21.15.203:0/2221243543 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc00bc10 con 0x7f3efc004e40 2021-01-06T22:06:04.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f11d9b700 1 --2- 172.21.15.203:0/2221243543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3efc0044d0 0x7f3efc004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:06:04.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1a59c700 1 --2- 172.21.15.203:0/2221243543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3efc003720 0x7f3efc003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:06:04.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1ad9d700 1 --2- 172.21.15.203:0/2221243543 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3efc004e40 0x7f3efc00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:06:04.800 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f11d9b700 1 -- 172.21.15.203:0/2221243543 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3efc004e40 msgr2=0x7f3efc00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:06:04.800 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f11d9b700 1 --2- 172.21.15.203:0/2221243543 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3efc004e40 0x7f3efc00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:04.800 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f11d9b700 1 -- 172.21.15.203:0/2221243543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3efc003720 msgr2=0x7f3efc003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:06:04.800 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f11d9b700 1 --2- 172.21.15.203:0/2221243543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3efc003720 0x7f3efc003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:04.801 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1ad9d700 1 -- 172.21.15.203:0/2221243543 reap_dead start 2021-01-06T22:06:04.801 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f11d9b700 1 -- 172.21.15.203:0/2221243543 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc003230 con 0x7f3efc0044d0 2021-01-06T22:06:04.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1a59c700 1 --2- 172.21.15.203:0/2221243543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3efc003720 0x7f3efc003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:06:04.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f11d9b700 1 --2- 172.21.15.203:0/2221243543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3efc0044d0 0x7f3efc004950 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f3f04007e40 tx=0x7f3f04007b80).ready entity=mon.1 client_cookie=acf6ec03da2903c4 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:06:04.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 -- 172.21.15.203:0/2221243543 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04007020 con 0x7f3efc0044d0 2021-01-06T22:06:04.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 -- 172.21.15.203:0/2221243543 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f04007180 con 0x7f3efc0044d0 2021-01-06T22:06:04.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f19d9b700 1 -- 172.21.15.203:0/2221243543 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0400c830 con 0x7f3efc0044d0 2021-01-06T22:06:04.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1b59e700 1 -- 172.21.15.203:0/2221243543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3efc0044d0 msgr2=0x7f3efc004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:06:04.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1b59e700 1 --2- 172.21.15.203:0/2221243543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3efc0044d0 0x7f3efc004950 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f3f04007e40 tx=0x7f3f04007b80).stop 2021-01-06T22:06:04.804 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1b59e700 1 -- 172.21.15.203:0/2221243543 shutdown_connections 2021-01-06T22:06:04.804 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1b59e700 1 --2- 172.21.15.203:0/2221243543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3efc0044d0 0x7f3efc004950 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:04.804 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1b59e700 1 -- 172.21.15.203:0/2221243543 >> 172.21.15.203:0/2221243543 conn(0x7f3f1c061530 msgr2=0x7f3f1c063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:06:04.805 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1b59e700 1 -- 172.21.15.203:0/2221243543 shutdown_connections 2021-01-06T22:06:04.805 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.791+0000 7f3f1b59e700 1 -- 172.21.15.203:0/2221243543 wait complete. 2021-01-06T22:06:04.805 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f1b59e700 1 Processor -- start 2021-01-06T22:06:04.805 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f1b59e700 1 -- start start 2021-01-06T22:06:04.805 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f1b59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3efc0044d0 0x7f3f1c0bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:06:04.806 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f1b59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f1c14d5f0 0x7f3f1c1453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:06:04.806 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f1b59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f1c1458d0 0x7f3f1c15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:06:04.806 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f1b59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0a5c50 con 0x7f3f1c1458d0 2021-01-06T22:06:04.806 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f1b59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c09f810 con 0x7f3f1c14d5f0 2021-01-06T22:06:04.807 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f1b59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c09f5b0 con 0x7f3efc0044d0 2021-01-06T22:06:04.807 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f11d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f1c14d5f0 0x7f3f1c1453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:06:04.807 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f11d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f1c14d5f0 0x7f3f1c1453e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50126/0 (socket says 172.21.15.203:50126) 2021-01-06T22:06:04.807 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f11d9b700 1 -- 172.21.15.203:0/3539208345 learned_addr learned my addr 172.21.15.203:0/3539208345 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:06:04.808 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f1a59c700 1 --2- 172.21.15.203:0/3539208345 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3efc0044d0 0x7f3f1c0bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:06:04.808 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f1ad9d700 1 --2- 172.21.15.203:0/3539208345 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f1c1458d0 0x7f3f1c15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:06:04.808 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f11d9b700 1 -- 172.21.15.203:0/3539208345 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3efc0044d0 msgr2=0x7f3f1c0bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:06:04.809 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f11d9b700 1 --2- 172.21.15.203:0/3539208345 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3efc0044d0 0x7f3f1c0bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:04.809 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f11d9b700 1 -- 172.21.15.203:0/3539208345 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f1c1458d0 msgr2=0x7f3f1c15bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:06:04.809 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f11d9b700 1 --2- 172.21.15.203:0/3539208345 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f1c1458d0 0x7f3f1c15bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:04.809 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f11d9b700 1 -- 172.21.15.203:0/3539208345 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c0a2170 con 0x7f3f1c14d5f0 2021-01-06T22:06:04.810 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f1ad9d700 1 --2- 172.21.15.203:0/3539208345 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f1c1458d0 0x7f3f1c15bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:06:04.810 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f11d9b700 1 --2- 172.21.15.203:0/3539208345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f1c14d5f0 0x7f3f1c1453e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f3f04011000 tx=0x7f3f04016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:06:04.810 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f12ffd700 1 -- 172.21.15.203:0/3539208345 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f040128d0 con 0x7f3f1c14d5f0 2021-01-06T22:06:04.810 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f12ffd700 1 -- 172.21.15.203:0/3539208345 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f04012a30 con 0x7f3f1c14d5f0 2021-01-06T22:06:04.811 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f1b59e700 1 -- 172.21.15.203:0/3539208345 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1c09e4c0 con 0x7f3f1c14d5f0 2021-01-06T22:06:04.811 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f12ffd700 1 -- 172.21.15.203:0/3539208345 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04012df0 con 0x7f3f1c14d5f0 2021-01-06T22:06:04.811 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.795+0000 7f3f1b59e700 1 -- 172.21.15.203:0/3539208345 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1c15b7f0 con 0x7f3f1c14d5f0 2021-01-06T22:06:04.813 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.799+0000 7f3f1b59e700 1 -- 172.21.15.203:0/3539208345 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f1c145cb0 con 0x7f3f1c14d5f0 2021-01-06T22:06:04.813 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.799+0000 7f3f12ffd700 1 -- 172.21.15.203:0/3539208345 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f3f04018070 con 0x7f3f1c14d5f0 2021-01-06T22:06:04.813 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.799+0000 7f3f12ffd700 1 --2- 172.21.15.203:0/3539208345 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3f000563d0 0x7f3f00058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:06:04.813 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.799+0000 7f3f12ffd700 1 -- 172.21.15.203:0/3539208345 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f3f04070b20 con 0x7f3f1c14d5f0 2021-01-06T22:06:04.814 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.799+0000 7f3f1a59c700 1 --2- 172.21.15.203:0/3539208345 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3f000563d0 0x7f3f00058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:06:04.814 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.799+0000 7f3f1a59c700 1 --2- 172.21.15.203:0/3539208345 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3f000563d0 0x7f3f00058890 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f3f0c000bc0 tx=0x7f3f1c066fa0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:06:04.814 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:04.803+0000 7f3f12ffd700 1 -- 172.21.15.203:0/3539208345 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f3f04046370 con 0x7f3f1c14d5f0 2021-01-06T22:06:05.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:05.031+0000 7f3f1b59e700 1 -- 172.21.15.203:0/3539208345 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f1c0450a0 con 0x7f3f1c14d5f0 2021-01-06T22:06:05.029 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:05.031+0000 7f3f12ffd700 1 -- 172.21.15.203:0/3539208345 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3227 (secure 0 0 0) 0x7f3f0402c020 con 0x7f3f1c14d5f0 2021-01-06T22:06:05.029 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:06:05.030 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:06:01.400641+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:06:05.031 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:05.031+0000 7f3f1b59e700 1 -- 172.21.15.203:0/3539208345 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3f000563d0 msgr2=0x7f3f00058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:06:05.031 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:05.031+0000 7f3f1b59e700 1 --2- 172.21.15.203:0/3539208345 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3f000563d0 0x7f3f00058890 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f3f0c000bc0 tx=0x7f3f1c066fa0).stop 2021-01-06T22:06:05.032 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:05.031+0000 7f3f1b59e700 1 -- 172.21.15.203:0/3539208345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f1c14d5f0 msgr2=0x7f3f1c1453e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:06:05.032 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:05.031+0000 7f3f1b59e700 1 --2- 172.21.15.203:0/3539208345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f1c14d5f0 0x7f3f1c1453e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f3f04011000 tx=0x7f3f04016000).stop 2021-01-06T22:06:05.032 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:05.031+0000 7f3f1b59e700 1 -- 172.21.15.203:0/3539208345 shutdown_connections 2021-01-06T22:06:05.032 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:05.031+0000 7f3f1b59e700 1 --2- 172.21.15.203:0/3539208345 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3f000563d0 0x7f3f00058890 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:05.033 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:05.031+0000 7f3f1b59e700 1 --2- 172.21.15.203:0/3539208345 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f1c1458d0 0x7f3f1c15bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:05.033 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:05.031+0000 7f3f1b59e700 1 --2- 172.21.15.203:0/3539208345 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3efc0044d0 0x7f3f1c0bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:05.033 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:05.031+0000 7f3f1b59e700 1 --2- 172.21.15.203:0/3539208345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f1c14d5f0 0x7f3f1c1453e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:05.033 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:05.031+0000 7f3f1b59e700 1 -- 172.21.15.203:0/3539208345 >> 172.21.15.203:0/3539208345 conn(0x7f3f1c061530 msgr2=0x7f3f1c0bde40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:06:05.034 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:05.031+0000 7f3f1b59e700 1 -- 172.21.15.203:0/3539208345 shutdown_connections 2021-01-06T22:06:05.034 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:05.031+0000 7f3f1b59e700 1 -- 172.21.15.203:0/3539208345 wait complete. 2021-01-06T22:06:05.034 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 25 2021-01-06T22:06:05.042 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is in active, standby or standby-replay 2021-01-06T22:06:07.044 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:06:07.176 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa865edf700 1 Processor -- start 2021-01-06T22:06:07.177 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa865edf700 1 -- start start 2021-01-06T22:06:07.178 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa865edf700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa860148d70 0x7fa8600c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:06:07.178 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa865edf700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa8600602b0 0x7fa860149150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:06:07.178 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa865edf700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa86014fa10 0x7fa86014bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:06:07.178 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa865edf700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86014f0a0 con 0x7fa86014fa10 2021-01-06T22:06:07.179 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa865edf700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86014c3c0 con 0x7fa8600602b0 2021-01-06T22:06:07.179 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa865edf700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86014c660 con 0x7fa860148d70 2021-01-06T22:06:07.179 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa85ffff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa8600602b0 0x7fa860149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:06:07.179 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa85ffff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa8600602b0 0x7fa860149150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50134/0 (socket says 172.21.15.203:50134) 2021-01-06T22:06:07.179 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa85ffff700 1 -- 172.21.15.203:0/3559485279 learned_addr learned my addr 172.21.15.203:0/3559485279 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:06:07.180 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa864edd700 1 --2- 172.21.15.203:0/3559485279 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa860148d70 0x7fa8600c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:06:07.180 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa8656de700 1 --2- 172.21.15.203:0/3559485279 >> v2:172.21.15.91:3300/0 conn(0x7fa86014fa10 0x7fa86014bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:06:07.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa85ffff700 1 -- 172.21.15.203:0/3559485279 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa860148d70 msgr2=0x7fa8600c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:06:07.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa85ffff700 1 --2- 172.21.15.203:0/3559485279 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa860148d70 0x7fa8600c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:07.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa85ffff700 1 -- 172.21.15.203:0/3559485279 >> v2:172.21.15.91:3300/0 conn(0x7fa86014fa10 msgr2=0x7fa86014bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:06:07.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa85ffff700 1 --2- 172.21.15.203:0/3559485279 >> v2:172.21.15.91:3300/0 conn(0x7fa86014fa10 0x7fa86014bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:07.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa85ffff700 1 -- 172.21.15.203:0/3559485279 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86014c900 con 0x7fa8600602b0 2021-01-06T22:06:07.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa85ffff700 1 --2- 172.21.15.203:0/3559485279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8600602b0 0x7fa860149150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fa86005c9b0 tx=0x7fa84c006ea0).ready entity=mon.1 client_cookie=ce92ece965b75c61 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:06:07.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa85f7fe700 1 -- 172.21.15.203:0/3559485279 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c010960 con 0x7fa8600602b0 2021-01-06T22:06:07.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa85f7fe700 1 -- 172.21.15.203:0/3559485279 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84c010ac0 con 0x7fa8600602b0 2021-01-06T22:06:07.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.179+0000 7fa85f7fe700 1 -- 172.21.15.203:0/3559485279 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c010c30 con 0x7fa8600602b0 2021-01-06T22:06:07.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.183+0000 7fa865edf700 1 -- 172.21.15.203:0/3559485279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8600602b0 msgr2=0x7fa860149150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:06:07.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.183+0000 7fa865edf700 1 --2- 172.21.15.203:0/3559485279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8600602b0 0x7fa860149150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fa86005c9b0 tx=0x7fa84c006ea0).stop 2021-01-06T22:06:07.185 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.183+0000 7fa865edf700 1 -- 172.21.15.203:0/3559485279 shutdown_connections 2021-01-06T22:06:07.185 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.183+0000 7fa865edf700 1 --2- 172.21.15.203:0/3559485279 >> v2:172.21.15.91:3300/0 conn(0x7fa86014fa10 0x7fa86014bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:07.185 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.183+0000 7fa865edf700 1 --2- 172.21.15.203:0/3559485279 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa860148d70 0x7fa8600c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:07.185 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.183+0000 7fa865edf700 1 --2- 172.21.15.203:0/3559485279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8600602b0 0x7fa860149150 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:07.185 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.183+0000 7fa865edf700 1 -- 172.21.15.203:0/3559485279 >> 172.21.15.203:0/3559485279 conn(0x7fa8600667b0 msgr2=0x7fa8600c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:06:07.186 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.183+0000 7fa865edf700 1 -- 172.21.15.203:0/3559485279 shutdown_connections 2021-01-06T22:06:07.186 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.183+0000 7fa865edf700 1 -- 172.21.15.203:0/3559485279 wait complete. 2021-01-06T22:06:07.186 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.183+0000 7fa865edf700 1 Processor -- start 2021-01-06T22:06:07.186 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.183+0000 7fa865edf700 1 -- start start 2021-01-06T22:06:07.188 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa865edf700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8600602b0 0x7fa86014a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:06:07.189 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa865edf700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa860148d70 0x7fa8600acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:06:07.189 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa865edf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa86014fa10 0x7fa860159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:06:07.189 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa865edf700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86015a250 con 0x7fa8600602b0 2021-01-06T22:06:07.189 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa865edf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8600af910 con 0x7fa86014fa10 2021-01-06T22:06:07.190 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa865edf700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8600afbb0 con 0x7fa860148d70 2021-01-06T22:06:07.190 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa8656de700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa86014fa10 0x7fa860159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:06:07.190 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa8656de700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa86014fa10 0x7fa860159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50142/0 (socket says 172.21.15.203:50142) 2021-01-06T22:06:07.190 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa8656de700 1 -- 172.21.15.203:0/2142303033 learned_addr learned my addr 172.21.15.203:0/2142303033 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:06:07.190 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa85ffff700 1 --2- 172.21.15.203:0/2142303033 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa860148d70 0x7fa8600acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:06:07.191 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa864edd700 1 --2- 172.21.15.203:0/2142303033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8600602b0 0x7fa86014a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:06:07.191 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa8656de700 1 -- 172.21.15.203:0/2142303033 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa860148d70 msgr2=0x7fa8600acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:06:07.191 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa8656de700 1 --2- 172.21.15.203:0/2142303033 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa860148d70 0x7fa8600acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:07.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa8656de700 1 -- 172.21.15.203:0/2142303033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8600602b0 msgr2=0x7fa86014a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:06:07.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa8656de700 1 --2- 172.21.15.203:0/2142303033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8600602b0 0x7fa86014a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:07.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa8656de700 1 -- 172.21.15.203:0/2142303033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa860149b50 con 0x7fa86014fa10 2021-01-06T22:06:07.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa8656de700 1 --2- 172.21.15.203:0/2142303033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa86014fa10 0x7fa860159d60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fa85800ef50 tx=0x7fa8580091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:06:07.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa85cff9700 1 -- 172.21.15.203:0/2142303033 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858009700 con 0x7fa86014fa10 2021-01-06T22:06:07.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa85cff9700 1 -- 172.21.15.203:0/2142303033 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85800cbd0 con 0x7fa86014fa10 2021-01-06T22:06:07.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa865edf700 1 -- 172.21.15.203:0/2142303033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa860149df0 con 0x7fa86014fa10 2021-01-06T22:06:07.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa85cff9700 1 -- 172.21.15.203:0/2142303033 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8580123f0 con 0x7fa86014fa10 2021-01-06T22:06:07.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa865edf700 1 -- 172.21.15.203:0/2142303033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86014b2c0 con 0x7fa86014fa10 2021-01-06T22:06:07.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa865edf700 1 -- 172.21.15.203:0/2142303033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8600450a0 con 0x7fa86014fa10 2021-01-06T22:06:07.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa85cff9700 1 -- 172.21.15.203:0/2142303033 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa858022070 con 0x7fa86014fa10 2021-01-06T22:06:07.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.187+0000 7fa85cff9700 1 --2- 172.21.15.203:0/2142303033 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8400563d0 0x7fa840058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:06:07.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.191+0000 7fa85cff9700 1 -- 172.21.15.203:0/2142303033 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fa85806fdc0 con 0x7fa86014fa10 2021-01-06T22:06:07.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.191+0000 7fa864edd700 1 --2- 172.21.15.203:0/2142303033 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8400563d0 0x7fa840058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:06:07.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.191+0000 7fa864edd700 1 --2- 172.21.15.203:0/2142303033 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8400563d0 0x7fa840058890 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fa85400cac0 tx=0x7fa854006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:06:07.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.195+0000 7fa85cff9700 1 -- 172.21.15.203:0/2142303033 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa858045590 con 0x7fa86014fa10 2021-01-06T22:06:07.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.415+0000 7fa865edf700 1 -- 172.21.15.203:0/2142303033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa860057340 con 0x7fa86014fa10 2021-01-06T22:06:07.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.415+0000 7fa85cff9700 1 -- 172.21.15.203:0/2142303033 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3580 (secure 0 0 0) 0x7fa858016030 con 0x7fa86014fa10 2021-01-06T22:06:07.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.419+0000 7fa865edf700 1 -- 172.21.15.203:0/2142303033 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8400563d0 msgr2=0x7fa840058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:06:07.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.419+0000 7fa865edf700 1 --2- 172.21.15.203:0/2142303033 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8400563d0 0x7fa840058890 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fa85400cac0 tx=0x7fa854006de0).stop 2021-01-06T22:06:07.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.419+0000 7fa865edf700 1 -- 172.21.15.203:0/2142303033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa86014fa10 msgr2=0x7fa860159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:06:07.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.419+0000 7fa865edf700 1 --2- 172.21.15.203:0/2142303033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa86014fa10 0x7fa860159d60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fa85800ef50 tx=0x7fa8580091d0).stop 2021-01-06T22:06:07.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.419+0000 7fa865edf700 1 -- 172.21.15.203:0/2142303033 shutdown_connections 2021-01-06T22:06:07.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.419+0000 7fa865edf700 1 --2- 172.21.15.203:0/2142303033 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8400563d0 0x7fa840058890 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:07.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.419+0000 7fa865edf700 1 --2- 172.21.15.203:0/2142303033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa86014fa10 0x7fa860159d60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:07.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.419+0000 7fa865edf700 1 --2- 172.21.15.203:0/2142303033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8600602b0 0x7fa86014a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:07.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.419+0000 7fa865edf700 1 --2- 172.21.15.203:0/2142303033 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa860148d70 0x7fa8600acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:06:07.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.419+0000 7fa865edf700 1 -- 172.21.15.203:0/2142303033 >> 172.21.15.203:0/2142303033 conn(0x7fa8600667b0 msgr2=0x7fa860148270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:06:07.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.419+0000 7fa865edf700 1 -- 172.21.15.203:0/2142303033 shutdown_connections 2021-01-06T22:06:07.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:06:07.419+0000 7fa865edf700 1 -- 172.21.15.203:0/2142303033 wait complete. 2021-01-06T22:06:07.489 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 26 2021-01-06T22:06:07.489 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:06:07.489 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:06:01.400641+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:06:07.490 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2021-01-06T22:06:07.490 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 113 secs before thrashing 2021-01-06T22:06:12.186 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:06:12.195 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:06:42.249 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:06:42.252 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:07:12.301 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:07:12.305 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:07:42.426 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:07:42.429 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:08:00.491 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:00.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.619+0000 7ff7a956f700 1 Processor -- start 2021-01-06T22:08:00.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.619+0000 7ff7a956f700 1 -- start start 2021-01-06T22:08:00.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.619+0000 7ff7a956f700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff7a40602b0 0x7ff7a40c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:00.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.619+0000 7ff7a956f700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a414f0a0 con 0x7ff7a4148d70 2021-01-06T22:08:00.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a956f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a41495a0 con 0x7ff7a4149150 2021-01-06T22:08:00.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a956f700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a414c3c0 con 0x7ff7a40602b0 2021-01-06T22:08:00.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a8d6e700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7ff7a4149150 0x7ff7a414bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48820/0 (socket says 172.21.15.203:48820) 2021-01-06T22:08:00.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a8d6e700 1 -- 172.21.15.203:0/449069117 learned_addr learned my addr 172.21.15.203:0/449069117 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:00.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a37fe700 1 --2- 172.21.15.203:0/449069117 >> v2:172.21.15.91:3300/0 conn(0x7ff7a40602b0 0x7ff7a40c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:00.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a2ffd700 1 -- 172.21.15.203:0/449069117 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620157372 0 0) 0x7ff79c001450 con 0x7ff7a4149150 2021-01-06T22:08:00.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a2ffd700 1 -- 172.21.15.203:0/449069117 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff790003230 con 0x7ff7a4149150 2021-01-06T22:08:00.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a37fe700 1 -- 172.21.15.203:0/449069117 >> v1:172.21.15.203:6789/0 conn(0x7ff7a4149150 legacy=0x7ff7a414bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:00.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a37fe700 1 -- 172.21.15.203:0/449069117 >> v1:172.21.15.91:6789/0 conn(0x7ff7a4148d70 legacy=0x7ff7a414fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:00.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a37fe700 1 -- 172.21.15.203:0/449069117 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a414c660 con 0x7ff7a40602b0 2021-01-06T22:08:00.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a37fe700 1 --2- 172.21.15.203:0/449069117 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7a40602b0 0x7ff7a40c0fa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7ff79800b980 tx=0x7ff798006de0).ready entity=mon.0 client_cookie=1bb4060ea123cb68 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:00.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a2ffd700 1 -- 172.21.15.203:0/449069117 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798009840 con 0x7ff7a40602b0 2021-01-06T22:08:00.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a2ffd700 1 -- 172.21.15.203:0/449069117 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7980099a0 con 0x7ff7a40602b0 2021-01-06T22:08:00.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a2ffd700 1 -- 172.21.15.203:0/449069117 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798009d20 con 0x7ff7a40602b0 2021-01-06T22:08:00.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a956f700 1 -- 172.21.15.203:0/449069117 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7a40602b0 msgr2=0x7ff7a40c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:00.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a956f700 1 --2- 172.21.15.203:0/449069117 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7a40602b0 0x7ff7a40c0fa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7ff79800b980 tx=0x7ff798006de0).stop 2021-01-06T22:08:00.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a956f700 1 -- 172.21.15.203:0/449069117 shutdown_connections 2021-01-06T22:08:00.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a956f700 1 --2- 172.21.15.203:0/449069117 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7a40602b0 0x7ff7a40c0fa0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:00.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a956f700 1 -- 172.21.15.203:0/449069117 >> 172.21.15.203:0/449069117 conn(0x7ff7a40667b0 msgr2=0x7ff7a40c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:00.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a956f700 1 -- 172.21.15.203:0/449069117 shutdown_connections 2021-01-06T22:08:00.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a956f700 1 -- 172.21.15.203:0/449069117 wait complete. 2021-01-06T22:08:00.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a956f700 1 Processor -- start 2021-01-06T22:08:00.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.623+0000 7ff7a956f700 1 -- start start 2021-01-06T22:08:00.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a956f700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7a40602b0 0x7ff7a415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:00.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a956f700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff7a4148d70 0x7ff7a4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:00.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a956f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4149150 0x7ff7a40b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:00.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a956f700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a415fa90 con 0x7ff7a40602b0 2021-01-06T22:08:00.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a956f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a40b4360 con 0x7ff7a4149150 2021-01-06T22:08:00.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a956f700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a40ad460 con 0x7ff7a4148d70 2021-01-06T22:08:00.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a8d6e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4149150 0x7ff7a40b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:00.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a8d6e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4149150 0x7ff7a40b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50232/0 (socket says 172.21.15.203:50232) 2021-01-06T22:08:00.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a8d6e700 1 -- 172.21.15.203:0/1730179744 learned_addr learned my addr 172.21.15.203:0/1730179744 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:00.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a3fff700 1 --2- 172.21.15.203:0/1730179744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7a40602b0 0x7ff7a415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:00.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a37fe700 1 --2- 172.21.15.203:0/1730179744 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff7a4148d70 0x7ff7a4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:00.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a8d6e700 1 -- 172.21.15.203:0/1730179744 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff7a4148d70 msgr2=0x7ff7a4068040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:00.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a8d6e700 1 --2- 172.21.15.203:0/1730179744 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff7a4148d70 0x7ff7a4068040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:00.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a8d6e700 1 -- 172.21.15.203:0/1730179744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7a40602b0 msgr2=0x7ff7a415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:00.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a8d6e700 1 --2- 172.21.15.203:0/1730179744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7a40602b0 0x7ff7a415f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:00.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a8d6e700 1 -- 172.21.15.203:0/1730179744 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a414fab0 con 0x7ff7a4149150 2021-01-06T22:08:00.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a8d6e700 1 --2- 172.21.15.203:0/1730179744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4149150 0x7ff7a40b4d60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7ff79c00ca60 tx=0x7ff79c007c70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:00.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff78bfff700 1 -- 172.21.15.203:0/1730179744 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c00ce30 con 0x7ff7a4149150 2021-01-06T22:08:00.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff78bfff700 1 -- 172.21.15.203:0/1730179744 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79c005b70 con 0x7ff7a4149150 2021-01-06T22:08:00.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a956f700 1 -- 172.21.15.203:0/1730179744 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a414fd50 con 0x7ff7a4149150 2021-01-06T22:08:00.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff78bfff700 1 -- 172.21.15.203:0/1730179744 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c007110 con 0x7ff7a4149150 2021-01-06T22:08:00.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.627+0000 7ff7a956f700 1 -- 172.21.15.203:0/1730179744 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a4150290 con 0x7ff7a4149150 2021-01-06T22:08:00.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.631+0000 7ff7a956f700 1 -- 172.21.15.203:0/1730179744 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a40450a0 con 0x7ff7a4149150 2021-01-06T22:08:00.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.631+0000 7ff78bfff700 1 -- 172.21.15.203:0/1730179744 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff79c010030 con 0x7ff7a4149150 2021-01-06T22:08:00.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.631+0000 7ff78bfff700 1 --2- 172.21.15.203:0/1730179744 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff7840563d0 0x7ff784058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:00.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.631+0000 7ff78bfff700 1 -- 172.21.15.203:0/1730179744 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7ff79c06f2e0 con 0x7ff7a4149150 2021-01-06T22:08:00.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.631+0000 7ff7a3fff700 1 --2- 172.21.15.203:0/1730179744 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff7840563d0 0x7ff784058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:00.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.631+0000 7ff7a3fff700 1 --2- 172.21.15.203:0/1730179744 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff7840563d0 0x7ff784058890 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7ff7940035f0 tx=0x7ff794007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:00.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.635+0000 7ff78bfff700 1 -- 172.21.15.203:0/1730179744 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff79c044ab0 con 0x7ff7a4149150 2021-01-06T22:08:00.835 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.839+0000 7ff7a956f700 1 -- 172.21.15.203:0/1730179744 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7a41495a0 con 0x7ff7a4149150 2021-01-06T22:08:00.836 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.839+0000 7ff78bfff700 1 -- 172.21.15.203:0/1730179744 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3578 (secure 0 0 0) 0x7ff7a41495a0 con 0x7ff7a4149150 2021-01-06T22:08:00.836 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:00.837 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:07:47.224782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:00.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.843+0000 7ff7a956f700 1 -- 172.21.15.203:0/1730179744 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff7840563d0 msgr2=0x7ff784058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:00.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.843+0000 7ff7a956f700 1 --2- 172.21.15.203:0/1730179744 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff7840563d0 0x7ff784058890 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7ff7940035f0 tx=0x7ff794007b80).stop 2021-01-06T22:08:00.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.843+0000 7ff7a956f700 1 -- 172.21.15.203:0/1730179744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4149150 msgr2=0x7ff7a40b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:00.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.843+0000 7ff7a956f700 1 --2- 172.21.15.203:0/1730179744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4149150 0x7ff7a40b4d60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7ff79c00ca60 tx=0x7ff79c007c70).stop 2021-01-06T22:08:00.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.843+0000 7ff7a956f700 1 -- 172.21.15.203:0/1730179744 shutdown_connections 2021-01-06T22:08:00.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.843+0000 7ff7a956f700 1 --2- 172.21.15.203:0/1730179744 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff7840563d0 0x7ff784058890 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:00.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.843+0000 7ff7a956f700 1 --2- 172.21.15.203:0/1730179744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4149150 0x7ff7a40b4d60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:00.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.843+0000 7ff7a956f700 1 --2- 172.21.15.203:0/1730179744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff7a40602b0 0x7ff7a415f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:00.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.843+0000 7ff7a956f700 1 --2- 172.21.15.203:0/1730179744 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff7a4148d70 0x7ff7a4068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:00.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.843+0000 7ff7a956f700 1 -- 172.21.15.203:0/1730179744 >> 172.21.15.203:0/1730179744 conn(0x7ff7a40667b0 msgr2=0x7ff7a414dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:00.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.843+0000 7ff7a956f700 1 -- 172.21.15.203:0/1730179744 shutdown_connections 2021-01-06T22:08:00.842 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.843+0000 7ff7a956f700 1 -- 172.21.15.203:0/1730179744 wait complete. 2021-01-06T22:08:00.842 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 27 2021-01-06T22:08:00.850 INFO:tasks.mds_thrash.fs.[cephfs]:thrashing max_mds: 3 -> 2 2021-01-06T22:08:00.851 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2021-01-06T22:08:00.981 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f218515a700 1 Processor -- start 2021-01-06T22:08:00.982 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f218515a700 1 -- start start 2021-01-06T22:08:00.983 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f218515a700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21800633f0 0x7f218014e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:00.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f218515a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f21800be1b0 0x7f2180147790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:00.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f218515a700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21800be990 con 0x7f2180063010 2021-01-06T22:08:00.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f218515a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f218014ec60 con 0x7f21800be1b0 2021-01-06T22:08:00.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f218515a700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21800be6f0 con 0x7f21800633f0 2021-01-06T22:08:00.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f2184959700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f21800be1b0 0x7f2180147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:00.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f2184959700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f21800be1b0 0x7f2180147790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50240/0 (socket says 172.21.15.203:50240) 2021-01-06T22:08:00.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f2184959700 1 -- 172.21.15.203:0/2008888097 learned_addr learned my addr 172.21.15.203:0/2008888097 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:00.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f21777fe700 1 --2- 172.21.15.203:0/2008888097 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21800633f0 0x7f218014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:00.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f2184959700 1 -- 172.21.15.203:0/2008888097 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21800633f0 msgr2=0x7f218014e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:00.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f2184959700 1 --2- 172.21.15.203:0/2008888097 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21800633f0 0x7f218014e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:00.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f2184959700 1 -- 172.21.15.203:0/2008888097 >> v1:172.21.15.91:6789/0 conn(0x7f2180063010 legacy=0x7f2180064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:00.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f2184959700 1 -- 172.21.15.203:0/2008888097 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800c1050 con 0x7f21800be1b0 2021-01-06T22:08:00.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f21777fe700 1 --2- 172.21.15.203:0/2008888097 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21800633f0 0x7f218014e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:08:00.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f2184959700 1 --2- 172.21.15.203:0/2008888097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21800be1b0 0x7f2180147790 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f2178009d50 tx=0x7f2178009290).ready entity=mon.1 client_cookie=aa997a5b59fced22 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:00.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f217f7fe700 1 -- 172.21.15.203:0/2008888097 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217800bcd0 con 0x7f21800be1b0 2021-01-06T22:08:00.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f217f7fe700 1 -- 172.21.15.203:0/2008888097 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217800be30 con 0x7f21800be1b0 2021-01-06T22:08:00.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f217f7fe700 1 -- 172.21.15.203:0/2008888097 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178011610 con 0x7f21800be1b0 2021-01-06T22:08:00.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f218515a700 1 -- 172.21.15.203:0/2008888097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21800be1b0 msgr2=0x7f2180147790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:00.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f218515a700 1 --2- 172.21.15.203:0/2008888097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21800be1b0 0x7f2180147790 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f2178009d50 tx=0x7f2178009290).stop 2021-01-06T22:08:00.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f218515a700 1 -- 172.21.15.203:0/2008888097 shutdown_connections 2021-01-06T22:08:00.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f218515a700 1 --2- 172.21.15.203:0/2008888097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21800be1b0 0x7f2180147790 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:00.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f218515a700 1 --2- 172.21.15.203:0/2008888097 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21800633f0 0x7f218014e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:00.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f218515a700 1 -- 172.21.15.203:0/2008888097 >> 172.21.15.203:0/2008888097 conn(0x7f2180067e20 msgr2=0x7f2180068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:00.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f218515a700 1 -- 172.21.15.203:0/2008888097 shutdown_connections 2021-01-06T22:08:00.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.987+0000 7f218515a700 1 -- 172.21.15.203:0/2008888097 wait complete. 2021-01-06T22:08:00.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f218515a700 1 Processor -- start 2021-01-06T22:08:00.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f218515a700 1 -- start start 2021-01-06T22:08:00.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f218515a700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2180063010 0x7f2180148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:00.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f218515a700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21800633f0 0x7f2180142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:00.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f218515a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21800be1b0 0x7f21801430a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:00.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f218515a700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218013eed0 con 0x7f2180063010 2021-01-06T22:08:00.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f218515a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2180142520 con 0x7f21800be1b0 2021-01-06T22:08:00.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f218515a700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2180147c90 con 0x7f21800633f0 2021-01-06T22:08:00.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f2184959700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21800be1b0 0x7f21801430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:00.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f2184959700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21800be1b0 0x7f21801430a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50244/0 (socket says 172.21.15.203:50244) 2021-01-06T22:08:00.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f2184959700 1 -- 172.21.15.203:0/537168358 learned_addr learned my addr 172.21.15.203:0/537168358 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:01.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f217ffff700 1 --2- 172.21.15.203:0/537168358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2180063010 0x7f2180148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:01.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f21777fe700 1 --2- 172.21.15.203:0/537168358 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21800633f0 0x7f2180142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:01.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f2184959700 1 -- 172.21.15.203:0/537168358 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21800633f0 msgr2=0x7f2180142bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:01.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f2184959700 1 --2- 172.21.15.203:0/537168358 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21800633f0 0x7f2180142bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:01.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f2184959700 1 -- 172.21.15.203:0/537168358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2180063010 msgr2=0x7f2180148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:01.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f2184959700 1 --2- 172.21.15.203:0/537168358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2180063010 0x7f2180148810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:01.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f2184959700 1 -- 172.21.15.203:0/537168358 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218014d750 con 0x7f21800be1b0 2021-01-06T22:08:01.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f2184959700 1 --2- 172.21.15.203:0/537168358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21800be1b0 0x7f21801430a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f2180068a30 tx=0x7f2178017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:01.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f217cff9700 1 -- 172.21.15.203:0/537168358 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178009e90 con 0x7f21800be1b0 2021-01-06T22:08:01.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f217cff9700 1 -- 172.21.15.203:0/537168358 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2178011e70 con 0x7f21800be1b0 2021-01-06T22:08:01.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f217cff9700 1 -- 172.21.15.203:0/537168358 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178010d20 con 0x7f21800be1b0 2021-01-06T22:08:01.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f218515a700 1 -- 172.21.15.203:0/537168358 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218014d9b0 con 0x7f21800be1b0 2021-01-06T22:08:01.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.991+0000 7f218515a700 1 -- 172.21.15.203:0/537168358 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2180147f40 con 0x7f21800be1b0 2021-01-06T22:08:01.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.995+0000 7f217cff9700 1 -- 172.21.15.203:0/537168358 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f2178023030 con 0x7f21800be1b0 2021-01-06T22:08:01.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.995+0000 7f218515a700 1 -- 172.21.15.203:0/537168358 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2180149210 con 0x7f21800be1b0 2021-01-06T22:08:01.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.995+0000 7f217cff9700 1 --2- 172.21.15.203:0/537168358 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f21640563d0 0x7f2164058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:01.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.995+0000 7f217cff9700 1 -- 172.21.15.203:0/537168358 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f2178070a70 con 0x7f21800be1b0 2021-01-06T22:08:01.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.995+0000 7f217ffff700 1 --2- 172.21.15.203:0/537168358 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f21640563d0 0x7f2164058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:01.007 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.995+0000 7f217ffff700 1 --2- 172.21.15.203:0/537168358 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f21640563d0 0x7f2164058890 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f21700036c0 tx=0x7f2170007b60).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:01.008 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:00.999+0000 7f217cff9700 1 -- 172.21.15.203:0/537168358 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f21780462f0 con 0x7f21800be1b0 2021-01-06T22:08:01.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.223+0000 7f218515a700 1 -- 172.21.15.203:0/537168358 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f21800450a0 con 0x7f21800be1b0 2021-01-06T22:08:01.379 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.383+0000 7f217cff9700 1 -- 172.21.15.203:0/537168358 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v28) v1 ==== 105+0+0 (secure 0 0 0) 0x7f2178070e60 con 0x7f21800be1b0 2021-01-06T22:08:01.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.387+0000 7f218515a700 1 -- 172.21.15.203:0/537168358 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f21640563d0 msgr2=0x7f2164058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:01.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.387+0000 7f218515a700 1 --2- 172.21.15.203:0/537168358 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f21640563d0 0x7f2164058890 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f21700036c0 tx=0x7f2170007b60).stop 2021-01-06T22:08:01.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.387+0000 7f218515a700 1 -- 172.21.15.203:0/537168358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21800be1b0 msgr2=0x7f21801430a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:01.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.387+0000 7f218515a700 1 --2- 172.21.15.203:0/537168358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21800be1b0 0x7f21801430a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f2180068a30 tx=0x7f2178017000).stop 2021-01-06T22:08:01.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.387+0000 7f218515a700 1 -- 172.21.15.203:0/537168358 shutdown_connections 2021-01-06T22:08:01.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.387+0000 7f218515a700 1 --2- 172.21.15.203:0/537168358 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f21640563d0 0x7f2164058890 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:01.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.387+0000 7f218515a700 1 --2- 172.21.15.203:0/537168358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21800be1b0 0x7f21801430a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:01.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.387+0000 7f218515a700 1 --2- 172.21.15.203:0/537168358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2180063010 0x7f2180148810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:01.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.387+0000 7f218515a700 1 --2- 172.21.15.203:0/537168358 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21800633f0 0x7f2180142bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:01.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.387+0000 7f218515a700 1 -- 172.21.15.203:0/537168358 >> 172.21.15.203:0/537168358 conn(0x7f2180067e20 msgr2=0x7f21800b8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:01.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.387+0000 7f218515a700 1 -- 172.21.15.203:0/537168358 shutdown_connections 2021-01-06T22:08:01.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.387+0000 7f218515a700 1 -- 172.21.15.203:0/537168358 wait complete. 2021-01-06T22:08:01.399 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2021-01-06T22:08:01.399 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:01.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.523+0000 7fba8f59e700 1 Processor -- start 2021-01-06T22:08:01.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.523+0000 7fba8f59e700 1 -- start start 2021-01-06T22:08:01.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.523+0000 7fba8f59e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fba900602b0 0x7fba900c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:01.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.523+0000 7fba8f59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba9014f0a0 con 0x7fba90148d70 2021-01-06T22:08:01.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.523+0000 7fba8f59e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba901495a0 con 0x7fba90149150 2021-01-06T22:08:01.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.523+0000 7fba8f59e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba9014c3c0 con 0x7fba900602b0 2021-01-06T22:08:01.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.523+0000 7fba85d9b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fba900602b0 0x7fba900c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:01.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.523+0000 7fba85d9b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fba900602b0 0x7fba900c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50256/0 (socket says 172.21.15.203:50256) 2021-01-06T22:08:01.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.523+0000 7fba85d9b700 1 -- 172.21.15.203:0/4142190626 learned_addr learned my addr 172.21.15.203:0/4142190626 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:01.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.523+0000 7fba85d9b700 1 -- 172.21.15.203:0/4142190626 >> v1:172.21.15.203:6789/0 conn(0x7fba90149150 legacy=0x7fba9014bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:01.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.523+0000 7fba85d9b700 1 -- 172.21.15.203:0/4142190626 >> v1:172.21.15.91:6789/0 conn(0x7fba90148d70 legacy=0x7fba9014fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:01.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.523+0000 7fba85d9b700 1 -- 172.21.15.203:0/4142190626 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9014c660 con 0x7fba900602b0 2021-01-06T22:08:01.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.523+0000 7fba85d9b700 1 --2- 172.21.15.203:0/4142190626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba900602b0 0x7fba900c0fa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fba7800b9e0 tx=0x7fba78006ea0).ready entity=mon.1 client_cookie=f93bb5c29948e382 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:01.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.523+0000 7fba8dd9b700 1 -- 172.21.15.203:0/4142190626 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78009960 con 0x7fba900602b0 2021-01-06T22:08:01.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.523+0000 7fba8dd9b700 1 -- 172.21.15.203:0/4142190626 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba78009ac0 con 0x7fba900602b0 2021-01-06T22:08:01.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.523+0000 7fba8dd9b700 1 -- 172.21.15.203:0/4142190626 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78009e40 con 0x7fba900602b0 2021-01-06T22:08:01.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8f59e700 1 -- 172.21.15.203:0/4142190626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba900602b0 msgr2=0x7fba900c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:01.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8f59e700 1 --2- 172.21.15.203:0/4142190626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba900602b0 0x7fba900c0fa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fba7800b9e0 tx=0x7fba78006ea0).stop 2021-01-06T22:08:01.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8f59e700 1 -- 172.21.15.203:0/4142190626 shutdown_connections 2021-01-06T22:08:01.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8f59e700 1 --2- 172.21.15.203:0/4142190626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba900602b0 0x7fba900c0fa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:01.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8f59e700 1 -- 172.21.15.203:0/4142190626 >> 172.21.15.203:0/4142190626 conn(0x7fba900667b0 msgr2=0x7fba900c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:01.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8f59e700 1 -- 172.21.15.203:0/4142190626 shutdown_connections 2021-01-06T22:08:01.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8f59e700 1 -- 172.21.15.203:0/4142190626 wait complete. 2021-01-06T22:08:01.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8f59e700 1 Processor -- start 2021-01-06T22:08:01.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8f59e700 1 -- start start 2021-01-06T22:08:01.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8f59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fba900602b0 0x7fba900b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:01.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8f59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fba90148d70 0x7fba900b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:01.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8f59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba90149150 0x7fba900b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:01.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8f59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba900b9d90 con 0x7fba900602b0 2021-01-06T22:08:01.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8f59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba900b27a0 con 0x7fba90149150 2021-01-06T22:08:01.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8f59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba900673e0 con 0x7fba90148d70 2021-01-06T22:08:01.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8ed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba90149150 0x7fba900b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:01.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8ed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba90149150 0x7fba900b82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50264/0 (socket says 172.21.15.203:50264) 2021-01-06T22:08:01.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8ed9d700 1 -- 172.21.15.203:0/516078619 learned_addr learned my addr 172.21.15.203:0/516078619 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:01.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba85d9b700 1 --2- 172.21.15.203:0/516078619 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fba90148d70 0x7fba900b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:01.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8e59c700 1 --2- 172.21.15.203:0/516078619 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fba900602b0 0x7fba900b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:01.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8ed9d700 1 -- 172.21.15.203:0/516078619 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fba90148d70 msgr2=0x7fba900b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:01.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8ed9d700 1 --2- 172.21.15.203:0/516078619 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fba90148d70 0x7fba900b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:01.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8ed9d700 1 -- 172.21.15.203:0/516078619 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fba900602b0 msgr2=0x7fba900b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:01.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8ed9d700 1 --2- 172.21.15.203:0/516078619 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fba900602b0 0x7fba900b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:01.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8ed9d700 1 -- 172.21.15.203:0/516078619 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba900686d0 con 0x7fba90149150 2021-01-06T22:08:01.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba8ed9d700 1 --2- 172.21.15.203:0/516078619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba90149150 0x7fba900b82e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fba8800cac0 tx=0x7fba88007f90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:01.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.527+0000 7fba86ffd700 1 -- 172.21.15.203:0/516078619 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8800a7c0 con 0x7fba90149150 2021-01-06T22:08:01.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.531+0000 7fba86ffd700 1 -- 172.21.15.203:0/516078619 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba88003030 con 0x7fba90149150 2021-01-06T22:08:01.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.531+0000 7fba86ffd700 1 -- 172.21.15.203:0/516078619 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8800ab50 con 0x7fba90149150 2021-01-06T22:08:01.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.531+0000 7fba8f59e700 1 -- 172.21.15.203:0/516078619 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba900b2e40 con 0x7fba90149150 2021-01-06T22:08:01.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.531+0000 7fba8f59e700 1 -- 172.21.15.203:0/516078619 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba900b93b0 con 0x7fba90149150 2021-01-06T22:08:01.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.531+0000 7fba8f59e700 1 -- 172.21.15.203:0/516078619 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba900450a0 con 0x7fba90149150 2021-01-06T22:08:01.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.531+0000 7fba86ffd700 1 -- 172.21.15.203:0/516078619 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fba88019030 con 0x7fba90149150 2021-01-06T22:08:01.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.531+0000 7fba86ffd700 1 --2- 172.21.15.203:0/516078619 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fba740563d0 0x7fba74058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:01.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.531+0000 7fba86ffd700 1 -- 172.21.15.203:0/516078619 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fba8806eb40 con 0x7fba90149150 2021-01-06T22:08:01.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.531+0000 7fba8e59c700 1 --2- 172.21.15.203:0/516078619 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fba740563d0 0x7fba74058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:01.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.531+0000 7fba8e59c700 1 --2- 172.21.15.203:0/516078619 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fba740563d0 0x7fba74058890 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fba800035f0 tx=0x7fba80007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:01.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.539+0000 7fba86ffd700 1 -- 172.21.15.203:0/516078619 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fba88044310 con 0x7fba90149150 2021-01-06T22:08:01.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.787+0000 7fba8f59e700 1 -- 172.21.15.203:0/516078619 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba901495a0 con 0x7fba90149150 2021-01-06T22:08:01.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.787+0000 7fba86ffd700 1 -- 172.21.15.203:0/516078619 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3580 (secure 0 0 0) 0x7fba901495a0 con 0x7fba90149150 2021-01-06T22:08:01.783 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:01.783 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:01.382197+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:stopping","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:01.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.787+0000 7fba8f59e700 1 -- 172.21.15.203:0/516078619 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fba740563d0 msgr2=0x7fba74058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:01.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.787+0000 7fba8f59e700 1 --2- 172.21.15.203:0/516078619 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fba740563d0 0x7fba74058890 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fba800035f0 tx=0x7fba80007b80).stop 2021-01-06T22:08:01.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.787+0000 7fba8f59e700 1 -- 172.21.15.203:0/516078619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba90149150 msgr2=0x7fba900b82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:01.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.787+0000 7fba8f59e700 1 --2- 172.21.15.203:0/516078619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba90149150 0x7fba900b82e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fba8800cac0 tx=0x7fba88007f90).stop 2021-01-06T22:08:01.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.787+0000 7fba8f59e700 1 -- 172.21.15.203:0/516078619 shutdown_connections 2021-01-06T22:08:01.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.787+0000 7fba8f59e700 1 --2- 172.21.15.203:0/516078619 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fba740563d0 0x7fba74058890 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:01.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.787+0000 7fba8f59e700 1 --2- 172.21.15.203:0/516078619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba90149150 0x7fba900b82e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:01.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.787+0000 7fba8f59e700 1 --2- 172.21.15.203:0/516078619 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fba900602b0 0x7fba900b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:01.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.787+0000 7fba8f59e700 1 --2- 172.21.15.203:0/516078619 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fba90148d70 0x7fba900b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:01.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.787+0000 7fba8f59e700 1 -- 172.21.15.203:0/516078619 >> 172.21.15.203:0/516078619 conn(0x7fba900667b0 msgr2=0x7fba900b9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:01.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.787+0000 7fba8f59e700 1 -- 172.21.15.203:0/516078619 shutdown_connections 2021-01-06T22:08:01.787 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:01.791+0000 7fba8f59e700 1 -- 172.21.15.203:0/516078619 wait complete. 2021-01-06T22:08:01.787 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 28 2021-01-06T22:08:01.797 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2021-01-06T22:08:01.798 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 28, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 28, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4273": { "addr": "172.21.15.203:6837/3277561222", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6836", "nonce": 3277561222, "type": "v2" }, { "addr": "172.21.15.203:6837", "nonce": 3277561222, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4273, "incarnation": 9, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 3 }, "gid_4274": { "addr": "172.21.15.91:6837/3888667247", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6835", "nonce": 3888667247, "type": "v2" }, { "addr": "172.21.15.91:6837", "nonce": 3888667247, "type": "v1" } ] }, "export_targets": [ 2 ], "features": 4540138297136906239, "flags": 0, "gid": 4274, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 2 }, "gid_4867": { "addr": "172.21.15.203:6835/3382231276", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6834", "nonce": 3382231276, "type": "v2" }, { "addr": "172.21.15.203:6835", "nonce": 3382231276, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4867, "incarnation": 20, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:stopping", "state_seq": 22 } }, "last_failure": 0, "last_failure_osd_epoch": 25, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2021-01-06T22:08:01.382197+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4274, "mds_1": 4273, "mds_2": 4867 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.91:6836/2428882934", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6834", "nonce": 2428882934, "type": "v2" }, { "addr": "172.21.15.91:6836", "nonce": 2428882934, "type": "v1" } ] }, "epoch": 26, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4724, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2021-01-06T22:08:03.799 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:03.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc7fff700 1 Processor -- start 2021-01-06T22:08:03.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc7fff700 1 -- start start 2021-01-06T22:08:03.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc7fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7dc81513f0 0x7f7dc80c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:03.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc7fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7dc814ad90 0x7f7dc814b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:03.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc7fff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc80bacb0 con 0x7f7dc80bccb0 2021-01-06T22:08:03.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc7fff700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc80bd160 con 0x7f7dc81513f0 2021-01-06T22:08:03.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc7fff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc814b6d0 con 0x7f7dc814ad90 2021-01-06T22:08:03.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc6ffd700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7dc81513f0 0x7f7dc80c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:03.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc6ffd700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7dc81513f0 0x7f7dc80c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:58118/0 (socket says 172.21.15.203:58118) 2021-01-06T22:08:03.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc77fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7dc814ad90 0x7f7dc814b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:03.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc6ffd700 1 -- 172.21.15.203:0/1516728192 learned_addr learned my addr 172.21.15.203:0/1516728192 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:03.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc67fc700 1 --1- 172.21.15.203:0/1516728192 >> v1:172.21.15.91:6789/0 conn(0x7f7dc80bccb0 0x7f7dc81517d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:43520/0 (socket says 172.21.15.203:43520) 2021-01-06T22:08:03.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc6ffd700 1 -- 172.21.15.203:0/1516728192 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7dc814ad90 msgr2=0x7f7dc814b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:03.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc6ffd700 1 --2- 172.21.15.203:0/1516728192 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7dc814ad90 0x7f7dc814b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:03.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc6ffd700 1 -- 172.21.15.203:0/1516728192 >> v1:172.21.15.91:6789/0 conn(0x7f7dc80bccb0 legacy=0x7f7dc81517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:03.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc5ffb700 1 -- 172.21.15.203:0/1516728192 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2683427151 0 0) 0x7f7dbc0010d0 con 0x7f7dc80bccb0 2021-01-06T22:08:03.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc6ffd700 1 -- 172.21.15.203:0/1516728192 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc8152000 con 0x7f7dc81513f0 2021-01-06T22:08:03.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc77fe700 1 --2- 172.21.15.203:0/1516728192 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7dc814ad90 0x7f7dc814b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:08:03.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc6ffd700 1 --2- 172.21.15.203:0/1516728192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dc81513f0 0x7f7dc80c0fa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f7db800ba10 tx=0x7f7db8006ea0).ready entity=mon.0 client_cookie=1b402d4437b780e8 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:03.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc5ffb700 1 -- 172.21.15.203:0/1516728192 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db80097d0 con 0x7f7dc81513f0 2021-01-06T22:08:03.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc5ffb700 1 -- 172.21.15.203:0/1516728192 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db8009930 con 0x7f7dc81513f0 2021-01-06T22:08:03.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.935+0000 7f7dc5ffb700 1 -- 172.21.15.203:0/1516728192 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db8009cb0 con 0x7f7dc81513f0 2021-01-06T22:08:03.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.939+0000 7f7dc7fff700 1 -- 172.21.15.203:0/1516728192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dc81513f0 msgr2=0x7f7dc80c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:03.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.939+0000 7f7dc7fff700 1 --2- 172.21.15.203:0/1516728192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dc81513f0 0x7f7dc80c0fa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f7db800ba10 tx=0x7f7db8006ea0).stop 2021-01-06T22:08:03.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.939+0000 7f7dc7fff700 1 -- 172.21.15.203:0/1516728192 shutdown_connections 2021-01-06T22:08:03.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.939+0000 7f7dc7fff700 1 --2- 172.21.15.203:0/1516728192 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7dc814ad90 0x7f7dc814b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:03.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.939+0000 7f7dc7fff700 1 --2- 172.21.15.203:0/1516728192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dc81513f0 0x7f7dc80c0fa0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:03.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.939+0000 7f7dc7fff700 1 -- 172.21.15.203:0/1516728192 >> 172.21.15.203:0/1516728192 conn(0x7f7dc80667b0 msgr2=0x7f7dc80c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:03.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.939+0000 7f7dc7fff700 1 -- 172.21.15.203:0/1516728192 shutdown_connections 2021-01-06T22:08:03.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.939+0000 7f7dc7fff700 1 -- 172.21.15.203:0/1516728192 wait complete. 2021-01-06T22:08:03.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.939+0000 7f7dc7fff700 1 Processor -- start 2021-01-06T22:08:03.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.939+0000 7f7dc7fff700 1 -- start start 2021-01-06T22:08:03.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc7fff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dc80bccb0 0x7f7dc80b8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:03.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dc814ad90 0x7f7dc80bfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:03.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc7fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7dc81513f0 0x7f7dc814f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:03.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc7fff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc8161960 con 0x7f7dc80bccb0 2021-01-06T22:08:03.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc7fff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc8160ab0 con 0x7f7dc814ad90 2021-01-06T22:08:03.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc7fff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc80b8be0 con 0x7f7dc81513f0 2021-01-06T22:08:03.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc67fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dc814ad90 0x7f7dc80bfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:03.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc67fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dc814ad90 0x7f7dc80bfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50278/0 (socket says 172.21.15.203:50278) 2021-01-06T22:08:03.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc67fc700 1 -- 172.21.15.203:0/3575170045 learned_addr learned my addr 172.21.15.203:0/3575170045 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:03.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc77fe700 1 --2- 172.21.15.203:0/3575170045 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7dc81513f0 0x7f7dc814f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:03.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc6ffd700 1 --2- 172.21.15.203:0/3575170045 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dc80bccb0 0x7f7dc80b8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:03.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc67fc700 1 -- 172.21.15.203:0/3575170045 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7dc81513f0 msgr2=0x7f7dc814f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:03.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc67fc700 1 --2- 172.21.15.203:0/3575170045 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7dc81513f0 0x7f7dc814f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:03.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc67fc700 1 -- 172.21.15.203:0/3575170045 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dc80bccb0 msgr2=0x7f7dc80b8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:03.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc67fc700 1 --2- 172.21.15.203:0/3575170045 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dc80bccb0 0x7f7dc80b8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:03.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc67fc700 1 -- 172.21.15.203:0/3575170045 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc8160ce0 con 0x7f7dc814ad90 2021-01-06T22:08:03.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc67fc700 1 --2- 172.21.15.203:0/3575170045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dc814ad90 0x7f7dc80bfe30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f7dbc00c770 tx=0x7f7dbc001790).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:03.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7daeffd700 1 -- 172.21.15.203:0/3575170045 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc0072e0 con 0x7f7dc814ad90 2021-01-06T22:08:03.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7daeffd700 1 -- 172.21.15.203:0/3575170045 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc005520 con 0x7f7dc814ad90 2021-01-06T22:08:03.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc7fff700 1 -- 172.21.15.203:0/3575170045 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc8160f80 con 0x7f7dc814ad90 2021-01-06T22:08:03.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7daeffd700 1 -- 172.21.15.203:0/3575170045 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc0058e0 con 0x7f7dc814ad90 2021-01-06T22:08:03.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc7fff700 1 -- 172.21.15.203:0/3575170045 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc81614c0 con 0x7f7dc814ad90 2021-01-06T22:08:03.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.943+0000 7f7dc7fff700 1 -- 172.21.15.203:0/3575170045 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc80450a0 con 0x7f7dc814ad90 2021-01-06T22:08:03.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.947+0000 7f7daeffd700 1 -- 172.21.15.203:0/3575170045 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f7dbc010070 con 0x7f7dc814ad90 2021-01-06T22:08:03.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.947+0000 7f7daeffd700 1 --2- 172.21.15.203:0/3575170045 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7da80563d0 0x7f7da8058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:03.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.947+0000 7f7daeffd700 1 -- 172.21.15.203:0/3575170045 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f7dbc06eae0 con 0x7f7dc814ad90 2021-01-06T22:08:03.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.947+0000 7f7dc6ffd700 1 --2- 172.21.15.203:0/3575170045 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7da80563d0 0x7f7da8058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:03.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.947+0000 7f7dc6ffd700 1 --2- 172.21.15.203:0/3575170045 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7da80563d0 0x7f7da8058890 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f7db8007fd0 tx=0x7f7db8016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:03.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:03.951+0000 7f7daeffd700 1 -- 172.21.15.203:0/3575170045 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f7dbc0442b0 con 0x7f7dc814ad90 2021-01-06T22:08:04.162 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:04.167+0000 7f7dc7fff700 1 -- 172.21.15.203:0/3575170045 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7dc80bac80 con 0x7f7dc814ad90 2021-01-06T22:08:04.163 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:04.167+0000 7f7daeffd700 1 -- 172.21.15.203:0/3575170045 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3580 (secure 0 0 0) 0x7f7dbc018030 con 0x7f7dc814ad90 2021-01-06T22:08:04.163 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:04.163 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:01.382197+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:stopping","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:04.166 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:04.171+0000 7f7dc7fff700 1 -- 172.21.15.203:0/3575170045 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7da80563d0 msgr2=0x7f7da8058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:04.167 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:04.171+0000 7f7dc7fff700 1 --2- 172.21.15.203:0/3575170045 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7da80563d0 0x7f7da8058890 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f7db8007fd0 tx=0x7f7db8016000).stop 2021-01-06T22:08:04.167 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:04.171+0000 7f7dc7fff700 1 -- 172.21.15.203:0/3575170045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dc814ad90 msgr2=0x7f7dc80bfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:04.167 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:04.171+0000 7f7dc7fff700 1 --2- 172.21.15.203:0/3575170045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dc814ad90 0x7f7dc80bfe30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f7dbc00c770 tx=0x7f7dbc001790).stop 2021-01-06T22:08:04.167 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:04.171+0000 7f7dc7fff700 1 -- 172.21.15.203:0/3575170045 shutdown_connections 2021-01-06T22:08:04.168 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:04.171+0000 7f7dc7fff700 1 --2- 172.21.15.203:0/3575170045 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7da80563d0 0x7f7da8058890 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:04.168 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:04.171+0000 7f7dc7fff700 1 --2- 172.21.15.203:0/3575170045 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7dc81513f0 0x7f7dc814f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:04.168 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:04.171+0000 7f7dc7fff700 1 --2- 172.21.15.203:0/3575170045 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dc80bccb0 0x7f7dc80b8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:04.168 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:04.171+0000 7f7dc7fff700 1 --2- 172.21.15.203:0/3575170045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dc814ad90 0x7f7dc80bfe30 secure :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0x7f7dbc00c770 tx=0x7f7dbc001790).stop 2021-01-06T22:08:04.168 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:04.171+0000 7f7dc7fff700 1 -- 172.21.15.203:0/3575170045 >> 172.21.15.203:0/3575170045 conn(0x7f7dc80667b0 msgr2=0x7f7dc8064c60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:04.169 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:04.171+0000 7f7dc7fff700 1 -- 172.21.15.203:0/3575170045 shutdown_connections 2021-01-06T22:08:04.169 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:04.171+0000 7f7dc7fff700 1 -- 172.21.15.203:0/3575170045 wait complete. 2021-01-06T22:08:04.169 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 28 2021-01-06T22:08:04.181 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:08:06.182 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:06.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f844792e700 1 Processor -- start 2021-01-06T22:08:06.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f844792e700 1 -- start start 2021-01-06T22:08:06.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f844792e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f84400be0c0 0x7f8440062e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:06.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f844792e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f84400be4a0 0x7f844014af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:06.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f844792e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f844014b3f0 0x7f844014e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:06.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f844792e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f844014b8a0 con 0x7f844014b3f0 2021-01-06T22:08:06.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f844792e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84400578d0 con 0x7f84400be0c0 2021-01-06T22:08:06.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f844792e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84401517e0 con 0x7f84400be4a0 2021-01-06T22:08:06.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f84456ca700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f84400be0c0 0x7f8440062e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:06.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f84456ca700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f84400be0c0 0x7f8440062e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50284/0 (socket says 172.21.15.203:50284) 2021-01-06T22:08:06.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f84456ca700 1 -- 172.21.15.203:0/3877738450 learned_addr learned my addr 172.21.15.203:0/3877738450 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:06.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f8444ec9700 1 --2- 172.21.15.203:0/3877738450 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f84400be4a0 0x7f844014af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:06.316 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f8445ecb700 1 --2- 172.21.15.203:0/3877738450 >> v2:172.21.15.91:3300/0 conn(0x7f844014b3f0 0x7f844014e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:06.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f84456ca700 1 -- 172.21.15.203:0/3877738450 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f84400be4a0 msgr2=0x7f844014af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:06.320 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f84456ca700 1 --2- 172.21.15.203:0/3877738450 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f84400be4a0 0x7f844014af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:06.320 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f84456ca700 1 -- 172.21.15.203:0/3877738450 >> v2:172.21.15.91:3300/0 conn(0x7f844014b3f0 msgr2=0x7f844014e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:06.320 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f84456ca700 1 --2- 172.21.15.203:0/3877738450 >> v2:172.21.15.91:3300/0 conn(0x7f844014b3f0 0x7f844014e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:06.320 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f84456ca700 1 -- 172.21.15.203:0/3877738450 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8440151a80 con 0x7f84400be0c0 2021-01-06T22:08:06.320 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f8445ecb700 1 --2- 172.21.15.203:0/3877738450 >> v2:172.21.15.91:3300/0 conn(0x7f844014b3f0 0x7f844014e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:08:06.321 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f84456ca700 1 --2- 172.21.15.203:0/3877738450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84400be0c0 0x7f8440062e50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f8440066f10 tx=0x7f8430006ea0).ready entity=mon.1 client_cookie=4f1d614ae748d86e server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:06.321 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f842ffff700 1 -- 172.21.15.203:0/3877738450 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430009810 con 0x7f84400be0c0 2021-01-06T22:08:06.321 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f842ffff700 1 -- 172.21.15.203:0/3877738450 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8430009970 con 0x7f84400be0c0 2021-01-06T22:08:06.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f842ffff700 1 -- 172.21.15.203:0/3877738450 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430009cf0 con 0x7f84400be0c0 2021-01-06T22:08:06.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f844792e700 1 -- 172.21.15.203:0/3877738450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84400be0c0 msgr2=0x7f8440062e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:06.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.319+0000 7f844792e700 1 --2- 172.21.15.203:0/3877738450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84400be0c0 0x7f8440062e50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f8440066f10 tx=0x7f8430006ea0).stop 2021-01-06T22:08:06.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.323+0000 7f844792e700 1 -- 172.21.15.203:0/3877738450 shutdown_connections 2021-01-06T22:08:06.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.323+0000 7f844792e700 1 --2- 172.21.15.203:0/3877738450 >> v2:172.21.15.91:3300/0 conn(0x7f844014b3f0 0x7f844014e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:06.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.323+0000 7f844792e700 1 --2- 172.21.15.203:0/3877738450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84400be0c0 0x7f8440062e50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:06.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.323+0000 7f844792e700 1 --2- 172.21.15.203:0/3877738450 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f84400be4a0 0x7f844014af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:06.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.323+0000 7f844792e700 1 -- 172.21.15.203:0/3877738450 >> 172.21.15.203:0/3877738450 conn(0x7f8440056cd0 msgr2=0x7f84400570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:06.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.323+0000 7f844792e700 1 -- 172.21.15.203:0/3877738450 shutdown_connections 2021-01-06T22:08:06.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.323+0000 7f844792e700 1 -- 172.21.15.203:0/3877738450 wait complete. 2021-01-06T22:08:06.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.323+0000 7f844792e700 1 Processor -- start 2021-01-06T22:08:06.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.323+0000 7f844792e700 1 -- start start 2021-01-06T22:08:06.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f844792e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84400be0c0 0x7f844015e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:06.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f844792e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84400be4a0 0x7f84401614e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:06.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f844792e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f844014b3f0 0x7f8440149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:06.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f844792e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844014d360 con 0x7f84400be4a0 2021-01-06T22:08:06.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f844792e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8440160b70 con 0x7f84400be0c0 2021-01-06T22:08:06.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f844792e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84400b8b50 con 0x7f844014b3f0 2021-01-06T22:08:06.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f84456ca700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84400be0c0 0x7f844015e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:06.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f84456ca700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84400be0c0 0x7f844015e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50290/0 (socket says 172.21.15.203:50290) 2021-01-06T22:08:06.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f84456ca700 1 -- 172.21.15.203:0/1837121084 learned_addr learned my addr 172.21.15.203:0/1837121084 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:06.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f8444ec9700 1 --2- 172.21.15.203:0/1837121084 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84400be4a0 0x7f84401614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:06.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f8445ecb700 1 --2- 172.21.15.203:0/1837121084 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f844014b3f0 0x7f8440149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:06.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f84456ca700 1 -- 172.21.15.203:0/1837121084 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f844014b3f0 msgr2=0x7f8440149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:06.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f84456ca700 1 --2- 172.21.15.203:0/1837121084 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f844014b3f0 0x7f8440149e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:06.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f84456ca700 1 -- 172.21.15.203:0/1837121084 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84400be4a0 msgr2=0x7f84401614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:06.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f84456ca700 1 --2- 172.21.15.203:0/1837121084 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84400be4a0 0x7f84401614e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:06.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f84456ca700 1 -- 172.21.15.203:0/1837121084 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84400b8df0 con 0x7f84400be0c0 2021-01-06T22:08:06.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f8445ecb700 1 --2- 172.21.15.203:0/1837121084 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f844014b3f0 0x7f8440149e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:08:06.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f84456ca700 1 --2- 172.21.15.203:0/1837121084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84400be0c0 0x7f844015e040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f843000b190 tx=0x7f8430016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:06.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f842d7fa700 1 -- 172.21.15.203:0/1837121084 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430012d60 con 0x7f84400be0c0 2021-01-06T22:08:06.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f842d7fa700 1 -- 172.21.15.203:0/1837121084 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843001c420 con 0x7f84400be0c0 2021-01-06T22:08:06.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f844792e700 1 -- 172.21.15.203:0/1837121084 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84400b9090 con 0x7f84400be0c0 2021-01-06T22:08:06.342 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f842d7fa700 1 -- 172.21.15.203:0/1837121084 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843001c7e0 con 0x7f84400be0c0 2021-01-06T22:08:06.342 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f844792e700 1 -- 172.21.15.203:0/1837121084 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8440161040 con 0x7f84400be0c0 2021-01-06T22:08:06.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f842d7fa700 1 -- 172.21.15.203:0/1837121084 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f843000f070 con 0x7f84400be0c0 2021-01-06T22:08:06.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f844792e700 1 -- 172.21.15.203:0/1837121084 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8440161200 con 0x7f84400be0c0 2021-01-06T22:08:06.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f842d7fa700 1 --2- 172.21.15.203:0/1837121084 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f84200563d0 0x7f8420058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:06.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f842d7fa700 1 -- 172.21.15.203:0/1837121084 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f8430070d00 con 0x7f84400be0c0 2021-01-06T22:08:06.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.327+0000 7f8444ec9700 1 --2- 172.21.15.203:0/1837121084 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f84200563d0 0x7f8420058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:06.345 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.331+0000 7f8444ec9700 1 --2- 172.21.15.203:0/1837121084 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f84200563d0 0x7f8420058890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f843c00bb50 tx=0x7f843c006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:06.345 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.335+0000 7f842d7fa700 1 -- 172.21.15.203:0/1837121084 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f8430074030 con 0x7f84400be0c0 2021-01-06T22:08:06.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.559+0000 7f844792e700 1 -- 172.21.15.203:0/1837121084 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f844005cfe0 con 0x7f84400be0c0 2021-01-06T22:08:06.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.559+0000 7f842d7fa700 1 -- 172.21.15.203:0/1837121084 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+3579 (secure 0 0 0) 0x7f84300464d0 con 0x7f84400be0c0 2021-01-06T22:08:06.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.563+0000 7f844792e700 1 -- 172.21.15.203:0/1837121084 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f84200563d0 msgr2=0x7f8420058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:06.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.563+0000 7f844792e700 1 --2- 172.21.15.203:0/1837121084 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f84200563d0 0x7f8420058890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f843c00bb50 tx=0x7f843c006de0).stop 2021-01-06T22:08:06.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.563+0000 7f844792e700 1 -- 172.21.15.203:0/1837121084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84400be0c0 msgr2=0x7f844015e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:06.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.563+0000 7f844792e700 1 --2- 172.21.15.203:0/1837121084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84400be0c0 0x7f844015e040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f843000b190 tx=0x7f8430016000).stop 2021-01-06T22:08:06.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.563+0000 7f844792e700 1 -- 172.21.15.203:0/1837121084 shutdown_connections 2021-01-06T22:08:06.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.563+0000 7f844792e700 1 --2- 172.21.15.203:0/1837121084 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f84200563d0 0x7f8420058890 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:06.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.563+0000 7f844792e700 1 --2- 172.21.15.203:0/1837121084 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f844014b3f0 0x7f8440149e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:06.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.563+0000 7f844792e700 1 --2- 172.21.15.203:0/1837121084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84400be0c0 0x7f844015e040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:06.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.563+0000 7f844792e700 1 --2- 172.21.15.203:0/1837121084 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84400be4a0 0x7f84401614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:06.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.563+0000 7f844792e700 1 -- 172.21.15.203:0/1837121084 >> 172.21.15.203:0/1837121084 conn(0x7f8440056cd0 msgr2=0x7f84400bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:06.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.563+0000 7f844792e700 1 -- 172.21.15.203:0/1837121084 shutdown_connections 2021-01-06T22:08:06.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:06.563+0000 7f844792e700 1 -- 172.21.15.203:0/1837121084 wait complete. 2021-01-06T22:08:06.869 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 29 2021-01-06T22:08:06.869 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:06.870 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:06.426663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:stopping","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:06.870 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:08:08.871 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:09.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.019+0000 7fef71a4d700 1 Processor -- start 2021-01-06T22:08:09.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.019+0000 7fef71a4d700 1 -- start start 2021-01-06T22:08:09.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.019+0000 7fef71a4d700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fef6c0be350 0x7fef6c0657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:09.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.019+0000 7fef71a4d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef6c14ba00 0x7fef6c14be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:09.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.019+0000 7fef71a4d700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6c14ee80 con 0x7fef6c0be730 2021-01-06T22:08:09.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.019+0000 7fef71a4d700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6c14f120 con 0x7fef6c0be350 2021-01-06T22:08:09.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.019+0000 7fef71a4d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c14f3c0 con 0x7fef6c14ba00 2021-01-06T22:08:09.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef6bfff700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fef6c0be730 0x7fef6c0ba350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:43548/0 (socket says 172.21.15.203:43548) 2021-01-06T22:08:09.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef6bfff700 1 -- 172.21.15.203:0/3048103820 learned_addr learned my addr 172.21.15.203:0/3048103820 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:09.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef7124c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef6c14ba00 0x7fef6c14be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:09.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef70a4b700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fef6c0be350 0x7fef6c0657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:09.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef70a4b700 1 -- 172.21.15.203:0/3048103820 >> v2:172.21.15.91:3300/0 conn(0x7fef6c0be350 msgr2=0x7fef6c0657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2021-01-06T22:08:09.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef70a4b700 1 -- 172.21.15.203:0/3048103820 >> v2:172.21.15.91:3300/0 conn(0x7fef6c0be350 msgr2=0x7fef6c0657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:08:09.083 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef70a4b700 1 --2- 172.21.15.203:0/3048103820 >> v2:172.21.15.91:3300/0 conn(0x7fef6c0be350 0x7fef6c0657c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:08:09.083 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef70a4b700 1 --2- 172.21.15.203:0/3048103820 >> v2:172.21.15.91:3300/0 conn(0x7fef6c0be350 0x7fef6c0657c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-01-06T22:08:09.083 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef6b7fe700 1 -- 172.21.15.203:0/3048103820 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220792378 0 0) 0x7fef6c14ee80 con 0x7fef6c0be730 2021-01-06T22:08:09.084 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef6b7fe700 1 -- 172.21.15.203:0/3048103820 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef50003230 con 0x7fef6c0be730 2021-01-06T22:08:09.084 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef7124c700 1 -- 172.21.15.203:0/3048103820 >> v2:172.21.15.91:3300/0 conn(0x7fef6c0be350 msgr2=0x7fef6c0657c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-01-06T22:08:09.084 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef7124c700 1 --2- 172.21.15.203:0/3048103820 >> v2:172.21.15.91:3300/0 conn(0x7fef6c0be350 0x7fef6c0657c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:09.084 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef7124c700 1 -- 172.21.15.203:0/3048103820 >> v1:172.21.15.91:6789/0 conn(0x7fef6c0be730 legacy=0x7fef6c0ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:09.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef7124c700 1 -- 172.21.15.203:0/3048103820 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c14f660 con 0x7fef6c14ba00 2021-01-06T22:08:09.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef7124c700 1 --2- 172.21.15.203:0/3048103820 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef6c14ba00 0x7fef6c14be60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fef64009c90 tx=0x7fef640091d0).ready entity=mon.2 client_cookie=653d0037fac06847 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:09.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef6b7fe700 1 -- 172.21.15.203:0/3048103820 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6400ba70 con 0x7fef6c14ba00 2021-01-06T22:08:09.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef6b7fe700 1 -- 172.21.15.203:0/3048103820 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6400bbd0 con 0x7fef6c14ba00 2021-01-06T22:08:09.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef6b7fe700 1 -- 172.21.15.203:0/3048103820 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64011400 con 0x7fef6c14ba00 2021-01-06T22:08:09.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef71a4d700 1 -- 172.21.15.203:0/3048103820 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef6c14ba00 msgr2=0x7fef6c14be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:09.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef71a4d700 1 --2- 172.21.15.203:0/3048103820 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef6c14ba00 0x7fef6c14be60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fef64009c90 tx=0x7fef640091d0).stop 2021-01-06T22:08:09.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef71a4d700 1 -- 172.21.15.203:0/3048103820 shutdown_connections 2021-01-06T22:08:09.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef71a4d700 1 --2- 172.21.15.203:0/3048103820 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef6c14ba00 0x7fef6c14be60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:09.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef71a4d700 1 --2- 172.21.15.203:0/3048103820 >> v2:172.21.15.91:3300/0 conn(0x7fef6c0be350 0x7fef6c0657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:09.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef71a4d700 1 -- 172.21.15.203:0/3048103820 >> 172.21.15.203:0/3048103820 conn(0x7fef6c067e20 msgr2=0x7fef6c068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:09.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef71a4d700 1 -- 172.21.15.203:0/3048103820 shutdown_connections 2021-01-06T22:08:09.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.023+0000 7fef71a4d700 1 -- 172.21.15.203:0/3048103820 wait complete. 2021-01-06T22:08:09.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef71a4d700 1 Processor -- start 2021-01-06T22:08:09.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef71a4d700 1 -- start start 2021-01-06T22:08:09.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef71a4d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef6c0be350 0x7fef6c13eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:09.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef71a4d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef6c0be730 0x7fef6c13f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:09.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef71a4d700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef6c14ba00 0x7fef6c140a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:09.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef71a4d700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c152050 con 0x7fef6c14ba00 2021-01-06T22:08:09.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef71a4d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c151b10 con 0x7fef6c0be350 2021-01-06T22:08:09.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef71a4d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c151db0 con 0x7fef6c0be730 2021-01-06T22:08:09.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef70a4b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef6c0be350 0x7fef6c13eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:09.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef70a4b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef6c0be350 0x7fef6c13eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50304/0 (socket says 172.21.15.203:50304) 2021-01-06T22:08:09.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef70a4b700 1 -- 172.21.15.203:0/3138288873 learned_addr learned my addr 172.21.15.203:0/3138288873 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:09.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef6bfff700 1 --2- 172.21.15.203:0/3138288873 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef6c0be730 0x7fef6c13f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:09.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef7124c700 1 --2- 172.21.15.203:0/3138288873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef6c14ba00 0x7fef6c140a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:09.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef70a4b700 1 -- 172.21.15.203:0/3138288873 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef6c0be730 msgr2=0x7fef6c13f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:09.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef70a4b700 1 --2- 172.21.15.203:0/3138288873 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef6c0be730 0x7fef6c13f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:09.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef70a4b700 1 -- 172.21.15.203:0/3138288873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef6c14ba00 msgr2=0x7fef6c140a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:09.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef70a4b700 1 --2- 172.21.15.203:0/3138288873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef6c14ba00 0x7fef6c140a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:09.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef70a4b700 1 -- 172.21.15.203:0/3138288873 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c13e180 con 0x7fef6c0be350 2021-01-06T22:08:09.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef7124c700 1 --2- 172.21.15.203:0/3138288873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef6c14ba00 0x7fef6c140a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:08:09.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef70a4b700 1 --2- 172.21.15.203:0/3138288873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef6c0be350 0x7fef6c13eba0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fef6000c920 tx=0x7fef60008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:09.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef68ff9700 1 -- 172.21.15.203:0/3138288873 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60007650 con 0x7fef6c0be350 2021-01-06T22:08:09.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef68ff9700 1 -- 172.21.15.203:0/3138288873 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef600077b0 con 0x7fef6c0be350 2021-01-06T22:08:09.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef68ff9700 1 -- 172.21.15.203:0/3138288873 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60007b70 con 0x7fef6c0be350 2021-01-06T22:08:09.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef71a4d700 1 -- 172.21.15.203:0/3138288873 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6c13e420 con 0x7fef6c0be350 2021-01-06T22:08:09.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.027+0000 7fef71a4d700 1 -- 172.21.15.203:0/3138288873 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c060d60 con 0x7fef6c0be350 2021-01-06T22:08:09.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.031+0000 7fef68ff9700 1 -- 172.21.15.203:0/3138288873 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fef6001b460 con 0x7fef6c0be350 2021-01-06T22:08:09.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.031+0000 7fef71a4d700 1 -- 172.21.15.203:0/3138288873 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6c1408e0 con 0x7fef6c0be350 2021-01-06T22:08:09.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.031+0000 7fef68ff9700 1 --2- 172.21.15.203:0/3138288873 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fef4c056400 0x7fef4c0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:09.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.031+0000 7fef6bfff700 1 --2- 172.21.15.203:0/3138288873 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fef4c056400 0x7fef4c0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:09.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.031+0000 7fef68ff9700 1 -- 172.21.15.203:0/3138288873 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fef6006f7e0 con 0x7fef6c0be350 2021-01-06T22:08:09.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.031+0000 7fef6bfff700 1 --2- 172.21.15.203:0/3138288873 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fef4c056400 0x7fef4c0588c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fef6c13e9d0 tx=0x7fef58009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:09.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.039+0000 7fef68ff9700 1 -- 172.21.15.203:0/3138288873 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fef60044e40 con 0x7fef6c0be350 2021-01-06T22:08:09.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.283+0000 7fef71a4d700 1 -- 172.21.15.203:0/3138288873 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef6c0450a0 con 0x7fef6c0be350 2021-01-06T22:08:09.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.287+0000 7fef68ff9700 1 -- 172.21.15.203:0/3138288873 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+3579 (secure 0 0 0) 0x7fef60048460 con 0x7fef6c0be350 2021-01-06T22:08:09.281 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:09.282 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:06.426663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:stopping","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:09.283 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.287+0000 7fef71a4d700 1 -- 172.21.15.203:0/3138288873 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fef4c056400 msgr2=0x7fef4c0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:09.283 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.287+0000 7fef71a4d700 1 --2- 172.21.15.203:0/3138288873 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fef4c056400 0x7fef4c0588c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fef6c13e9d0 tx=0x7fef58009040).stop 2021-01-06T22:08:09.284 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.287+0000 7fef71a4d700 1 -- 172.21.15.203:0/3138288873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef6c0be350 msgr2=0x7fef6c13eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:09.284 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.287+0000 7fef71a4d700 1 --2- 172.21.15.203:0/3138288873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef6c0be350 0x7fef6c13eba0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fef6000c920 tx=0x7fef60008040).stop 2021-01-06T22:08:09.285 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.287+0000 7fef71a4d700 1 -- 172.21.15.203:0/3138288873 shutdown_connections 2021-01-06T22:08:09.286 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.287+0000 7fef71a4d700 1 --2- 172.21.15.203:0/3138288873 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fef4c056400 0x7fef4c0588c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:09.286 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.287+0000 7fef71a4d700 1 --2- 172.21.15.203:0/3138288873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef6c14ba00 0x7fef6c140a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:09.286 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.287+0000 7fef71a4d700 1 --2- 172.21.15.203:0/3138288873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef6c0be350 0x7fef6c13eba0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:09.287 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.287+0000 7fef71a4d700 1 --2- 172.21.15.203:0/3138288873 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef6c0be730 0x7fef6c13f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:09.287 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.287+0000 7fef71a4d700 1 -- 172.21.15.203:0/3138288873 >> 172.21.15.203:0/3138288873 conn(0x7fef6c067e20 msgr2=0x7fef6c062d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:09.287 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.287+0000 7fef71a4d700 1 -- 172.21.15.203:0/3138288873 shutdown_connections 2021-01-06T22:08:09.288 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:09.287+0000 7fef71a4d700 1 -- 172.21.15.203:0/3138288873 wait complete. 2021-01-06T22:08:09.288 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 29 2021-01-06T22:08:09.301 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:08:11.301 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:11.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.447+0000 7ff6ae263700 1 Processor -- start 2021-01-06T22:08:11.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.447+0000 7ff6ae263700 1 -- start start 2021-01-06T22:08:11.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.447+0000 7ff6ae263700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff6a80c48d0 0x7ff6a8150e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:11.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.447+0000 7ff6ae263700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a80c0f50 con 0x7ff6a80c44f0 2021-01-06T22:08:11.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.447+0000 7ff6ae263700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a8066cf0 con 0x7ff6a80c1a90 2021-01-06T22:08:11.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.447+0000 7ff6ae263700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a8060750 con 0x7ff6a80c48d0 2021-01-06T22:08:11.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.447+0000 7ff6ada62700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff6a80c48d0 0x7ff6a8150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:11.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.447+0000 7ff6aca60700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7ff6a80c1a90 0x7ff6a8150730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48908/0 (socket says 172.21.15.203:48908) 2021-01-06T22:08:11.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.447+0000 7ff6aca60700 1 -- 172.21.15.203:0/2213289592 learned_addr learned my addr 172.21.15.203:0/2213289592 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:11.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.451+0000 7ff6ada62700 1 -- 172.21.15.203:0/2213289592 >> v1:172.21.15.203:6789/0 conn(0x7ff6a80c1a90 legacy=0x7ff6a8150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:11.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.451+0000 7ff6ada62700 1 -- 172.21.15.203:0/2213289592 >> v1:172.21.15.91:6789/0 conn(0x7ff6a80c44f0 legacy=0x7ff6a80c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:11.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.451+0000 7ff6ada62700 1 -- 172.21.15.203:0/2213289592 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a80609f0 con 0x7ff6a80c48d0 2021-01-06T22:08:11.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.451+0000 7ff6ada62700 1 --2- 172.21.15.203:0/2213289592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a80c48d0 0x7ff6a8150e40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7ff6a0009c90 tx=0x7ff6a00091d0).ready entity=mon.1 client_cookie=dd861421a512ab8c server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:11.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.451+0000 7ff697fff700 1 -- 172.21.15.203:0/2213289592 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a000bbc0 con 0x7ff6a80c48d0 2021-01-06T22:08:11.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.451+0000 7ff697fff700 1 -- 172.21.15.203:0/2213289592 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a000bd20 con 0x7ff6a80c48d0 2021-01-06T22:08:11.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.451+0000 7ff697fff700 1 -- 172.21.15.203:0/2213289592 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0011610 con 0x7ff6a80c48d0 2021-01-06T22:08:11.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.451+0000 7ff6ae263700 1 -- 172.21.15.203:0/2213289592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a80c48d0 msgr2=0x7ff6a8150e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:11.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.451+0000 7ff6ae263700 1 --2- 172.21.15.203:0/2213289592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a80c48d0 0x7ff6a8150e40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7ff6a0009c90 tx=0x7ff6a00091d0).stop 2021-01-06T22:08:11.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.451+0000 7ff6ae263700 1 -- 172.21.15.203:0/2213289592 shutdown_connections 2021-01-06T22:08:11.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.451+0000 7ff6ae263700 1 --2- 172.21.15.203:0/2213289592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a80c48d0 0x7ff6a8150e40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:11.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.451+0000 7ff6ae263700 1 -- 172.21.15.203:0/2213289592 >> 172.21.15.203:0/2213289592 conn(0x7ff6a8056cd0 msgr2=0x7ff6a8061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:11.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.451+0000 7ff6ae263700 1 -- 172.21.15.203:0/2213289592 shutdown_connections 2021-01-06T22:08:11.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.451+0000 7ff6ae263700 1 -- 172.21.15.203:0/2213289592 wait complete. 2021-01-06T22:08:11.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.451+0000 7ff6ae263700 1 Processor -- start 2021-01-06T22:08:11.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.451+0000 7ff6ae263700 1 -- start start 2021-01-06T22:08:11.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6ae263700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff6a80c1a90 0x7ff6a80b9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:11.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6ae263700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a80c44f0 0x7ff6a8153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:11.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6ae263700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6a80c48d0 0x7ff6a814eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:11.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6ae263700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a8153420 con 0x7ff6a80c48d0 2021-01-06T22:08:11.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6ae263700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a80b8f90 con 0x7ff6a80c44f0 2021-01-06T22:08:11.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6ae263700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a80b8a50 con 0x7ff6a80c1a90 2021-01-06T22:08:11.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6aca60700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a80c44f0 0x7ff6a8153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:11.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6aca60700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a80c44f0 0x7ff6a8153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50320/0 (socket says 172.21.15.203:50320) 2021-01-06T22:08:11.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6aca60700 1 -- 172.21.15.203:0/2566833046 learned_addr learned my addr 172.21.15.203:0/2566833046 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:11.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6ad261700 1 --2- 172.21.15.203:0/2566833046 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff6a80c1a90 0x7ff6a80b9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:11.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6ada62700 1 --2- 172.21.15.203:0/2566833046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6a80c48d0 0x7ff6a814eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:11.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6aca60700 1 -- 172.21.15.203:0/2566833046 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff6a80c1a90 msgr2=0x7ff6a80b9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:11.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6aca60700 1 --2- 172.21.15.203:0/2566833046 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff6a80c1a90 0x7ff6a80b9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:11.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6aca60700 1 -- 172.21.15.203:0/2566833046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6a80c48d0 msgr2=0x7ff6a814eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:11.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6aca60700 1 --2- 172.21.15.203:0/2566833046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6a80c48d0 0x7ff6a814eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:11.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6aca60700 1 -- 172.21.15.203:0/2566833046 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a80b8cf0 con 0x7ff6a80c44f0 2021-01-06T22:08:11.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6ad261700 1 --2- 172.21.15.203:0/2566833046 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff6a80c1a90 0x7ff6a80b9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:08:11.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6aca60700 1 --2- 172.21.15.203:0/2566833046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a80c44f0 0x7ff6a8153ca0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7ff69800c710 tx=0x7ff698007c00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:11.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6957fa700 1 -- 172.21.15.203:0/2566833046 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698007320 con 0x7ff6a80c44f0 2021-01-06T22:08:11.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6957fa700 1 -- 172.21.15.203:0/2566833046 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff698007480 con 0x7ff6a80c44f0 2021-01-06T22:08:11.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6957fa700 1 -- 172.21.15.203:0/2566833046 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6980057f0 con 0x7ff6a80c44f0 2021-01-06T22:08:11.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6ae263700 1 -- 172.21.15.203:0/2566833046 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a8153680 con 0x7ff6a80c44f0 2021-01-06T22:08:11.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6ae263700 1 -- 172.21.15.203:0/2566833046 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a8154190 con 0x7ff6a80c44f0 2021-01-06T22:08:11.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6957fa700 1 -- 172.21.15.203:0/2566833046 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff698010030 con 0x7ff6a80c44f0 2021-01-06T22:08:11.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.455+0000 7ff6957fa700 1 --2- 172.21.15.203:0/2566833046 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6880563d0 0x7ff688058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:11.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.459+0000 7ff6957fa700 1 -- 172.21.15.203:0/2566833046 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7ff69806f380 con 0x7ff6a80c44f0 2021-01-06T22:08:11.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.459+0000 7ff6ad261700 1 --2- 172.21.15.203:0/2566833046 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6880563d0 0x7ff688058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:11.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.459+0000 7ff6ae263700 1 -- 172.21.15.203:0/2566833046 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a80450a0 con 0x7ff6a80c44f0 2021-01-06T22:08:11.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.459+0000 7ff6ad261700 1 --2- 172.21.15.203:0/2566833046 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6880563d0 0x7ff688058890 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7ff6a4001530 tx=0x7ff6a4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:11.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.463+0000 7ff6957fa700 1 -- 172.21.15.203:0/2566833046 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff698044af0 con 0x7ff6a80c44f0 2021-01-06T22:08:11.691 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.695+0000 7ff6ae263700 1 -- 172.21.15.203:0/2566833046 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6a80be7f0 con 0x7ff6a80c44f0 2021-01-06T22:08:11.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.695+0000 7ff6957fa700 1 -- 172.21.15.203:0/2566833046 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3580 (secure 0 0 0) 0x7ff698048110 con 0x7ff6a80c44f0 2021-01-06T22:08:11.692 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:11.693 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:10.461108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:stopping","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[0],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:11.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.699+0000 7ff6ae263700 1 -- 172.21.15.203:0/2566833046 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6880563d0 msgr2=0x7ff688058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:11.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.699+0000 7ff6ae263700 1 --2- 172.21.15.203:0/2566833046 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6880563d0 0x7ff688058890 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7ff6a4001530 tx=0x7ff6a4009040).stop 2021-01-06T22:08:11.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.699+0000 7ff6ae263700 1 -- 172.21.15.203:0/2566833046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a80c44f0 msgr2=0x7ff6a8153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:11.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.699+0000 7ff6ae263700 1 --2- 172.21.15.203:0/2566833046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a80c44f0 0x7ff6a8153ca0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7ff69800c710 tx=0x7ff698007c00).stop 2021-01-06T22:08:11.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.699+0000 7ff6ae263700 1 -- 172.21.15.203:0/2566833046 shutdown_connections 2021-01-06T22:08:11.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.699+0000 7ff6ae263700 1 --2- 172.21.15.203:0/2566833046 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6880563d0 0x7ff688058890 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:11.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.699+0000 7ff6ae263700 1 --2- 172.21.15.203:0/2566833046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6a80c48d0 0x7ff6a814eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:11.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.699+0000 7ff6ae263700 1 --2- 172.21.15.203:0/2566833046 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff6a80c1a90 0x7ff6a80b9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:11.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.699+0000 7ff6ae263700 1 --2- 172.21.15.203:0/2566833046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a80c44f0 0x7ff6a8153ca0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:11.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.699+0000 7ff6ae263700 1 -- 172.21.15.203:0/2566833046 >> 172.21.15.203:0/2566833046 conn(0x7ff6a8056cd0 msgr2=0x7ff6a814fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:11.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.699+0000 7ff6ae263700 1 -- 172.21.15.203:0/2566833046 shutdown_connections 2021-01-06T22:08:11.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:11.699+0000 7ff6ae263700 1 -- 172.21.15.203:0/2566833046 wait complete. 2021-01-06T22:08:11.698 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 30 2021-01-06T22:08:11.710 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:08:12.458 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:08:12.463 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:08:13.711 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:13.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.855+0000 7f8b11070700 1 Processor -- start 2021-01-06T22:08:13.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.855+0000 7f8b11070700 1 -- start start 2021-01-06T22:08:13.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b11070700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b0c0c1a90 0x7f8b0c150730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:13.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b11070700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8b0c0c48d0 0x7f8b0c150c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:13.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b11070700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0c0c0f50 con 0x7f8b0c0c44f0 2021-01-06T22:08:13.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b11070700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b0c066cf0 con 0x7f8b0c0c48d0 2021-01-06T22:08:13.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b11070700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c060750 con 0x7f8b0c0c1a90 2021-01-06T22:08:13.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b1086f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8b0c0c48d0 0x7f8b0c150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:13.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b0bfff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f8b0c0c44f0 0x7f8b0c0c1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48920/0 (socket says 172.21.15.203:48920) 2021-01-06T22:08:13.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b0bfff700 1 -- 172.21.15.203:0/1774254612 learned_addr learned my addr 172.21.15.203:0/1774254612 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:13.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b0b7fe700 1 --2- 172.21.15.203:0/1774254612 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b0c0c1a90 0x7f8b0c150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:13.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b1086f700 1 -- 172.21.15.203:0/1774254612 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b0c0c1a90 msgr2=0x7f8b0c150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:13.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b1086f700 1 --2- 172.21.15.203:0/1774254612 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b0c0c1a90 0x7f8b0c150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:13.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b1086f700 1 -- 172.21.15.203:0/1774254612 >> v1:172.21.15.203:6789/0 conn(0x7f8b0c0c44f0 legacy=0x7f8b0c0c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:13.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b0affd700 1 -- 172.21.15.203:0/1774254612 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546286929 0 0) 0x7f8b0c0c0f50 con 0x7f8b0c0c44f0 2021-01-06T22:08:13.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b1086f700 1 -- 172.21.15.203:0/1774254612 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c0609f0 con 0x7f8b0c0c48d0 2021-01-06T22:08:13.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b0b7fe700 1 --2- 172.21.15.203:0/1774254612 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b0c0c1a90 0x7f8b0c150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:08:13.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b1086f700 1 --2- 172.21.15.203:0/1774254612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b0c0c48d0 0x7f8b0c150c20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f8b04009c90 tx=0x7f8b040091d0).ready entity=mon.1 client_cookie=1c08de7024643c6e server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:13.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b0affd700 1 -- 172.21.15.203:0/1774254612 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0400bbc0 con 0x7f8b0c0c48d0 2021-01-06T22:08:13.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b0affd700 1 -- 172.21.15.203:0/1774254612 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0400bd20 con 0x7f8b0c0c48d0 2021-01-06T22:08:13.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b0affd700 1 -- 172.21.15.203:0/1774254612 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b04011610 con 0x7f8b0c0c48d0 2021-01-06T22:08:13.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b11070700 1 -- 172.21.15.203:0/1774254612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b0c0c48d0 msgr2=0x7f8b0c150c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:13.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b11070700 1 --2- 172.21.15.203:0/1774254612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b0c0c48d0 0x7f8b0c150c20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f8b04009c90 tx=0x7f8b040091d0).stop 2021-01-06T22:08:13.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b11070700 1 -- 172.21.15.203:0/1774254612 shutdown_connections 2021-01-06T22:08:13.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b11070700 1 --2- 172.21.15.203:0/1774254612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b0c0c48d0 0x7f8b0c150c20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:13.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b11070700 1 --2- 172.21.15.203:0/1774254612 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b0c0c1a90 0x7f8b0c150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:13.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b11070700 1 -- 172.21.15.203:0/1774254612 >> 172.21.15.203:0/1774254612 conn(0x7f8b0c056cd0 msgr2=0x7f8b0c061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:13.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b11070700 1 -- 172.21.15.203:0/1774254612 shutdown_connections 2021-01-06T22:08:13.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.859+0000 7f8b11070700 1 -- 172.21.15.203:0/1774254612 wait complete. 2021-01-06T22:08:13.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b11070700 1 Processor -- start 2021-01-06T22:08:13.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b11070700 1 -- start start 2021-01-06T22:08:13.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b11070700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b0c0c1a90 0x7f8b0c0ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:13.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b11070700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b0c0c44f0 0x7f8b0c161660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:13.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b11070700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b0c0c48d0 0x7f8b0c0b9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:13.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b11070700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c0bb400 con 0x7f8b0c0c48d0 2021-01-06T22:08:13.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b11070700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c161b50 con 0x7f8b0c0c44f0 2021-01-06T22:08:13.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b11070700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c0b8780 con 0x7f8b0c0c1a90 2021-01-06T22:08:13.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b0b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b0c0c44f0 0x7f8b0c161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:13.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b0b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b0c0c44f0 0x7f8b0c161660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50334/0 (socket says 172.21.15.203:50334) 2021-01-06T22:08:13.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b0b7fe700 1 -- 172.21.15.203:0/954825181 learned_addr learned my addr 172.21.15.203:0/954825181 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:13.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b1086f700 1 --2- 172.21.15.203:0/954825181 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b0c0c48d0 0x7f8b0c0b9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:13.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b0bfff700 1 --2- 172.21.15.203:0/954825181 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b0c0c1a90 0x7f8b0c0ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:13.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b0b7fe700 1 -- 172.21.15.203:0/954825181 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b0c0c1a90 msgr2=0x7f8b0c0ba7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:13.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b0b7fe700 1 --2- 172.21.15.203:0/954825181 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b0c0c1a90 0x7f8b0c0ba7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:13.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b0b7fe700 1 -- 172.21.15.203:0/954825181 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b0c0c48d0 msgr2=0x7f8b0c0b9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:13.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b0b7fe700 1 --2- 172.21.15.203:0/954825181 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b0c0c48d0 0x7f8b0c0b9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:13.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b0b7fe700 1 -- 172.21.15.203:0/954825181 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c1522a0 con 0x7f8b0c0c44f0 2021-01-06T22:08:13.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b1086f700 1 --2- 172.21.15.203:0/954825181 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b0c0c48d0 0x7f8b0c0b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:08:13.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b0b7fe700 1 --2- 172.21.15.203:0/954825181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b0c0c44f0 0x7f8b0c161660 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f8b0000db30 tx=0x7f8b00008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:13.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8af3fff700 1 -- 172.21.15.203:0/954825181 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b000097c0 con 0x7f8b0c0c44f0 2021-01-06T22:08:13.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8af3fff700 1 -- 172.21.15.203:0/954825181 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b00009920 con 0x7f8b0c0c44f0 2021-01-06T22:08:13.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b11070700 1 -- 172.21.15.203:0/954825181 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0c152540 con 0x7f8b0c0c44f0 2021-01-06T22:08:13.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8af3fff700 1 -- 172.21.15.203:0/954825181 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00009ca0 con 0x7f8b0c0c44f0 2021-01-06T22:08:13.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.863+0000 7f8b11070700 1 -- 172.21.15.203:0/954825181 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0c1513e0 con 0x7f8b0c0c44f0 2021-01-06T22:08:13.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.867+0000 7f8b11070700 1 -- 172.21.15.203:0/954825181 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0c1515a0 con 0x7f8b0c0c44f0 2021-01-06T22:08:13.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.867+0000 7f8af3fff700 1 -- 172.21.15.203:0/954825181 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f8b00013690 con 0x7f8b0c0c44f0 2021-01-06T22:08:13.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.867+0000 7f8af3fff700 1 --2- 172.21.15.203:0/954825181 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8aec0563d0 0x7f8aec058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:13.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.867+0000 7f8af3fff700 1 -- 172.21.15.203:0/954825181 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f8b0006f6d0 con 0x7f8b0c0c44f0 2021-01-06T22:08:13.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.867+0000 7f8b0bfff700 1 --2- 172.21.15.203:0/954825181 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8aec0563d0 0x7f8aec058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:13.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.867+0000 7f8b0bfff700 1 --2- 172.21.15.203:0/954825181 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8aec0563d0 0x7f8aec058890 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f8afc0011a0 tx=0x7f8afc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:13.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:13.871+0000 7f8af3fff700 1 -- 172.21.15.203:0/954825181 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f8b00044ea0 con 0x7f8b0c0c44f0 2021-01-06T22:08:14.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:14.103+0000 7f8b11070700 1 -- 172.21.15.203:0/954825181 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b0c0450a0 con 0x7f8b0c0c44f0 2021-01-06T22:08:14.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:14.103+0000 7f8af3fff700 1 -- 172.21.15.203:0/954825181 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3580 (secure 0 0 0) 0x7f8b00009e00 con 0x7f8b0c0c44f0 2021-01-06T22:08:14.099 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:14.100 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:10.461108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:stopping","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[0],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:14.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:14.107+0000 7f8b11070700 1 -- 172.21.15.203:0/954825181 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8aec0563d0 msgr2=0x7f8aec058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:14.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:14.107+0000 7f8b11070700 1 --2- 172.21.15.203:0/954825181 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8aec0563d0 0x7f8aec058890 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f8afc0011a0 tx=0x7f8afc009040).stop 2021-01-06T22:08:14.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:14.107+0000 7f8b11070700 1 -- 172.21.15.203:0/954825181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b0c0c44f0 msgr2=0x7f8b0c161660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:14.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:14.107+0000 7f8b11070700 1 --2- 172.21.15.203:0/954825181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b0c0c44f0 0x7f8b0c161660 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f8b0000db30 tx=0x7f8b00008040).stop 2021-01-06T22:08:14.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:14.107+0000 7f8b11070700 1 -- 172.21.15.203:0/954825181 shutdown_connections 2021-01-06T22:08:14.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:14.107+0000 7f8b11070700 1 --2- 172.21.15.203:0/954825181 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8aec0563d0 0x7f8aec058890 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:14.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:14.107+0000 7f8b11070700 1 --2- 172.21.15.203:0/954825181 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b0c0c48d0 0x7f8b0c0b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:14.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:14.107+0000 7f8b11070700 1 --2- 172.21.15.203:0/954825181 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b0c0c1a90 0x7f8b0c0ba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:14.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:14.107+0000 7f8b11070700 1 --2- 172.21.15.203:0/954825181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b0c0c44f0 0x7f8b0c161660 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:14.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:14.107+0000 7f8b11070700 1 -- 172.21.15.203:0/954825181 >> 172.21.15.203:0/954825181 conn(0x7f8b0c056cd0 msgr2=0x7f8b0c14fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:14.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:14.107+0000 7f8b11070700 1 -- 172.21.15.203:0/954825181 shutdown_connections 2021-01-06T22:08:14.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:14.107+0000 7f8b11070700 1 -- 172.21.15.203:0/954825181 wait complete. 2021-01-06T22:08:14.105 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 30 2021-01-06T22:08:14.114 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2021-01-06T22:08:14.115 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 30, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 30, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4273": { "addr": "172.21.15.203:6837/3277561222", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6836", "nonce": 3277561222, "type": "v2" }, { "addr": "172.21.15.203:6837", "nonce": 3277561222, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4273, "incarnation": 9, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 3 }, "gid_4274": { "addr": "172.21.15.91:6837/3888667247", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6835", "nonce": 3888667247, "type": "v2" }, { "addr": "172.21.15.91:6837", "nonce": 3888667247, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4274, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 2 }, "gid_4867": { "addr": "172.21.15.203:6835/3382231276", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6834", "nonce": 3382231276, "type": "v2" }, { "addr": "172.21.15.203:6835", "nonce": 3382231276, "type": "v1" } ] }, "export_targets": [ 0 ], "features": 4540138297136906239, "flags": 0, "gid": 4867, "incarnation": 20, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:stopping", "state_seq": 22 } }, "last_failure": 0, "last_failure_osd_epoch": 25, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2021-01-06T22:08:10.461108+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4274, "mds_1": 4273, "mds_2": 4867 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.91:6836/2428882934", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6834", "nonce": 2428882934, "type": "v2" }, { "addr": "172.21.15.91:6836", "nonce": 2428882934, "type": "v1" } ] }, "epoch": 26, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4724, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2021-01-06T22:08:16.116 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:16.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84e49d700 1 Processor -- start 2021-01-06T22:08:16.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84e49d700 1 -- start start 2021-01-06T22:08:16.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84e49d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa8481513f0 0x7fa8480c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:16.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84e49d700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa84814ad90 0x7fa84814b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:16.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84e49d700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8480bacb0 con 0x7fa8480bccb0 2021-01-06T22:08:16.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84e49d700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8480bd160 con 0x7fa84814ad90 2021-01-06T22:08:16.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84e49d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84814b680 con 0x7fa8481513f0 2021-01-06T22:08:16.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84d49b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa8481513f0 0x7fa8480c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:16.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84cc9a700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fa8480bccb0 0x7fa8481517d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48936/0 (socket says 172.21.15.203:48936) 2021-01-06T22:08:16.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84cc9a700 1 -- 172.21.15.203:0/2769333902 learned_addr learned my addr 172.21.15.203:0/2769333902 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:16.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84dc9c700 1 --2- 172.21.15.203:0/2769333902 >> v2:172.21.15.91:3300/0 conn(0x7fa84814ad90 0x7fa84814b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:16.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84d49b700 1 -- 172.21.15.203:0/2769333902 >> v2:172.21.15.91:3300/0 conn(0x7fa84814ad90 msgr2=0x7fa84814b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:16.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84d49b700 1 --2- 172.21.15.203:0/2769333902 >> v2:172.21.15.91:3300/0 conn(0x7fa84814ad90 0x7fa84814b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:16.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84d49b700 1 -- 172.21.15.203:0/2769333902 >> v1:172.21.15.203:6789/0 conn(0x7fa8480bccb0 legacy=0x7fa8481517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:16.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa837fff700 1 -- 172.21.15.203:0/2769333902 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379351027 0 0) 0x7fa8380010d0 con 0x7fa8480bccb0 2021-01-06T22:08:16.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84d49b700 1 -- 172.21.15.203:0/2769333902 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa848152000 con 0x7fa8481513f0 2021-01-06T22:08:16.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84d49b700 1 --2- 172.21.15.203:0/2769333902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8481513f0 0x7fa8480c0fa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fa848061ee0 tx=0x7fa844006de0).ready entity=mon.1 client_cookie=bfe6db641b02aa96 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:16.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa837fff700 1 -- 172.21.15.203:0/2769333902 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8440097b0 con 0x7fa8481513f0 2021-01-06T22:08:16.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa837fff700 1 -- 172.21.15.203:0/2769333902 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa844009910 con 0x7fa8481513f0 2021-01-06T22:08:16.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa837fff700 1 -- 172.21.15.203:0/2769333902 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844009c90 con 0x7fa8481513f0 2021-01-06T22:08:16.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84e49d700 1 -- 172.21.15.203:0/2769333902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8481513f0 msgr2=0x7fa8480c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:16.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84e49d700 1 --2- 172.21.15.203:0/2769333902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8481513f0 0x7fa8480c0fa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fa848061ee0 tx=0x7fa844006de0).stop 2021-01-06T22:08:16.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84e49d700 1 -- 172.21.15.203:0/2769333902 shutdown_connections 2021-01-06T22:08:16.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84e49d700 1 --2- 172.21.15.203:0/2769333902 >> v2:172.21.15.91:3300/0 conn(0x7fa84814ad90 0x7fa84814b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:16.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84e49d700 1 --2- 172.21.15.203:0/2769333902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8481513f0 0x7fa8480c0fa0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:16.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84e49d700 1 -- 172.21.15.203:0/2769333902 >> 172.21.15.203:0/2769333902 conn(0x7fa8480667b0 msgr2=0x7fa8480c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:16.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84e49d700 1 -- 172.21.15.203:0/2769333902 shutdown_connections 2021-01-06T22:08:16.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.255+0000 7fa84e49d700 1 -- 172.21.15.203:0/2769333902 wait complete. 2021-01-06T22:08:16.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84e49d700 1 Processor -- start 2021-01-06T22:08:16.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84e49d700 1 -- start start 2021-01-06T22:08:16.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84e49d700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8480bccb0 0x7fa848147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:16.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84e49d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa84814ad90 0x7fa848160f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:16.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84e49d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8481513f0 0x7fa848161410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:16.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84e49d700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa848150460 con 0x7fa8480bccb0 2021-01-06T22:08:16.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84e49d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa848161900 con 0x7fa8481513f0 2021-01-06T22:08:16.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84e49d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84815fce0 con 0x7fa84814ad90 2021-01-06T22:08:16.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84dc9c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8481513f0 0x7fa848161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:16.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84dc9c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8481513f0 0x7fa848161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50350/0 (socket says 172.21.15.203:50350) 2021-01-06T22:08:16.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84dc9c700 1 -- 172.21.15.203:0/3763908861 learned_addr learned my addr 172.21.15.203:0/3763908861 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:16.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84d49b700 1 --2- 172.21.15.203:0/3763908861 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8480bccb0 0x7fa848147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:16.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84cc9a700 1 --2- 172.21.15.203:0/3763908861 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa84814ad90 0x7fa848160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:16.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84dc9c700 1 -- 172.21.15.203:0/3763908861 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa84814ad90 msgr2=0x7fa848160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:16.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84dc9c700 1 --2- 172.21.15.203:0/3763908861 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa84814ad90 0x7fa848160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:16.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84dc9c700 1 -- 172.21.15.203:0/3763908861 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8480bccb0 msgr2=0x7fa848147840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:16.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84dc9c700 1 --2- 172.21.15.203:0/3763908861 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8480bccb0 0x7fa848147840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:16.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84dc9c700 1 -- 172.21.15.203:0/3763908861 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84815ff80 con 0x7fa8481513f0 2021-01-06T22:08:16.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84dc9c700 1 --2- 172.21.15.203:0/3763908861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8481513f0 0x7fa848161410 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7fa84000ebc0 tx=0x7fa840008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:16.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa8357fa700 1 -- 172.21.15.203:0/3763908861 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84000c800 con 0x7fa8481513f0 2021-01-06T22:08:16.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa8357fa700 1 -- 172.21.15.203:0/3763908861 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84000c960 con 0x7fa8481513f0 2021-01-06T22:08:16.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84e49d700 1 -- 172.21.15.203:0/3763908861 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa848160220 con 0x7fa8481513f0 2021-01-06T22:08:16.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa8357fa700 1 -- 172.21.15.203:0/3763908861 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84000cd00 con 0x7fa8481513f0 2021-01-06T22:08:16.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.259+0000 7fa84e49d700 1 -- 172.21.15.203:0/3763908861 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa848160760 con 0x7fa8481513f0 2021-01-06T22:08:16.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.263+0000 7fa84e49d700 1 -- 172.21.15.203:0/3763908861 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa848160c50 con 0x7fa8481513f0 2021-01-06T22:08:16.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.263+0000 7fa8357fa700 1 -- 172.21.15.203:0/3763908861 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa840013640 con 0x7fa8481513f0 2021-01-06T22:08:16.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.263+0000 7fa8357fa700 1 --2- 172.21.15.203:0/3763908861 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8280563d0 0x7fa828058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:16.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.263+0000 7fa8357fa700 1 -- 172.21.15.203:0/3763908861 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fa84006f730 con 0x7fa8481513f0 2021-01-06T22:08:16.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.263+0000 7fa84d49b700 1 --2- 172.21.15.203:0/3763908861 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8280563d0 0x7fa828058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:16.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.263+0000 7fa84d49b700 1 --2- 172.21.15.203:0/3763908861 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8280563d0 0x7fa828058890 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fa84400b130 tx=0x7fa844016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:16.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.267+0000 7fa8357fa700 1 -- 172.21.15.203:0/3763908861 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa840044f00 con 0x7fa8481513f0 2021-01-06T22:08:16.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.471+0000 7fa84e49d700 1 -- 172.21.15.203:0/3763908861 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8480450a0 con 0x7fa8481513f0 2021-01-06T22:08:16.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.471+0000 7fa8357fa700 1 -- 172.21.15.203:0/3763908861 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3580 (secure 0 0 0) 0x7fa84000ce60 con 0x7fa8481513f0 2021-01-06T22:08:16.467 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:16.467 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:10.461108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:stopping","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[0],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:16.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.475+0000 7fa84e49d700 1 -- 172.21.15.203:0/3763908861 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8280563d0 msgr2=0x7fa828058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:16.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.475+0000 7fa84e49d700 1 --2- 172.21.15.203:0/3763908861 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8280563d0 0x7fa828058890 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fa84400b130 tx=0x7fa844016000).stop 2021-01-06T22:08:16.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.475+0000 7fa84e49d700 1 -- 172.21.15.203:0/3763908861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8481513f0 msgr2=0x7fa848161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:16.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.475+0000 7fa84e49d700 1 --2- 172.21.15.203:0/3763908861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8481513f0 0x7fa848161410 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7fa84000ebc0 tx=0x7fa840008040).stop 2021-01-06T22:08:16.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.475+0000 7fa84e49d700 1 -- 172.21.15.203:0/3763908861 shutdown_connections 2021-01-06T22:08:16.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.475+0000 7fa84e49d700 1 --2- 172.21.15.203:0/3763908861 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8280563d0 0x7fa828058890 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:16.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.475+0000 7fa84e49d700 1 --2- 172.21.15.203:0/3763908861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8481513f0 0x7fa848161410 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:16.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.475+0000 7fa84e49d700 1 --2- 172.21.15.203:0/3763908861 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8480bccb0 0x7fa848147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:16.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.475+0000 7fa84e49d700 1 --2- 172.21.15.203:0/3763908861 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa84814ad90 0x7fa848160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:16.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.475+0000 7fa84e49d700 1 -- 172.21.15.203:0/3763908861 >> 172.21.15.203:0/3763908861 conn(0x7fa8480667b0 msgr2=0x7fa848064de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:16.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.475+0000 7fa84e49d700 1 -- 172.21.15.203:0/3763908861 shutdown_connections 2021-01-06T22:08:16.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:16.475+0000 7fa84e49d700 1 -- 172.21.15.203:0/3763908861 wait complete. 2021-01-06T22:08:16.473 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 30 2021-01-06T22:08:16.487 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:08:18.489 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:18.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c2cb08700 1 Processor -- start 2021-01-06T22:08:18.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c2cb08700 1 -- start start 2021-01-06T22:08:18.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c2cb08700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3c280bb3f0 0x7f3c280ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:18.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c2cb08700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c280a5e50 0x7f3c280669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:18.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c2cb08700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c280bb7d0 0x7f3c280602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:18.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c2cb08700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c280646b0 con 0x7f3c280bb3f0 2021-01-06T22:08:18.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c2cb08700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c28152cc0 con 0x7f3c280a5e50 2021-01-06T22:08:18.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c2cb08700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c28064450 con 0x7f3c280bb7d0 2021-01-06T22:08:18.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c26ffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c280a5e50 0x7f3c280669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:18.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c26ffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c280a5e50 0x7f3c280669b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50360/0 (socket says 172.21.15.203:50360) 2021-01-06T22:08:18.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c26ffd700 1 -- 172.21.15.203:0/1119547332 learned_addr learned my addr 172.21.15.203:0/1119547332 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:18.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c27fff700 1 --2- 172.21.15.203:0/1119547332 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c280bb7d0 0x7f3c280602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:18.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c277fe700 1 --2- 172.21.15.203:0/1119547332 >> v2:172.21.15.91:3300/0 conn(0x7f3c280bb3f0 0x7f3c280ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:18.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c26ffd700 1 -- 172.21.15.203:0/1119547332 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c280bb7d0 msgr2=0x7f3c280602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:18.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c26ffd700 1 --2- 172.21.15.203:0/1119547332 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c280bb7d0 0x7f3c280602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:18.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c26ffd700 1 -- 172.21.15.203:0/1119547332 >> v2:172.21.15.91:3300/0 conn(0x7f3c280bb3f0 msgr2=0x7f3c280ba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:18.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c26ffd700 1 --2- 172.21.15.203:0/1119547332 >> v2:172.21.15.91:3300/0 conn(0x7f3c280bb3f0 0x7f3c280ba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:18.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c26ffd700 1 -- 172.21.15.203:0/1119547332 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c28068530 con 0x7f3c280a5e50 2021-01-06T22:08:18.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.627+0000 7f3c26ffd700 1 --2- 172.21.15.203:0/1119547332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c280a5e50 0x7f3c280669b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f3c28064c50 tx=0x7f3c10006ea0).ready entity=mon.1 client_cookie=2fdfad5ecc606ede server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:18.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.631+0000 7f3c267fc700 1 -- 172.21.15.203:0/1119547332 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c100096e0 con 0x7f3c280a5e50 2021-01-06T22:08:18.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.631+0000 7f3c267fc700 1 -- 172.21.15.203:0/1119547332 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c10009840 con 0x7f3c280a5e50 2021-01-06T22:08:18.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.631+0000 7f3c267fc700 1 -- 172.21.15.203:0/1119547332 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c10009bc0 con 0x7f3c280a5e50 2021-01-06T22:08:18.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.631+0000 7f3c2cb08700 1 -- 172.21.15.203:0/1119547332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c280a5e50 msgr2=0x7f3c280669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:18.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.631+0000 7f3c2cb08700 1 --2- 172.21.15.203:0/1119547332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c280a5e50 0x7f3c280669b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f3c28064c50 tx=0x7f3c10006ea0).stop 2021-01-06T22:08:18.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.631+0000 7f3c2cb08700 1 -- 172.21.15.203:0/1119547332 shutdown_connections 2021-01-06T22:08:18.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.631+0000 7f3c2cb08700 1 --2- 172.21.15.203:0/1119547332 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c280bb7d0 0x7f3c280602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:18.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.631+0000 7f3c2cb08700 1 --2- 172.21.15.203:0/1119547332 >> v2:172.21.15.91:3300/0 conn(0x7f3c280bb3f0 0x7f3c280ba9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:18.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.631+0000 7f3c2cb08700 1 --2- 172.21.15.203:0/1119547332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c280a5e50 0x7f3c280669b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:18.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.631+0000 7f3c2cb08700 1 -- 172.21.15.203:0/1119547332 >> 172.21.15.203:0/1119547332 conn(0x7f3c28061530 msgr2=0x7f3c28063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:18.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.631+0000 7f3c2cb08700 1 -- 172.21.15.203:0/1119547332 shutdown_connections 2021-01-06T22:08:18.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.631+0000 7f3c2cb08700 1 -- 172.21.15.203:0/1119547332 wait complete. 2021-01-06T22:08:18.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.631+0000 7f3c2cb08700 1 Processor -- start 2021-01-06T22:08:18.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.631+0000 7f3c2cb08700 1 -- start start 2021-01-06T22:08:18.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c2cb08700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c280bb3f0 0x7f3c280b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:18.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c2cb08700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c280bb7d0 0x7f3c2814d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:18.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c2cb08700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c28065e00 0x7f3c2814d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:18.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c2cb08700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c28162a60 con 0x7f3c280bb7d0 2021-01-06T22:08:18.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c2cb08700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2814dde0 con 0x7f3c28065e00 2021-01-06T22:08:18.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c2cb08700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c28162520 con 0x7f3c280bb3f0 2021-01-06T22:08:18.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c26ffd700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c280bb7d0 0x7f3c2814d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:18.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c27fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c28065e00 0x7f3c2814d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:18.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c26ffd700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c280bb7d0 0x7f3c2814d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:58212/0 (socket says 172.21.15.203:58212) 2021-01-06T22:08:18.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c27fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c28065e00 0x7f3c2814d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50366/0 (socket says 172.21.15.203:50366) 2021-01-06T22:08:18.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c26ffd700 1 -- 172.21.15.203:0/4192739806 learned_addr learned my addr 172.21.15.203:0/4192739806 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:18.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c277fe700 1 --2- 172.21.15.203:0/4192739806 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c280bb3f0 0x7f3c280b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:18.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c27fff700 1 -- 172.21.15.203:0/4192739806 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c280bb3f0 msgr2=0x7f3c280b8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:18.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c27fff700 1 --2- 172.21.15.203:0/4192739806 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c280bb3f0 0x7f3c280b8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:18.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c27fff700 1 -- 172.21.15.203:0/4192739806 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c280bb7d0 msgr2=0x7f3c2814d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:18.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c27fff700 1 --2- 172.21.15.203:0/4192739806 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c280bb7d0 0x7f3c2814d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:18.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c27fff700 1 -- 172.21.15.203:0/4192739806 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c281627c0 con 0x7f3c28065e00 2021-01-06T22:08:18.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c27fff700 1 --2- 172.21.15.203:0/4192739806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c28065e00 0x7f3c2814d8f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f3c2000ed30 tx=0x7f3c200091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:18.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c1f7fe700 1 -- 172.21.15.203:0/4192739806 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000cbb0 con 0x7f3c28065e00 2021-01-06T22:08:18.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c1f7fe700 1 -- 172.21.15.203:0/4192739806 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2000cd10 con 0x7f3c28065e00 2021-01-06T22:08:18.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c1f7fe700 1 -- 172.21.15.203:0/4192739806 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20012640 con 0x7f3c28065e00 2021-01-06T22:08:18.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c2cb08700 1 -- 172.21.15.203:0/4192739806 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c280b7fe0 con 0x7f3c28065e00 2021-01-06T22:08:18.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.635+0000 7f3c2cb08700 1 -- 172.21.15.203:0/4192739806 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c280b8520 con 0x7f3c28065e00 2021-01-06T22:08:18.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.639+0000 7f3c1f7fe700 1 -- 172.21.15.203:0/4192739806 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f3c20022070 con 0x7f3c28065e00 2021-01-06T22:08:18.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.639+0000 7f3c2cb08700 1 -- 172.21.15.203:0/4192739806 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c280450a0 con 0x7f3c28065e00 2021-01-06T22:08:18.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.639+0000 7f3c1f7fe700 1 --2- 172.21.15.203:0/4192739806 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3c0c0563d0 0x7f3c0c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:18.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.639+0000 7f3c1f7fe700 1 -- 172.21.15.203:0/4192739806 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f3c2006fd20 con 0x7f3c28065e00 2021-01-06T22:08:18.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.639+0000 7f3c277fe700 1 --2- 172.21.15.203:0/4192739806 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3c0c0563d0 0x7f3c0c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:18.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.639+0000 7f3c277fe700 1 --2- 172.21.15.203:0/4192739806 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3c0c0563d0 0x7f3c0c058890 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f3c18006bc0 tx=0x7f3c18008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:18.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.647+0000 7f3c1f7fe700 1 -- 172.21.15.203:0/4192739806 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f3c200454f0 con 0x7f3c28065e00 2021-01-06T22:08:18.837 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.843+0000 7f3c2cb08700 1 -- 172.21.15.203:0/4192739806 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c28057980 con 0x7f3c28065e00 2021-01-06T22:08:18.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.843+0000 7f3c1f7fe700 1 -- 172.21.15.203:0/4192739806 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3580 (secure 0 0 0) 0x7f3c20016030 con 0x7f3c28065e00 2021-01-06T22:08:18.838 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:18.839 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:10.461108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:stopping","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[0],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:18.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.843+0000 7f3c2cb08700 1 -- 172.21.15.203:0/4192739806 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3c0c0563d0 msgr2=0x7f3c0c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:18.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.843+0000 7f3c2cb08700 1 --2- 172.21.15.203:0/4192739806 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3c0c0563d0 0x7f3c0c058890 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f3c18006bc0 tx=0x7f3c18008040).stop 2021-01-06T22:08:18.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.843+0000 7f3c2cb08700 1 -- 172.21.15.203:0/4192739806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c28065e00 msgr2=0x7f3c2814d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:18.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.843+0000 7f3c2cb08700 1 --2- 172.21.15.203:0/4192739806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c28065e00 0x7f3c2814d8f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f3c2000ed30 tx=0x7f3c200091d0).stop 2021-01-06T22:08:18.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.843+0000 7f3c2cb08700 1 -- 172.21.15.203:0/4192739806 shutdown_connections 2021-01-06T22:08:18.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.847+0000 7f3c2cb08700 1 --2- 172.21.15.203:0/4192739806 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3c0c0563d0 0x7f3c0c058890 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:18.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.847+0000 7f3c2cb08700 1 --2- 172.21.15.203:0/4192739806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c28065e00 0x7f3c2814d8f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:18.842 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.847+0000 7f3c2cb08700 1 --2- 172.21.15.203:0/4192739806 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c280bb3f0 0x7f3c280b8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:18.842 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.847+0000 7f3c2cb08700 1 --2- 172.21.15.203:0/4192739806 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c280bb7d0 0x7f3c2814d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:18.842 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.847+0000 7f3c2cb08700 1 -- 172.21.15.203:0/4192739806 >> 172.21.15.203:0/4192739806 conn(0x7f3c28061530 msgr2=0x7f3c28155750 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:18.842 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.847+0000 7f3c2cb08700 1 -- 172.21.15.203:0/4192739806 shutdown_connections 2021-01-06T22:08:18.842 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:18.847+0000 7f3c2cb08700 1 -- 172.21.15.203:0/4192739806 wait complete. 2021-01-06T22:08:18.843 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 30 2021-01-06T22:08:18.853 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:08:20.854 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:20.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.995+0000 7f19c484f700 1 Processor -- start 2021-01-06T22:08:20.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.995+0000 7f19c484f700 1 -- start start 2021-01-06T22:08:20.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19c484f700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19c00c44f0 0x7f19c00c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:20.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19c484f700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19c0060fc0 0x7f19c0150e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:20.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19c484f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c00c0f50 con 0x7f19c00c48d0 2021-01-06T22:08:20.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19c484f700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c0066cf0 con 0x7f19c00c44f0 2021-01-06T22:08:20.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19c484f700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c0060750 con 0x7f19c0060fc0 2021-01-06T22:08:20.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19beffd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f19c00c48d0 0x7f19c0150730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48968/0 (socket says 172.21.15.203:48968) 2021-01-06T22:08:20.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19beffd700 1 -- 172.21.15.203:0/883349735 learned_addr learned my addr 172.21.15.203:0/883349735 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:20.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19bffff700 1 --2- 172.21.15.203:0/883349735 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19c0060fc0 0x7f19c0150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:20.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 -- 172.21.15.203:0/883349735 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1588505404 0 0) 0x7f19c00c0f50 con 0x7f19c00c48d0 2021-01-06T22:08:20.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 -- 172.21.15.203:0/883349735 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19a0003230 con 0x7f19c00c48d0 2021-01-06T22:08:20.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19bf7fe700 1 --2- 172.21.15.203:0/883349735 >> v2:172.21.15.91:3300/0 conn(0x7f19c00c44f0 0x7f19c00c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:20.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 -- 172.21.15.203:0/883349735 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2782078603 0 0) 0x7f19a0003230 con 0x7f19c00c48d0 2021-01-06T22:08:20.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 -- 172.21.15.203:0/883349735 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19c0060fc0 msgr2=0x7f19c0150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:20.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 --2- 172.21.15.203:0/883349735 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19c0060fc0 0x7f19c0150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:20.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 -- 172.21.15.203:0/883349735 >> v2:172.21.15.91:3300/0 conn(0x7f19c00c44f0 msgr2=0x7f19c00c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:20.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 --2- 172.21.15.203:0/883349735 >> v2:172.21.15.91:3300/0 conn(0x7f19c00c44f0 0x7f19c00c1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:20.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 -- 172.21.15.203:0/883349735 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c00609f0 con 0x7f19c00c48d0 2021-01-06T22:08:21.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 -- 172.21.15.203:0/883349735 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f19a80014e0 con 0x7f19c00c48d0 2021-01-06T22:08:21.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 -- 172.21.15.203:0/883349735 >> v1:172.21.15.203:6789/0 conn(0x7f19c00c48d0 legacy=0x7f19c0150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:21.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 --2- 172.21.15.203:0/883349735 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19c0060fc0 0x7f19a0003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:21.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 --2- 172.21.15.203:0/883349735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19a0003c30 0x7f19a00044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:21.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 --2- 172.21.15.203:0/883349735 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a00049c0 0x7f19a0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:21.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 -- 172.21.15.203:0/883349735 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c0066cf0 con 0x7f19a00049c0 2021-01-06T22:08:21.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 -- 172.21.15.203:0/883349735 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c0060750 con 0x7f19a0003c30 2021-01-06T22:08:21.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 -- 172.21.15.203:0/883349735 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a00041f0 con 0x7f19c0060fc0 2021-01-06T22:08:21.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19beffd700 1 --2- 172.21.15.203:0/883349735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19a0003c30 0x7f19a00044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:21.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19bf7fe700 1 --2- 172.21.15.203:0/883349735 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19c0060fc0 0x7f19a0003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:21.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19bffff700 1 --2- 172.21.15.203:0/883349735 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a00049c0 0x7f19a0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:21.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19beffd700 1 -- 172.21.15.203:0/883349735 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19c0060fc0 msgr2=0x7f19a0003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:21.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19beffd700 1 --2- 172.21.15.203:0/883349735 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19c0060fc0 0x7f19a0003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:21.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19beffd700 1 -- 172.21.15.203:0/883349735 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a00049c0 msgr2=0x7f19a0004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:21.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19beffd700 1 --2- 172.21.15.203:0/883349735 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a00049c0 0x7f19a0004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:21.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19beffd700 1 -- 172.21.15.203:0/883349735 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a0003230 con 0x7f19a0003c30 2021-01-06T22:08:21.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19beffd700 1 --2- 172.21.15.203:0/883349735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19a0003c30 0x7f19a00044d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f19a800c440 tx=0x7f19a80077f0).ready entity=mon.1 client_cookie=27bc779eedb19d05 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:21.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 -- 172.21.15.203:0/883349735 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a80072e0 con 0x7f19a0003c30 2021-01-06T22:08:21.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 -- 172.21.15.203:0/883349735 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a8007440 con 0x7f19a0003c30 2021-01-06T22:08:21.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:20.999+0000 7f19be7fc700 1 -- 172.21.15.203:0/883349735 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a8005810 con 0x7f19a0003c30 2021-01-06T22:08:21.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.003+0000 7f19c484f700 1 -- 172.21.15.203:0/883349735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19a0003c30 msgr2=0x7f19a00044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:21.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.003+0000 7f19c484f700 1 --2- 172.21.15.203:0/883349735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19a0003c30 0x7f19a00044d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f19a800c440 tx=0x7f19a80077f0).stop 2021-01-06T22:08:21.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.003+0000 7f19bffff700 1 -- 172.21.15.203:0/883349735 reap_dead start 2021-01-06T22:08:21.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.003+0000 7f19c484f700 1 -- 172.21.15.203:0/883349735 shutdown_connections 2021-01-06T22:08:21.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.003+0000 7f19c484f700 1 -- 172.21.15.203:0/883349735 >> 172.21.15.203:0/883349735 conn(0x7f19c0056cd0 msgr2=0x7f19c0061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:21.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.003+0000 7f19c484f700 1 -- 172.21.15.203:0/883349735 shutdown_connections 2021-01-06T22:08:21.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.003+0000 7f19c484f700 1 -- 172.21.15.203:0/883349735 wait complete. 2021-01-06T22:08:21.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.003+0000 7f19c484f700 1 Processor -- start 2021-01-06T22:08:21.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.003+0000 7f19c484f700 1 -- start start 2021-01-06T22:08:21.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19c484f700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a0003c30 0x7f19c00b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:21.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19c484f700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19c0151ae0 0x7f19c014db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:21.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19c484f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19c0161480 0x7f19c01618e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:21.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19c484f700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00bd190 con 0x7f19a0003c30 2021-01-06T22:08:21.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19c484f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c0160df0 con 0x7f19c0161480 2021-01-06T22:08:21.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19c484f700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00bb2b0 con 0x7f19c0151ae0 2021-01-06T22:08:21.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19bffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19c0161480 0x7f19c01618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:21.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19bffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19c0161480 0x7f19c01618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50390/0 (socket says 172.21.15.203:50390) 2021-01-06T22:08:21.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19bffff700 1 -- 172.21.15.203:0/3466815093 learned_addr learned my addr 172.21.15.203:0/3466815093 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:21.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19beffd700 1 --2- 172.21.15.203:0/3466815093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19c0151ae0 0x7f19c014db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:21.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19bf7fe700 1 --2- 172.21.15.203:0/3466815093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a0003c30 0x7f19c00b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:21.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19bffff700 1 -- 172.21.15.203:0/3466815093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19c0151ae0 msgr2=0x7f19c014db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:21.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19bffff700 1 --2- 172.21.15.203:0/3466815093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19c0151ae0 0x7f19c014db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:21.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19bffff700 1 -- 172.21.15.203:0/3466815093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a0003c30 msgr2=0x7f19c00b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:21.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19bffff700 1 --2- 172.21.15.203:0/3466815093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a0003c30 0x7f19c00b8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:21.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19bffff700 1 -- 172.21.15.203:0/3466815093 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c00ba620 con 0x7f19c0161480 2021-01-06T22:08:21.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19bffff700 1 --2- 172.21.15.203:0/3466815093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19c0161480 0x7f19c01618e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f19c00c1f70 tx=0x7f19b8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:21.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19b77fe700 1 -- 172.21.15.203:0/3466815093 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800e900 con 0x7f19c0161480 2021-01-06T22:08:21.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19b77fe700 1 -- 172.21.15.203:0/3466815093 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b800eaa0 con 0x7f19c0161480 2021-01-06T22:08:21.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19c484f700 1 -- 172.21.15.203:0/3466815093 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c00ba880 con 0x7f19c0161480 2021-01-06T22:08:21.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19b77fe700 1 -- 172.21.15.203:0/3466815093 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800ee60 con 0x7f19c0161480 2021-01-06T22:08:21.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.007+0000 7f19c484f700 1 -- 172.21.15.203:0/3466815093 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c0160fe0 con 0x7f19c0161480 2021-01-06T22:08:21.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.011+0000 7f19c484f700 1 -- 172.21.15.203:0/3466815093 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c00450a0 con 0x7f19c0161480 2021-01-06T22:08:21.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.011+0000 7f19b77fe700 1 -- 172.21.15.203:0/3466815093 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f19b8014640 con 0x7f19c0161480 2021-01-06T22:08:21.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.011+0000 7f19b77fe700 1 --2- 172.21.15.203:0/3466815093 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f19a40563d0 0x7f19a4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:21.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.011+0000 7f19b77fe700 1 -- 172.21.15.203:0/3466815093 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f19b806fc60 con 0x7f19c0161480 2021-01-06T22:08:21.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.011+0000 7f19bf7fe700 1 --2- 172.21.15.203:0/3466815093 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f19a40563d0 0x7f19a4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:21.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.011+0000 7f19bf7fe700 1 --2- 172.21.15.203:0/3466815093 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f19a40563d0 0x7f19a4058890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f19b000bb30 tx=0x7f19b0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:21.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.019+0000 7f19b77fe700 1 -- 172.21.15.203:0/3466815093 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f19b80453d0 con 0x7f19c0161480 2021-01-06T22:08:21.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.223+0000 7f19c484f700 1 -- 172.21.15.203:0/3466815093 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19c00aa090 con 0x7f19c0161480 2021-01-06T22:08:21.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.223+0000 7f19b77fe700 1 -- 172.21.15.203:0/3466815093 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3580 (secure 0 0 0) 0x7f19b80489f0 con 0x7f19c0161480 2021-01-06T22:08:21.218 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:21.218 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:10.461108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:stopping","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[0],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:21.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.227+0000 7f19c484f700 1 -- 172.21.15.203:0/3466815093 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f19a40563d0 msgr2=0x7f19a4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:21.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.227+0000 7f19c484f700 1 --2- 172.21.15.203:0/3466815093 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f19a40563d0 0x7f19a4058890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f19b000bb30 tx=0x7f19b0008040).stop 2021-01-06T22:08:21.222 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.227+0000 7f19c484f700 1 -- 172.21.15.203:0/3466815093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19c0161480 msgr2=0x7f19c01618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:21.222 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.227+0000 7f19c484f700 1 --2- 172.21.15.203:0/3466815093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19c0161480 0x7f19c01618e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f19c00c1f70 tx=0x7f19b8008040).stop 2021-01-06T22:08:21.222 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.227+0000 7f19c484f700 1 -- 172.21.15.203:0/3466815093 shutdown_connections 2021-01-06T22:08:21.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.227+0000 7f19c484f700 1 --2- 172.21.15.203:0/3466815093 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f19a40563d0 0x7f19a4058890 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:21.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.227+0000 7f19c484f700 1 --2- 172.21.15.203:0/3466815093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19c0161480 0x7f19c01618e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:21.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.227+0000 7f19c484f700 1 --2- 172.21.15.203:0/3466815093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a0003c30 0x7f19c00b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:21.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.227+0000 7f19c484f700 1 --2- 172.21.15.203:0/3466815093 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19c0151ae0 0x7f19c014db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:21.224 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.227+0000 7f19c484f700 1 -- 172.21.15.203:0/3466815093 >> 172.21.15.203:0/3466815093 conn(0x7f19c0056cd0 msgr2=0x7f19c00bb660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:21.224 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.227+0000 7f19c484f700 1 -- 172.21.15.203:0/3466815093 shutdown_connections 2021-01-06T22:08:21.224 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:21.227+0000 7f19c484f700 1 -- 172.21.15.203:0/3466815093 wait complete. 2021-01-06T22:08:21.225 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 30 2021-01-06T22:08:21.240 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:08:23.241 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:23.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a9f74700 1 Processor -- start 2021-01-06T22:08:23.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a9f74700 1 -- start start 2021-01-06T22:08:23.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a9f74700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45a40bf290 0x7f45a4063730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:23.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a9f74700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f45a414e680 0x7f45a40c1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:23.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a9f74700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a414eb50 con 0x7f45a40bf670 2021-01-06T22:08:23.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a9f74700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a40c0f80 con 0x7f45a414e680 2021-01-06T22:08:23.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a9f74700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40c1220 con 0x7f45a40bf290 2021-01-06T22:08:23.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a2ffd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f45a40bf670 0x7f45a414df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:48988/0 (socket says 172.21.15.203:48988) 2021-01-06T22:08:23.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a2ffd700 1 -- 172.21.15.203:0/966762304 learned_addr learned my addr 172.21.15.203:0/966762304 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:23.373 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a3fff700 1 --2- 172.21.15.203:0/966762304 >> v2:172.21.15.203:3300/0 conn(0x7f45a414e680 0x7f45a40c1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:23.374 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a37fe700 1 --2- 172.21.15.203:0/966762304 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45a40bf290 0x7f45a4063730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:23.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a3fff700 1 -- 172.21.15.203:0/966762304 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45a40bf290 msgr2=0x7f45a4063730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:23.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a3fff700 1 --2- 172.21.15.203:0/966762304 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45a40bf290 0x7f45a4063730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:23.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a3fff700 1 -- 172.21.15.203:0/966762304 >> v1:172.21.15.203:6789/0 conn(0x7f45a40bf670 legacy=0x7f45a414df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:23.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a3fff700 1 -- 172.21.15.203:0/966762304 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a40c14c0 con 0x7f45a414e680 2021-01-06T22:08:23.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a37fe700 1 --2- 172.21.15.203:0/966762304 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45a40bf290 0x7f45a4063730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:08:23.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.375+0000 7f45a27fc700 1 -- 172.21.15.203:0/966762304 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3526436492 0 0) 0x7f45a414eb50 con 0x7f45a40bf670 2021-01-06T22:08:23.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.379+0000 7f45a3fff700 1 --2- 172.21.15.203:0/966762304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a414e680 0x7f45a40c1c00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f459c009c90 tx=0x7f459c0091d0).ready entity=mon.1 client_cookie=454c2bfb0287753c server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:23.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.379+0000 7f45a27fc700 1 -- 172.21.15.203:0/966762304 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c00bc50 con 0x7f45a414e680 2021-01-06T22:08:23.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.379+0000 7f45a27fc700 1 -- 172.21.15.203:0/966762304 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c00bdb0 con 0x7f45a414e680 2021-01-06T22:08:23.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.379+0000 7f45a27fc700 1 -- 172.21.15.203:0/966762304 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c011610 con 0x7f45a414e680 2021-01-06T22:08:23.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.379+0000 7f45a9f74700 1 -- 172.21.15.203:0/966762304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a414e680 msgr2=0x7f45a40c1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:23.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.379+0000 7f45a9f74700 1 --2- 172.21.15.203:0/966762304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a414e680 0x7f45a40c1c00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f459c009c90 tx=0x7f459c0091d0).stop 2021-01-06T22:08:23.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.379+0000 7f45a9f74700 1 -- 172.21.15.203:0/966762304 shutdown_connections 2021-01-06T22:08:23.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.379+0000 7f45a9f74700 1 --2- 172.21.15.203:0/966762304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a414e680 0x7f45a40c1c00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:23.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.379+0000 7f45a9f74700 1 --2- 172.21.15.203:0/966762304 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45a40bf290 0x7f45a4063730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:23.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.379+0000 7f45a9f74700 1 -- 172.21.15.203:0/966762304 >> 172.21.15.203:0/966762304 conn(0x7f45a40626e0 msgr2=0x7f45a4061a30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:23.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.379+0000 7f45a9f74700 1 -- 172.21.15.203:0/966762304 shutdown_connections 2021-01-06T22:08:23.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.379+0000 7f45a9f74700 1 -- 172.21.15.203:0/966762304 wait complete. 2021-01-06T22:08:23.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.379+0000 7f45a9f74700 1 Processor -- start 2021-01-06T22:08:23.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.379+0000 7f45a9f74700 1 -- start start 2021-01-06T22:08:23.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a9f74700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45a40bf290 0x7f45a414d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:23.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a9f74700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45a40bf670 0x7f45a415e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:23.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a9f74700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a414e680 0x7f45a415e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:23.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a9f74700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a415da00 con 0x7f45a40bf670 2021-01-06T22:08:23.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a9f74700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a4152100 con 0x7f45a414e680 2021-01-06T22:08:23.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a9f74700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a4151960 con 0x7f45a40bf290 2021-01-06T22:08:23.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a414e680 0x7f45a415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:23.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a414e680 0x7f45a415e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50404/0 (socket says 172.21.15.203:50404) 2021-01-06T22:08:23.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a3fff700 1 -- 172.21.15.203:0/3578353573 learned_addr learned my addr 172.21.15.203:0/3578353573 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:23.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a37fe700 1 --2- 172.21.15.203:0/3578353573 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45a40bf290 0x7f45a414d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:23.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a2ffd700 1 --2- 172.21.15.203:0/3578353573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45a40bf670 0x7f45a415e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:23.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a3fff700 1 -- 172.21.15.203:0/3578353573 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45a40bf290 msgr2=0x7f45a414d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:23.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a3fff700 1 --2- 172.21.15.203:0/3578353573 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45a40bf290 0x7f45a414d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:23.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a3fff700 1 -- 172.21.15.203:0/3578353573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45a40bf670 msgr2=0x7f45a415e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:23.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a3fff700 1 --2- 172.21.15.203:0/3578353573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45a40bf670 0x7f45a415e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:23.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a3fff700 1 -- 172.21.15.203:0/3578353573 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a40c3d40 con 0x7f45a414e680 2021-01-06T22:08:23.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a3fff700 1 --2- 172.21.15.203:0/3578353573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a414e680 0x7f45a415e580 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f459c009210 tx=0x7f459c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:23.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f458b7fe700 1 -- 172.21.15.203:0/3578353573 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c01c700 con 0x7f45a414e680 2021-01-06T22:08:23.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f458b7fe700 1 -- 172.21.15.203:0/3578353573 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c01c860 con 0x7f45a414e680 2021-01-06T22:08:23.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a9f74700 1 -- 172.21.15.203:0/3578353573 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a4151bc0 con 0x7f45a414e680 2021-01-06T22:08:23.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f458b7fe700 1 -- 172.21.15.203:0/3578353573 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c01cbe0 con 0x7f45a414e680 2021-01-06T22:08:23.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a9f74700 1 -- 172.21.15.203:0/3578353573 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a415dbf0 con 0x7f45a414e680 2021-01-06T22:08:23.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.383+0000 7f45a9f74700 1 -- 172.21.15.203:0/3578353573 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a415ddb0 con 0x7f45a414e680 2021-01-06T22:08:23.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.387+0000 7f458b7fe700 1 -- 172.21.15.203:0/3578353573 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f459c024030 con 0x7f45a414e680 2021-01-06T22:08:23.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.387+0000 7f458b7fe700 1 --2- 172.21.15.203:0/3578353573 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f45840563d0 0x7f4584058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:23.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.387+0000 7f458b7fe700 1 -- 172.21.15.203:0/3578353573 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f459c070d90 con 0x7f45a414e680 2021-01-06T22:08:23.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.387+0000 7f45a37fe700 1 --2- 172.21.15.203:0/3578353573 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f45840563d0 0x7f4584058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:23.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.387+0000 7f45a37fe700 1 --2- 172.21.15.203:0/3578353573 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f45840563d0 0x7f4584058890 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f459800bb10 tx=0x7f4598008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:23.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.391+0000 7f458b7fe700 1 -- 172.21.15.203:0/3578353573 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f459c046610 con 0x7f45a414e680 2021-01-06T22:08:23.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.591+0000 7f45a9f74700 1 -- 172.21.15.203:0/3578353573 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45a40447f0 con 0x7f45a414e680 2021-01-06T22:08:23.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.591+0000 7f458b7fe700 1 -- 172.21.15.203:0/3578353573 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3580 (secure 0 0 0) 0x7f459c022030 con 0x7f45a414e680 2021-01-06T22:08:23.586 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:23.587 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:10.461108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:stopping","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[0],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:23.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.591+0000 7f45a9f74700 1 -- 172.21.15.203:0/3578353573 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f45840563d0 msgr2=0x7f4584058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:23.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.591+0000 7f45a9f74700 1 --2- 172.21.15.203:0/3578353573 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f45840563d0 0x7f4584058890 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f459800bb10 tx=0x7f4598008040).stop 2021-01-06T22:08:23.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.591+0000 7f45a9f74700 1 -- 172.21.15.203:0/3578353573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a414e680 msgr2=0x7f45a415e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:23.590 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.591+0000 7f45a9f74700 1 --2- 172.21.15.203:0/3578353573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a414e680 0x7f45a415e580 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f459c009210 tx=0x7f459c017000).stop 2021-01-06T22:08:23.590 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.591+0000 7f45a9f74700 1 -- 172.21.15.203:0/3578353573 shutdown_connections 2021-01-06T22:08:23.591 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.591+0000 7f45a9f74700 1 --2- 172.21.15.203:0/3578353573 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f45840563d0 0x7f4584058890 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:23.591 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.591+0000 7f45a9f74700 1 --2- 172.21.15.203:0/3578353573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a414e680 0x7f45a415e580 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:23.591 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.591+0000 7f45a9f74700 1 --2- 172.21.15.203:0/3578353573 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45a40bf290 0x7f45a414d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:23.592 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.591+0000 7f45a9f74700 1 --2- 172.21.15.203:0/3578353573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45a40bf670 0x7f45a415e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:23.592 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.591+0000 7f45a9f74700 1 -- 172.21.15.203:0/3578353573 >> 172.21.15.203:0/3578353573 conn(0x7f45a40626e0 msgr2=0x7f45a4142480 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:23.593 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.591+0000 7f45a9f74700 1 -- 172.21.15.203:0/3578353573 shutdown_connections 2021-01-06T22:08:23.593 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:23.591+0000 7f45a9f74700 1 -- 172.21.15.203:0/3578353573 wait complete. 2021-01-06T22:08:23.593 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 30 2021-01-06T22:08:23.607 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:08:25.608 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:25.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.743+0000 7faf3e004700 1 Processor -- start 2021-01-06T22:08:25.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.743+0000 7faf3e004700 1 -- start start 2021-01-06T22:08:25.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.743+0000 7faf3e004700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faf380633f0 0x7faf3814e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:25.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.743+0000 7faf3e004700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf380be990 con 0x7faf38063010 2021-01-06T22:08:25.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.743+0000 7faf3e004700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf380be6f0 con 0x7faf380be1b0 2021-01-06T22:08:25.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.743+0000 7faf3e004700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3814ec60 con 0x7faf380633f0 2021-01-06T22:08:25.742 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf3d803700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7faf380be1b0 0x7faf38147790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49006/0 (socket says 172.21.15.203:49006) 2021-01-06T22:08:25.742 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf3d803700 1 -- 172.21.15.203:0/3090516730 learned_addr learned my addr 172.21.15.203:0/3090516730 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:25.742 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf3c801700 1 --2- 172.21.15.203:0/3090516730 >> v2:172.21.15.91:3300/0 conn(0x7faf380633f0 0x7faf3814e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:25.742 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf37fff700 1 -- 172.21.15.203:0/3090516730 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3429667316 0 0) 0x7faf380be6f0 con 0x7faf380be1b0 2021-01-06T22:08:25.743 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf37fff700 1 -- 172.21.15.203:0/3090516730 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf24003230 con 0x7faf380be1b0 2021-01-06T22:08:25.743 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf37fff700 1 -- 172.21.15.203:0/3090516730 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3736059924 0 0) 0x7faf24003230 con 0x7faf380be1b0 2021-01-06T22:08:25.743 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf3c801700 1 -- 172.21.15.203:0/3090516730 >> v1:172.21.15.203:6789/0 conn(0x7faf380be1b0 legacy=0x7faf38147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:25.743 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf3c801700 1 -- 172.21.15.203:0/3090516730 >> v1:172.21.15.91:6789/0 conn(0x7faf38063010 legacy=0x7faf38064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:25.744 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf3c801700 1 -- 172.21.15.203:0/3090516730 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380c1050 con 0x7faf380633f0 2021-01-06T22:08:25.749 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf3c801700 1 --2- 172.21.15.203:0/3090516730 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf380633f0 0x7faf3814e770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7faf2c00b900 tx=0x7faf2c006de0).ready entity=mon.0 client_cookie=b7ff6f15e5b12849 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:25.749 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf37fff700 1 -- 172.21.15.203:0/3090516730 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30001860 con 0x7faf380633f0 2021-01-06T22:08:25.749 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf37fff700 1 -- 172.21.15.203:0/3090516730 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c012410 con 0x7faf380633f0 2021-01-06T22:08:25.749 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf37fff700 1 -- 172.21.15.203:0/3090516730 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c012790 con 0x7faf380633f0 2021-01-06T22:08:25.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf3e004700 1 -- 172.21.15.203:0/3090516730 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf380633f0 msgr2=0x7faf3814e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:25.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf3e004700 1 --2- 172.21.15.203:0/3090516730 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf380633f0 0x7faf3814e770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7faf2c00b900 tx=0x7faf2c006de0).stop 2021-01-06T22:08:25.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf3e004700 1 -- 172.21.15.203:0/3090516730 shutdown_connections 2021-01-06T22:08:25.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf3e004700 1 --2- 172.21.15.203:0/3090516730 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf380633f0 0x7faf3814e770 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:25.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf3e004700 1 -- 172.21.15.203:0/3090516730 >> 172.21.15.203:0/3090516730 conn(0x7faf38067e20 msgr2=0x7faf38068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:25.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf3e004700 1 -- 172.21.15.203:0/3090516730 shutdown_connections 2021-01-06T22:08:25.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf3e004700 1 -- 172.21.15.203:0/3090516730 wait complete. 2021-01-06T22:08:25.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.747+0000 7faf3e004700 1 Processor -- start 2021-01-06T22:08:25.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3e004700 1 -- start start 2021-01-06T22:08:25.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3e004700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf38063010 0x7faf3815ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:25.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3e004700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf380633f0 0x7faf38148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:25.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3e004700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faf380be1b0 0x7faf3815ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:25.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3e004700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf38138040 con 0x7faf380633f0 2021-01-06T22:08:25.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3e004700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3815b240 con 0x7faf38063010 2021-01-06T22:08:25.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3e004700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3815e8a0 con 0x7faf380be1b0 2021-01-06T22:08:25.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3d002700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf38063010 0x7faf3815ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:25.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3d002700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf38063010 0x7faf3815ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50414/0 (socket says 172.21.15.203:50414) 2021-01-06T22:08:25.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3d002700 1 -- 172.21.15.203:0/2948944750 learned_addr learned my addr 172.21.15.203:0/2948944750 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:25.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3c801700 1 --2- 172.21.15.203:0/2948944750 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf380633f0 0x7faf38148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:25.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3d803700 1 --2- 172.21.15.203:0/2948944750 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faf380be1b0 0x7faf3815ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:25.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3d002700 1 -- 172.21.15.203:0/2948944750 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faf380be1b0 msgr2=0x7faf3815ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:25.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3d002700 1 --2- 172.21.15.203:0/2948944750 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faf380be1b0 0x7faf3815ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:25.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3d002700 1 -- 172.21.15.203:0/2948944750 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf380633f0 msgr2=0x7faf38148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:25.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3d002700 1 --2- 172.21.15.203:0/2948944750 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf380633f0 0x7faf38148810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:25.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3d002700 1 -- 172.21.15.203:0/2948944750 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3814b400 con 0x7faf38063010 2021-01-06T22:08:25.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3d002700 1 --2- 172.21.15.203:0/2948944750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf38063010 0x7faf3815ad50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7faf2800c7c0 tx=0x7faf28007bf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:25.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf357fa700 1 -- 172.21.15.203:0/2948944750 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2800e560 con 0x7faf38063010 2021-01-06T22:08:25.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3e004700 1 -- 172.21.15.203:0/2948944750 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3814a700 con 0x7faf38063010 2021-01-06T22:08:25.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf357fa700 1 -- 172.21.15.203:0/2948944750 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2800e6c0 con 0x7faf38063010 2021-01-06T22:08:25.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf357fa700 1 -- 172.21.15.203:0/2948944750 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2800ea60 con 0x7faf38063010 2021-01-06T22:08:25.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3e004700 1 -- 172.21.15.203:0/2948944750 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3814ac40 con 0x7faf38063010 2021-01-06T22:08:25.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.751+0000 7faf3e004700 1 -- 172.21.15.203:0/2948944750 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3815ea30 con 0x7faf38063010 2021-01-06T22:08:25.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.755+0000 7faf357fa700 1 -- 172.21.15.203:0/2948944750 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7faf28019030 con 0x7faf38063010 2021-01-06T22:08:25.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.755+0000 7faf357fa700 1 --2- 172.21.15.203:0/2948944750 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faf18056400 0x7faf180588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:25.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.755+0000 7faf357fa700 1 -- 172.21.15.203:0/2948944750 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7faf2806ec80 con 0x7faf38063010 2021-01-06T22:08:25.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.755+0000 7faf3c801700 1 --2- 172.21.15.203:0/2948944750 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faf18056400 0x7faf180588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:25.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.755+0000 7faf3c801700 1 --2- 172.21.15.203:0/2948944750 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faf18056400 0x7faf180588c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7faf2c006b70 tx=0x7faf3814b7e0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:25.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.763+0000 7faf357fa700 1 -- 172.21.15.203:0/2948944750 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7faf3815ea30 con 0x7faf38063010 2021-01-06T22:08:25.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.963+0000 7faf3e004700 1 -- 172.21.15.203:0/2948944750 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf380450a0 con 0x7faf38063010 2021-01-06T22:08:25.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.963+0000 7faf357fa700 1 -- 172.21.15.203:0/2948944750 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3580 (secure 0 0 0) 0x7faf28044450 con 0x7faf38063010 2021-01-06T22:08:25.959 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:25.960 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:10.461108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:stopping","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[0],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:25.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.967+0000 7faf3e004700 1 -- 172.21.15.203:0/2948944750 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faf18056400 msgr2=0x7faf180588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:25.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.967+0000 7faf3e004700 1 --2- 172.21.15.203:0/2948944750 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faf18056400 0x7faf180588c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7faf2c006b70 tx=0x7faf3814b7e0).stop 2021-01-06T22:08:25.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.967+0000 7faf3e004700 1 -- 172.21.15.203:0/2948944750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf38063010 msgr2=0x7faf3815ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:25.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.967+0000 7faf3e004700 1 --2- 172.21.15.203:0/2948944750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf38063010 0x7faf3815ad50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7faf2800c7c0 tx=0x7faf28007bf0).stop 2021-01-06T22:08:25.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.967+0000 7faf3e004700 1 -- 172.21.15.203:0/2948944750 shutdown_connections 2021-01-06T22:08:25.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.967+0000 7faf3e004700 1 --2- 172.21.15.203:0/2948944750 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faf18056400 0x7faf180588c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:25.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.967+0000 7faf3e004700 1 --2- 172.21.15.203:0/2948944750 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faf380be1b0 0x7faf3815ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:25.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.967+0000 7faf3e004700 1 --2- 172.21.15.203:0/2948944750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf38063010 0x7faf3815ad50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:25.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.967+0000 7faf3e004700 1 --2- 172.21.15.203:0/2948944750 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf380633f0 0x7faf38148810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:25.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.967+0000 7faf3e004700 1 -- 172.21.15.203:0/2948944750 >> 172.21.15.203:0/2948944750 conn(0x7faf38067e20 msgr2=0x7faf3813ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:25.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.967+0000 7faf3e004700 1 -- 172.21.15.203:0/2948944750 shutdown_connections 2021-01-06T22:08:25.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:25.967+0000 7faf3e004700 1 -- 172.21.15.203:0/2948944750 wait complete. 2021-01-06T22:08:25.967 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 30 2021-01-06T22:08:25.974 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2021-01-06T22:08:25.975 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 30, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 30, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4273": { "addr": "172.21.15.203:6837/3277561222", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6836", "nonce": 3277561222, "type": "v2" }, { "addr": "172.21.15.203:6837", "nonce": 3277561222, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4273, "incarnation": 9, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 3 }, "gid_4274": { "addr": "172.21.15.91:6837/3888667247", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6835", "nonce": 3888667247, "type": "v2" }, { "addr": "172.21.15.91:6837", "nonce": 3888667247, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4274, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 2 }, "gid_4867": { "addr": "172.21.15.203:6835/3382231276", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6834", "nonce": 3382231276, "type": "v2" }, { "addr": "172.21.15.203:6835", "nonce": 3382231276, "type": "v1" } ] }, "export_targets": [ 0 ], "features": 4540138297136906239, "flags": 0, "gid": 4867, "incarnation": 20, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:stopping", "state_seq": 22 } }, "last_failure": 0, "last_failure_osd_epoch": 25, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2021-01-06T22:08:10.461108+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4274, "mds_1": 4273, "mds_2": 4867 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.91:6836/2428882934", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6834", "nonce": 2428882934, "type": "v2" }, { "addr": "172.21.15.91:6836", "nonce": 2428882934, "type": "v1" } ] }, "epoch": 26, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4724, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2021-01-06T22:08:27.976 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:28.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25ca310700 1 Processor -- start 2021-01-06T22:08:28.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25ca310700 1 -- start start 2021-01-06T22:08:28.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25ca310700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25c40bccb0 0x7f25c40641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:28.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25ca310700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c40bacb0 con 0x7f25c41513f0 2021-01-06T22:08:28.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25ca310700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c4152000 con 0x7f25c40c0fa0 2021-01-06T22:08:28.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25ca310700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c4067e50 con 0x7f25c40bccb0 2021-01-06T22:08:28.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25c8b0d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f25c40c0fa0 0x7f25c414ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49018/0 (socket says 172.21.15.203:49018) 2021-01-06T22:08:28.114 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25c8b0d700 1 -- 172.21.15.203:0/3666406244 learned_addr learned my addr 172.21.15.203:0/3666406244 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:28.114 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25c9b0f700 1 --2- 172.21.15.203:0/3666406244 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25c40bccb0 0x7f25c40641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:28.115 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 -- 172.21.15.203:0/3666406244 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841072963 0 0) 0x7f25c4152000 con 0x7f25c40c0fa0 2021-01-06T22:08:28.115 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 -- 172.21.15.203:0/3666406244 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25ac003230 con 0x7f25c40c0fa0 2021-01-06T22:08:28.115 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 -- 172.21.15.203:0/3666406244 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125433612 0 0) 0x7f25c00010d0 con 0x7f25c41513f0 2021-01-06T22:08:28.115 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 -- 172.21.15.203:0/3666406244 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25c4152000 con 0x7f25c41513f0 2021-01-06T22:08:28.116 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 -- 172.21.15.203:0/3666406244 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3273357685 0 0) 0x7f25ac003230 con 0x7f25c40c0fa0 2021-01-06T22:08:28.116 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 -- 172.21.15.203:0/3666406244 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25c40bccb0 msgr2=0x7f25c40641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:28.116 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 --2- 172.21.15.203:0/3666406244 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25c40bccb0 0x7f25c40641e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:28.116 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 -- 172.21.15.203:0/3666406244 >> v1:172.21.15.91:6789/0 conn(0x7f25c41513f0 legacy=0x7f25c41517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:28.117 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 -- 172.21.15.203:0/3666406244 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c40680f0 con 0x7f25c40c0fa0 2021-01-06T22:08:28.117 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 -- 172.21.15.203:0/3666406244 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f25b40014e0 con 0x7f25c40c0fa0 2021-01-06T22:08:28.117 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 -- 172.21.15.203:0/3666406244 >> v1:172.21.15.203:6789/0 conn(0x7f25c40c0fa0 legacy=0x7f25c414ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:28.117 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 --2- 172.21.15.203:0/3666406244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25ac0038b0 0x7f25ac003c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:28.117 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 --2- 172.21.15.203:0/3666406244 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25c40bccb0 0x7f25ac0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:28.118 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 --2- 172.21.15.203:0/3666406244 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25ac0049c0 0x7f25ac004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:28.118 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 -- 172.21.15.203:0/3666406244 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c4067e50 con 0x7f25ac0049c0 2021-01-06T22:08:28.118 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 -- 172.21.15.203:0/3666406244 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0042c0 con 0x7f25ac0038b0 2021-01-06T22:08:28.118 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25c930e700 1 --2- 172.21.15.203:0/3666406244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25ac0038b0 0x7f25ac003c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:28.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 -- 172.21.15.203:0/3666406244 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac00b720 con 0x7f25c40bccb0 2021-01-06T22:08:28.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25c8b0d700 1 --2- 172.21.15.203:0/3666406244 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25c40bccb0 0x7f25ac0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:28.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25c9b0f700 1 --2- 172.21.15.203:0/3666406244 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25ac0049c0 0x7f25ac004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:28.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25c930e700 1 -- 172.21.15.203:0/3666406244 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25c40bccb0 msgr2=0x7f25ac0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:28.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25c930e700 1 --2- 172.21.15.203:0/3666406244 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25c40bccb0 0x7f25ac0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:28.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25c930e700 1 -- 172.21.15.203:0/3666406244 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25ac0049c0 msgr2=0x7f25ac004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:28.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25c930e700 1 --2- 172.21.15.203:0/3666406244 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25ac0049c0 0x7f25ac004e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:28.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25c930e700 1 -- 172.21.15.203:0/3666406244 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac003230 con 0x7f25ac0038b0 2021-01-06T22:08:28.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25c930e700 1 --2- 172.21.15.203:0/3666406244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25ac0038b0 0x7f25ac003c90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f25c0007c80 tx=0x7f25c0001b40).ready entity=mon.1 client_cookie=40b1d80ab4f71b7e server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:28.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 -- 172.21.15.203:0/3666406244 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c000cb80 con 0x7f25ac0038b0 2021-01-06T22:08:28.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 -- 172.21.15.203:0/3666406244 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c000cce0 con 0x7f25ac0038b0 2021-01-06T22:08:28.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.115+0000 7f25b3fff700 1 -- 172.21.15.203:0/3666406244 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c0012630 con 0x7f25ac0038b0 2021-01-06T22:08:28.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25ca310700 1 -- 172.21.15.203:0/3666406244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25ac0038b0 msgr2=0x7f25ac003c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:28.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25ca310700 1 --2- 172.21.15.203:0/3666406244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25ac0038b0 0x7f25ac003c90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f25c0007c80 tx=0x7f25c0001b40).stop 2021-01-06T22:08:28.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25c9b0f700 1 -- 172.21.15.203:0/3666406244 reap_dead start 2021-01-06T22:08:28.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25ca310700 1 -- 172.21.15.203:0/3666406244 shutdown_connections 2021-01-06T22:08:28.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25ca310700 1 -- 172.21.15.203:0/3666406244 >> 172.21.15.203:0/3666406244 conn(0x7f25c40667b0 msgr2=0x7f25c40c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:28.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25ca310700 1 -- 172.21.15.203:0/3666406244 shutdown_connections 2021-01-06T22:08:28.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25ca310700 1 -- 172.21.15.203:0/3666406244 wait complete. 2021-01-06T22:08:28.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25ca310700 1 Processor -- start 2021-01-06T22:08:28.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25ca310700 1 -- start start 2021-01-06T22:08:28.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25ca310700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25c4145190 0x7f25c415dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:28.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25ca310700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25c4145570 0x7f25c414fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:28.128 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25ca310700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25c41459f0 0x7f25c4150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:28.128 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25ca310700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c414d0d0 con 0x7f25c4145190 2021-01-06T22:08:28.128 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25ca310700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c414c8f0 con 0x7f25c41459f0 2021-01-06T22:08:28.128 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25ca310700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c414cb90 con 0x7f25c4145570 2021-01-06T22:08:28.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25c9b0f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25c41459f0 0x7f25c4150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:28.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25c9b0f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25c41459f0 0x7f25c4150060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50438/0 (socket says 172.21.15.203:50438) 2021-01-06T22:08:28.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25c9b0f700 1 -- 172.21.15.203:0/1539803976 learned_addr learned my addr 172.21.15.203:0/1539803976 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:28.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25c930e700 1 --2- 172.21.15.203:0/1539803976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25c4145190 0x7f25c415dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:28.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25c8b0d700 1 --2- 172.21.15.203:0/1539803976 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25c4145570 0x7f25c414fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:28.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25c9b0f700 1 -- 172.21.15.203:0/1539803976 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25c4145570 msgr2=0x7f25c414fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:28.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25c9b0f700 1 --2- 172.21.15.203:0/1539803976 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25c4145570 0x7f25c414fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:28.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25c9b0f700 1 -- 172.21.15.203:0/1539803976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25c4145190 msgr2=0x7f25c415dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:28.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25c9b0f700 1 --2- 172.21.15.203:0/1539803976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25c4145190 0x7f25c415dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:28.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25c9b0f700 1 -- 172.21.15.203:0/1539803976 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c414ce30 con 0x7f25c41459f0 2021-01-06T22:08:28.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.119+0000 7f25c9b0f700 1 --2- 172.21.15.203:0/1539803976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25c41459f0 0x7f25c4150060 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f25bc0090e0 tx=0x7f25bc008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:28.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.123+0000 7f25b17fa700 1 -- 172.21.15.203:0/1539803976 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc014670 con 0x7f25c41459f0 2021-01-06T22:08:28.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.123+0000 7f25b17fa700 1 -- 172.21.15.203:0/1539803976 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc0147f0 con 0x7f25c41459f0 2021-01-06T22:08:28.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.123+0000 7f25b17fa700 1 -- 172.21.15.203:0/1539803976 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc014bb0 con 0x7f25c41459f0 2021-01-06T22:08:28.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.123+0000 7f25ca310700 1 -- 172.21.15.203:0/1539803976 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c414f190 con 0x7f25c41459f0 2021-01-06T22:08:28.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.123+0000 7f25ca310700 1 -- 172.21.15.203:0/1539803976 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c414f6d0 con 0x7f25c41459f0 2021-01-06T22:08:28.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.123+0000 7f25ca310700 1 -- 172.21.15.203:0/1539803976 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c415e3d0 con 0x7f25c41459f0 2021-01-06T22:08:28.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.123+0000 7f25b17fa700 1 -- 172.21.15.203:0/1539803976 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f25bc01f460 con 0x7f25c41459f0 2021-01-06T22:08:28.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.123+0000 7f25b17fa700 1 --2- 172.21.15.203:0/1539803976 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f25a40563d0 0x7f25a4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:28.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.123+0000 7f25b17fa700 1 -- 172.21.15.203:0/1539803976 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f25bc06fa80 con 0x7f25c41459f0 2021-01-06T22:08:28.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.123+0000 7f25c930e700 1 --2- 172.21.15.203:0/1539803976 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f25a40563d0 0x7f25a4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:28.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.123+0000 7f25c930e700 1 --2- 172.21.15.203:0/1539803976 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f25a40563d0 0x7f25a4058890 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f25c0000bc0 tx=0x7f25c0017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:28.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.131+0000 7f25b17fa700 1 -- 172.21.15.203:0/1539803976 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f25bc0451f0 con 0x7f25c41459f0 2021-01-06T22:08:28.331 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.335+0000 7f25ca310700 1 -- 172.21.15.203:0/1539803976 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25c40450a0 con 0x7f25c41459f0 2021-01-06T22:08:28.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.339+0000 7f25b17fa700 1 -- 172.21.15.203:0/1539803976 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3580 (secure 0 0 0) 0x7f25bc048810 con 0x7f25c41459f0 2021-01-06T22:08:28.333 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:28.333 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:10.461108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:stopping","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[0],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:28.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.339+0000 7f25ca310700 1 -- 172.21.15.203:0/1539803976 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f25a40563d0 msgr2=0x7f25a4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:28.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.339+0000 7f25ca310700 1 --2- 172.21.15.203:0/1539803976 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f25a40563d0 0x7f25a4058890 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f25c0000bc0 tx=0x7f25c0017000).stop 2021-01-06T22:08:28.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.339+0000 7f25ca310700 1 -- 172.21.15.203:0/1539803976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25c41459f0 msgr2=0x7f25c4150060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:28.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.339+0000 7f25ca310700 1 --2- 172.21.15.203:0/1539803976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25c41459f0 0x7f25c4150060 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f25bc0090e0 tx=0x7f25bc008040).stop 2021-01-06T22:08:28.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.339+0000 7f25ca310700 1 -- 172.21.15.203:0/1539803976 shutdown_connections 2021-01-06T22:08:28.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.339+0000 7f25ca310700 1 --2- 172.21.15.203:0/1539803976 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f25a40563d0 0x7f25a4058890 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:28.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.339+0000 7f25ca310700 1 --2- 172.21.15.203:0/1539803976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25c41459f0 0x7f25c4150060 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:28.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.339+0000 7f25ca310700 1 --2- 172.21.15.203:0/1539803976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25c4145190 0x7f25c415dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:28.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.339+0000 7f25ca310700 1 --2- 172.21.15.203:0/1539803976 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25c4145570 0x7f25c414fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:28.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.339+0000 7f25ca310700 1 -- 172.21.15.203:0/1539803976 >> 172.21.15.203:0/1539803976 conn(0x7f25c40667b0 msgr2=0x7f25c40bf440 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:28.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.339+0000 7f25ca310700 1 -- 172.21.15.203:0/1539803976 shutdown_connections 2021-01-06T22:08:28.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:28.339+0000 7f25ca310700 1 -- 172.21.15.203:0/1539803976 wait complete. 2021-01-06T22:08:28.338 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 30 2021-01-06T22:08:28.347 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:08:30.348 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:30.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.483+0000 7f7f928f2700 1 Processor -- start 2021-01-06T22:08:30.480 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.483+0000 7f7f928f2700 1 -- start start 2021-01-06T22:08:30.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f928f2700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7f8c0a7f90 0x7f7f8c065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:30.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f928f2700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7f8c0bc220 0x7f7f8c060f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:30.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f928f2700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8c064700 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f928f2700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8c152d60 con 0x7f7f8c0bc220 2021-01-06T22:08:30.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f928f2700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c153000 con 0x7f7f8c0a7f90 2021-01-06T22:08:30.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f918f0700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f7f8c0a7bb0 0x7f7f8c0626e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49038/0 (socket says 172.21.15.203:49038) 2021-01-06T22:08:30.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f918f0700 1 -- 172.21.15.203:0/1099413644 learned_addr learned my addr 172.21.15.203:0/1099413644 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:30.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f920f1700 1 --2- 172.21.15.203:0/1099413644 >> v2:172.21.15.91:3300/0 conn(0x7f7f8c0bc220 0x7f7f8c060f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:30.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f910ef700 1 --2- 172.21.15.203:0/1099413644 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7f8c0a7f90 0x7f7f8c065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:30.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f908ee700 1 -- 172.21.15.203:0/1099413644 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 909287318 0 0) 0x7f7f8c064700 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f908ee700 1 -- 172.21.15.203:0/1099413644 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f74003230 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f908ee700 1 -- 172.21.15.203:0/1099413644 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1469755646 0 0) 0x7f7f74003230 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f920f1700 1 -- 172.21.15.203:0/1099413644 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7f8c0a7f90 msgr2=0x7f7f8c065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:30.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f920f1700 1 --2- 172.21.15.203:0/1099413644 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7f8c0a7f90 0x7f7f8c065c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:30.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f920f1700 1 -- 172.21.15.203:0/1099413644 >> v1:172.21.15.203:6789/0 conn(0x7f7f8c0a7bb0 legacy=0x7f7f8c0626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:30.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f920f1700 1 -- 172.21.15.203:0/1099413644 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c060790 con 0x7f7f8c0bc220 2021-01-06T22:08:30.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f920f1700 1 --2- 172.21.15.203:0/1099413644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7f8c0bc220 0x7f7f8c060f20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f7f84009c90 tx=0x7f7f840091d0).ready entity=mon.0 client_cookie=b774ea995bdb263e server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:30.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f908ee700 1 -- 172.21.15.203:0/1099413644 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f880014e0 con 0x7f7f8c0bc220 2021-01-06T22:08:30.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f908ee700 1 -- 172.21.15.203:0/1099413644 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8400b6b0 con 0x7f7f8c0bc220 2021-01-06T22:08:30.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f908ee700 1 -- 172.21.15.203:0/1099413644 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8400ba30 con 0x7f7f8c0bc220 2021-01-06T22:08:30.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f928f2700 1 -- 172.21.15.203:0/1099413644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7f8c0bc220 msgr2=0x7f7f8c060f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:30.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f928f2700 1 --2- 172.21.15.203:0/1099413644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7f8c0bc220 0x7f7f8c060f20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f7f84009c90 tx=0x7f7f840091d0).stop 2021-01-06T22:08:30.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f928f2700 1 -- 172.21.15.203:0/1099413644 shutdown_connections 2021-01-06T22:08:30.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f928f2700 1 --2- 172.21.15.203:0/1099413644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7f8c0bc220 0x7f7f8c060f20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:30.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f928f2700 1 --2- 172.21.15.203:0/1099413644 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7f8c0a7f90 0x7f7f8c065c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:30.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f928f2700 1 -- 172.21.15.203:0/1099413644 >> 172.21.15.203:0/1099413644 conn(0x7f7f8c0648f0 msgr2=0x7f7f8c064cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:30.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f928f2700 1 -- 172.21.15.203:0/1099413644 shutdown_connections 2021-01-06T22:08:30.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.487+0000 7f7f928f2700 1 -- 172.21.15.203:0/1099413644 wait complete. 2021-01-06T22:08:30.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f928f2700 1 Processor -- start 2021-01-06T22:08:30.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f928f2700 1 -- start start 2021-01-06T22:08:30.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f928f2700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7f8c0a7bb0 0x7f7f8c062880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:30.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f928f2700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7f8c0a7f90 0x7f7f8c161a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:30.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f928f2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f8c0bc220 0x7f7f8c161f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:30.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f928f2700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c162430 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f928f2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c165040 con 0x7f7f8c0bc220 2021-01-06T22:08:30.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f928f2700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c1652a0 con 0x7f7f8c0a7f90 2021-01-06T22:08:30.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f920f1700 1 -- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f8c0bc220 msgr2=0x7f7f8c161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2021-01-06T22:08:30.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f920f1700 1 -- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f8c0bc220 msgr2=0x7f7f8c161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2021-01-06T22:08:30.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f920f1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f8c0bc220 0x7f7f8c161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner read peer banner failed r=-1 ((1) Operation not permitted) 2021-01-06T22:08:30.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f920f1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f8c0bc220 0x7f7f8c161f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2021-01-06T22:08:30.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f910ef700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7f8c0a7f90 0x7f7f8c161a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:30.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f918f0700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7f8c0a7bb0 0x7f7f8c062880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:30.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f910ef700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7f8c0a7f90 0x7f7f8c161a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:55530/0 (socket says 172.21.15.203:55530) 2021-01-06T22:08:30.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f910ef700 1 -- 172.21.15.203:0/357778568 learned_addr learned my addr 172.21.15.203:0/357778568 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:30.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f918f0700 1 -- 172.21.15.203:0/357778568 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7f8c0a7f90 msgr2=0x7f7f8c161a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:30.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f918f0700 1 --2- 172.21.15.203:0/357778568 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7f8c0a7f90 0x7f7f8c161a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:30.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f918f0700 1 -- 172.21.15.203:0/357778568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f8c0bc220 msgr2=0x7f7f8c161f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2021-01-06T22:08:30.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f918f0700 1 --2- 172.21.15.203:0/357778568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f8c0bc220 0x7f7f8c161f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-01-06T22:08:30.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f918f0700 1 -- 172.21.15.203:0/357778568 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c165540 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f918f0700 1 --2- 172.21.15.203:0/357778568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7f8c0a7bb0 0x7f7f8c062880 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f7f8800c860 tx=0x7f7f88001280).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:30.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f7dffb700 1 -- 172.21.15.203:0/357778568 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88007170 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f928f2700 1 -- 172.21.15.203:0/357778568 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8c1657e0 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f7dffb700 1 -- 172.21.15.203:0/357778568 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f880072d0 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f7dffb700 1 -- 172.21.15.203:0/357778568 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88004930 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f928f2700 1 -- 172.21.15.203:0/357778568 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c0c1a90 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.491+0000 7f7f928f2700 1 -- 172.21.15.203:0/357778568 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8c165e40 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.495+0000 7f7f7dffb700 1 -- 172.21.15.203:0/357778568 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f7f88010070 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.495+0000 7f7f7dffb700 1 --2- 172.21.15.203:0/357778568 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7f6c0563d0 0x7f7f6c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:30.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.495+0000 7f7f7dffb700 1 -- 172.21.15.203:0/357778568 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f7f8806f450 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.495+0000 7f7f910ef700 1 --2- 172.21.15.203:0/357778568 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7f6c0563d0 0x7f7f6c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:30.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.495+0000 7f7f910ef700 1 --2- 172.21.15.203:0/357778568 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7f6c0563d0 0x7f7f6c058890 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f7f8c0c3140 tx=0x7f7f80006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:30.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.503+0000 7f7f7dffb700 1 -- 172.21.15.203:0/357778568 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f7f88044c20 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.711+0000 7f7f928f2700 1 -- 172.21.15.203:0/357778568 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f8c165c80 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.711+0000 7f7f7dffb700 1 -- 172.21.15.203:0/357778568 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3580 (secure 0 0 0) 0x7f7f8c165c80 con 0x7f7f8c0a7bb0 2021-01-06T22:08:30.706 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:30.706 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:10.461108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:stopping","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[0],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:30.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.711+0000 7f7f928f2700 1 -- 172.21.15.203:0/357778568 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7f6c0563d0 msgr2=0x7f7f6c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:30.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.711+0000 7f7f928f2700 1 --2- 172.21.15.203:0/357778568 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7f6c0563d0 0x7f7f6c058890 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f7f8c0c3140 tx=0x7f7f80006de0).stop 2021-01-06T22:08:30.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.711+0000 7f7f928f2700 1 -- 172.21.15.203:0/357778568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7f8c0a7bb0 msgr2=0x7f7f8c062880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:30.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.711+0000 7f7f928f2700 1 --2- 172.21.15.203:0/357778568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7f8c0a7bb0 0x7f7f8c062880 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f7f8800c860 tx=0x7f7f88001280).stop 2021-01-06T22:08:30.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.711+0000 7f7f928f2700 1 -- 172.21.15.203:0/357778568 shutdown_connections 2021-01-06T22:08:30.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.711+0000 7f7f928f2700 1 --2- 172.21.15.203:0/357778568 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7f6c0563d0 0x7f7f6c058890 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:30.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.711+0000 7f7f928f2700 1 --2- 172.21.15.203:0/357778568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f8c0bc220 0x7f7f8c161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-01-06T22:08:30.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.711+0000 7f7f928f2700 1 --2- 172.21.15.203:0/357778568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7f8c0a7bb0 0x7f7f8c062880 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:30.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.711+0000 7f7f928f2700 1 --2- 172.21.15.203:0/357778568 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7f8c0a7f90 0x7f7f8c161a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:30.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.711+0000 7f7f928f2700 1 -- 172.21.15.203:0/357778568 >> 172.21.15.203:0/357778568 conn(0x7f7f8c0648f0 msgr2=0x7f7f8c150e00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:30.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.711+0000 7f7f928f2700 1 -- 172.21.15.203:0/357778568 shutdown_connections 2021-01-06T22:08:30.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:30.711+0000 7f7f928f2700 1 -- 172.21.15.203:0/357778568 wait complete. 2021-01-06T22:08:30.710 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 30 2021-01-06T22:08:30.720 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:08:32.721 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:32.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.863+0000 7ffaa4a97700 1 Processor -- start 2021-01-06T22:08:32.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.863+0000 7ffaa4a97700 1 -- start start 2021-01-06T22:08:32.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffaa4a97700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ffaa00c04c0 0x7ffaa0066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:32.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffaa4a97700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ffaa014ab60 0x7ffaa014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:32.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffaa4a97700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa00bb860 con 0x7ffaa00c08a0 2021-01-06T22:08:32.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffaa4a97700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa014b940 con 0x7ffaa014ab60 2021-01-06T22:08:32.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffaa4a97700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa00bba90 con 0x7ffaa00c04c0 2021-01-06T22:08:32.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffa95d9b700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7ffaa00c08a0 0x7ffaa0147cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49052/0 (socket says 172.21.15.203:49052) 2021-01-06T22:08:32.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffa95d9b700 1 -- 172.21.15.203:0/1937083039 learned_addr learned my addr 172.21.15.203:0/1937083039 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:32.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffa9e59c700 1 --2- 172.21.15.203:0/1937083039 >> v2:172.21.15.203:3300/0 conn(0x7ffaa00c04c0 0x7ffaa0066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:32.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffa9ed9d700 1 --2- 172.21.15.203:0/1937083039 >> v2:172.21.15.91:3300/0 conn(0x7ffaa014ab60 0x7ffaa014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:32.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffa9dd9b700 1 -- 172.21.15.203:0/1937083039 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678953308 0 0) 0x7ffaa00bb860 con 0x7ffaa00c08a0 2021-01-06T22:08:32.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffa9dd9b700 1 -- 172.21.15.203:0/1937083039 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa80003230 con 0x7ffaa00c08a0 2021-01-06T22:08:32.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffa9e59c700 1 -- 172.21.15.203:0/1937083039 >> v2:172.21.15.91:3300/0 conn(0x7ffaa014ab60 msgr2=0x7ffaa014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:32.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffa9e59c700 1 --2- 172.21.15.203:0/1937083039 >> v2:172.21.15.91:3300/0 conn(0x7ffaa014ab60 0x7ffaa014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:32.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffa9dd9b700 1 -- 172.21.15.203:0/1937083039 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2027751572 0 0) 0x7ffa80003230 con 0x7ffaa00c08a0 2021-01-06T22:08:32.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffa9e59c700 1 -- 172.21.15.203:0/1937083039 >> v1:172.21.15.203:6789/0 conn(0x7ffaa00c08a0 legacy=0x7ffaa0147cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:32.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffa9e59c700 1 -- 172.21.15.203:0/1937083039 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa00bbd30 con 0x7ffaa00c04c0 2021-01-06T22:08:32.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffa9e59c700 1 --2- 172.21.15.203:0/1937083039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffaa00c04c0 0x7ffaa0066cd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7ffaa0061730 tx=0x7ffa90006de0).ready entity=mon.1 client_cookie=8b45bf1872f520fc server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:32.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffa9dd9b700 1 -- 172.21.15.203:0/1937083039 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa880014e0 con 0x7ffaa00c04c0 2021-01-06T22:08:32.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffa9dd9b700 1 -- 172.21.15.203:0/1937083039 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa90011400 con 0x7ffaa00c04c0 2021-01-06T22:08:32.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffa9dd9b700 1 -- 172.21.15.203:0/1937083039 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa90011780 con 0x7ffaa00c04c0 2021-01-06T22:08:32.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffaa4a97700 1 -- 172.21.15.203:0/1937083039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffaa00c04c0 msgr2=0x7ffaa0066cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:32.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffaa4a97700 1 --2- 172.21.15.203:0/1937083039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffaa00c04c0 0x7ffaa0066cd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7ffaa0061730 tx=0x7ffa90006de0).stop 2021-01-06T22:08:32.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffaa4a97700 1 -- 172.21.15.203:0/1937083039 shutdown_connections 2021-01-06T22:08:32.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffaa4a97700 1 --2- 172.21.15.203:0/1937083039 >> v2:172.21.15.91:3300/0 conn(0x7ffaa014ab60 0x7ffaa014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:32.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffaa4a97700 1 --2- 172.21.15.203:0/1937083039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffaa00c04c0 0x7ffaa0066cd0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:32.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffaa4a97700 1 -- 172.21.15.203:0/1937083039 >> 172.21.15.203:0/1937083039 conn(0x7ffaa0056cd0 msgr2=0x7ffaa00626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:32.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffaa4a97700 1 -- 172.21.15.203:0/1937083039 shutdown_connections 2021-01-06T22:08:32.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.867+0000 7ffaa4a97700 1 -- 172.21.15.203:0/1937083039 wait complete. 2021-01-06T22:08:32.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffaa4a97700 1 Processor -- start 2021-01-06T22:08:32.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffaa4a97700 1 -- start start 2021-01-06T22:08:32.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffaa4a97700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffaa00c04c0 0x7ffaa015e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:32.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffaa4a97700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffaa00c08a0 0x7ffaa0161620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:32.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffaa4a97700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffaa014ab60 0x7ffaa0149880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:32.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffaa4a97700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa015d680 con 0x7ffaa00c04c0 2021-01-06T22:08:32.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffaa4a97700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa0149340 con 0x7ffaa014ab60 2021-01-06T22:08:32.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffaa4a97700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa01495e0 con 0x7ffaa00c08a0 2021-01-06T22:08:32.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffa9ed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffaa014ab60 0x7ffaa0149880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:32.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffa9ed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffaa014ab60 0x7ffaa0149880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50466/0 (socket says 172.21.15.203:50466) 2021-01-06T22:08:32.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffa9ed9d700 1 -- 172.21.15.203:0/3297059893 learned_addr learned my addr 172.21.15.203:0/3297059893 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:32.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffa9e59c700 1 --2- 172.21.15.203:0/3297059893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffaa00c04c0 0x7ffaa015e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:32.878 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffa95d9b700 1 --2- 172.21.15.203:0/3297059893 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffaa00c08a0 0x7ffaa0161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:32.878 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffa9ed9d700 1 -- 172.21.15.203:0/3297059893 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffaa00c08a0 msgr2=0x7ffaa0161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:32.878 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffa9ed9d700 1 --2- 172.21.15.203:0/3297059893 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffaa00c08a0 0x7ffaa0161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:32.879 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffa9ed9d700 1 -- 172.21.15.203:0/3297059893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffaa00c04c0 msgr2=0x7ffaa015e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:32.879 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffa9ed9d700 1 --2- 172.21.15.203:0/3297059893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffaa00c04c0 0x7ffaa015e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:32.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffa9ed9d700 1 -- 172.21.15.203:0/3297059893 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa015d870 con 0x7ffaa014ab60 2021-01-06T22:08:32.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffa9ed9d700 1 --2- 172.21.15.203:0/3297059893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffaa014ab60 0x7ffaa0149880 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7ffa9800ed20 tx=0x7ffa980091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:32.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffa96ffd700 1 -- 172.21.15.203:0/3297059893 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98009580 con 0x7ffaa014ab60 2021-01-06T22:08:32.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffa96ffd700 1 -- 172.21.15.203:0/3297059893 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9800cc40 con 0x7ffaa014ab60 2021-01-06T22:08:32.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffa96ffd700 1 -- 172.21.15.203:0/3297059893 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa980123f0 con 0x7ffaa014ab60 2021-01-06T22:08:32.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffaa4a97700 1 -- 172.21.15.203:0/3297059893 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa015db10 con 0x7ffaa014ab60 2021-01-06T22:08:32.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.871+0000 7ffaa4a97700 1 -- 172.21.15.203:0/3297059893 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa0161b10 con 0x7ffaa014ab60 2021-01-06T22:08:32.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.875+0000 7ffaa4a97700 1 -- 172.21.15.203:0/3297059893 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa005cfe0 con 0x7ffaa014ab60 2021-01-06T22:08:32.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.875+0000 7ffa96ffd700 1 -- 172.21.15.203:0/3297059893 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ffa98022070 con 0x7ffaa014ab60 2021-01-06T22:08:32.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.875+0000 7ffa96ffd700 1 --2- 172.21.15.203:0/3297059893 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ffa840563d0 0x7ffa84058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:32.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.875+0000 7ffa9e59c700 1 --2- 172.21.15.203:0/3297059893 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ffa840563d0 0x7ffa84058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:32.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.875+0000 7ffa96ffd700 1 -- 172.21.15.203:0/3297059893 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7ffa9806feb0 con 0x7ffaa014ab60 2021-01-06T22:08:32.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.875+0000 7ffa9e59c700 1 --2- 172.21.15.203:0/3297059893 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ffa840563d0 0x7ffa84058890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7ffaa0150bd0 tx=0x7ffa90016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:32.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:32.883+0000 7ffa96ffd700 1 -- 172.21.15.203:0/3297059893 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ffa98045570 con 0x7ffaa014ab60 2021-01-06T22:08:33.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:33.087+0000 7ffaa4a97700 1 -- 172.21.15.203:0/3297059893 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaa00576d0 con 0x7ffaa014ab60 2021-01-06T22:08:33.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:33.087+0000 7ffa96ffd700 1 -- 172.21.15.203:0/3297059893 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3580 (secure 0 0 0) 0x7ffa98016030 con 0x7ffaa014ab60 2021-01-06T22:08:33.083 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:33.083 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:10.461108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4274,"mds_1":4273,"mds_2":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4867":{"gid":4867,"name":"c","rank":2,"incarnation":20,"state":"up:stopping","state_seq":22,"addr":"172.21.15.203:6835/3382231276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3382231276},{"type":"v1","addr":"172.21.15.203:6835","nonce":3382231276}]},"join_fscid":-1,"export_targets":[0],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:33.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:33.091+0000 7ffaa4a97700 1 -- 172.21.15.203:0/3297059893 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ffa840563d0 msgr2=0x7ffa84058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:33.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:33.091+0000 7ffaa4a97700 1 --2- 172.21.15.203:0/3297059893 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ffa840563d0 0x7ffa84058890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7ffaa0150bd0 tx=0x7ffa90016000).stop 2021-01-06T22:08:33.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:33.091+0000 7ffaa4a97700 1 -- 172.21.15.203:0/3297059893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffaa014ab60 msgr2=0x7ffaa0149880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:33.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:33.091+0000 7ffaa4a97700 1 --2- 172.21.15.203:0/3297059893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffaa014ab60 0x7ffaa0149880 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7ffa9800ed20 tx=0x7ffa980091d0).stop 2021-01-06T22:08:33.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:33.091+0000 7ffaa4a97700 1 -- 172.21.15.203:0/3297059893 shutdown_connections 2021-01-06T22:08:33.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:33.091+0000 7ffaa4a97700 1 --2- 172.21.15.203:0/3297059893 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ffa840563d0 0x7ffa84058890 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:33.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:33.091+0000 7ffaa4a97700 1 --2- 172.21.15.203:0/3297059893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffaa014ab60 0x7ffaa0149880 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:33.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:33.091+0000 7ffaa4a97700 1 --2- 172.21.15.203:0/3297059893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffaa00c04c0 0x7ffaa015e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:33.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:33.091+0000 7ffaa4a97700 1 --2- 172.21.15.203:0/3297059893 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffaa00c08a0 0x7ffaa0161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:33.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:33.091+0000 7ffaa4a97700 1 -- 172.21.15.203:0/3297059893 >> 172.21.15.203:0/3297059893 conn(0x7ffaa0056cd0 msgr2=0x7ffaa014afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:33.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:33.091+0000 7ffaa4a97700 1 -- 172.21.15.203:0/3297059893 shutdown_connections 2021-01-06T22:08:33.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:33.091+0000 7ffaa4a97700 1 -- 172.21.15.203:0/3297059893 wait complete. 2021-01-06T22:08:33.088 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 30 2021-01-06T22:08:33.097 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:08:35.099 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:35.212 INFO:tasks.ceph.mds.c.smithi203.stdout:starting mds.c at 2021-01-06T22:08:35.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44391d4700 1 Processor -- start 2021-01-06T22:08:35.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44391d4700 1 -- start start 2021-01-06T22:08:35.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44391d4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f44340c44f0 0x7f44340c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:35.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44391d4700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44340c0f50 con 0x7f44340c48d0 2021-01-06T22:08:35.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44391d4700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4434066cf0 con 0x7f4434060fc0 2021-01-06T22:08:35.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44391d4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4434060750 con 0x7f44340c44f0 2021-01-06T22:08:35.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f4433fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f44340c44f0 0x7f44340c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:35.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44389d3700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f4434060fc0 0x7f4434158700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49080/0 (socket says 172.21.15.203:49080) 2021-01-06T22:08:35.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44389d3700 1 -- 172.21.15.203:0/1304013033 learned_addr learned my addr 172.21.15.203:0/1304013033 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:35.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44337fe700 1 -- 172.21.15.203:0/1304013033 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 412629514 0 0) 0x7f4434066cf0 con 0x7f4434060fc0 2021-01-06T22:08:35.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44337fe700 1 -- 172.21.15.203:0/1304013033 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4414003230 con 0x7f4434060fc0 2021-01-06T22:08:35.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44337fe700 1 -- 172.21.15.203:0/1304013033 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757907217 0 0) 0x7f44340c0f50 con 0x7f44340c48d0 2021-01-06T22:08:35.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f4433fff700 1 -- 172.21.15.203:0/1304013033 >> v1:172.21.15.203:6789/0 conn(0x7f4434060fc0 legacy=0x7f4434158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:35.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f4433fff700 1 -- 172.21.15.203:0/1304013033 >> v1:172.21.15.91:6789/0 conn(0x7f44340c48d0 legacy=0x7f4434150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:35.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f4433fff700 1 -- 172.21.15.203:0/1304013033 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44340609f0 con 0x7f44340c44f0 2021-01-06T22:08:35.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f4433fff700 1 --2- 172.21.15.203:0/1304013033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44340c44f0 0x7f44340c1380 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f442400b980 tx=0x7f4424006de0).ready entity=mon.1 client_cookie=ffe474565f95bd1f server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:35.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44337fe700 1 -- 172.21.15.203:0/1304013033 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44240097b0 con 0x7f44340c44f0 2021-01-06T22:08:35.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44337fe700 1 -- 172.21.15.203:0/1304013033 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4424009910 con 0x7f44340c44f0 2021-01-06T22:08:35.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44337fe700 1 -- 172.21.15.203:0/1304013033 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424009c90 con 0x7f44340c44f0 2021-01-06T22:08:35.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44391d4700 1 -- 172.21.15.203:0/1304013033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44340c44f0 msgr2=0x7f44340c1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:35.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44391d4700 1 --2- 172.21.15.203:0/1304013033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44340c44f0 0x7f44340c1380 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f442400b980 tx=0x7f4424006de0).stop 2021-01-06T22:08:35.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44391d4700 1 -- 172.21.15.203:0/1304013033 shutdown_connections 2021-01-06T22:08:35.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44391d4700 1 --2- 172.21.15.203:0/1304013033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44340c44f0 0x7f44340c1380 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:35.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.255+0000 7f44391d4700 1 -- 172.21.15.203:0/1304013033 >> 172.21.15.203:0/1304013033 conn(0x7f4434056cd0 msgr2=0x7f4434061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:35.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f44391d4700 1 -- 172.21.15.203:0/1304013033 shutdown_connections 2021-01-06T22:08:35.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f44391d4700 1 -- 172.21.15.203:0/1304013033 wait complete. 2021-01-06T22:08:35.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f44391d4700 1 Processor -- start 2021-01-06T22:08:35.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f44391d4700 1 -- start start 2021-01-06T22:08:35.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f44391d4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4434060fc0 0x7f4434160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:35.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f44391d4700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44340c44f0 0x7f44340ae310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:35.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f44391d4700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44340c48d0 0x7f44340bb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:35.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f44391d4700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4434160930 con 0x7f44340c44f0 2021-01-06T22:08:35.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f44391d4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4434160ca0 con 0x7f4434060fc0 2021-01-06T22:08:35.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f44391d4700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4434160f40 con 0x7f44340c48d0 2021-01-06T22:08:35.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f4433fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4434060fc0 0x7f4434160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:35.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f4433fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4434060fc0 0x7f4434160440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50488/0 (socket says 172.21.15.203:50488) 2021-01-06T22:08:35.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f4433fff700 1 -- 172.21.15.203:0/3082233575 learned_addr learned my addr 172.21.15.203:0/3082233575 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:35.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f442bfff700 1 --2- 172.21.15.203:0/3082233575 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44340c44f0 0x7f44340ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:35.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f44389d3700 1 --2- 172.21.15.203:0/3082233575 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44340c48d0 0x7f44340bb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:35.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f4433fff700 1 -- 172.21.15.203:0/3082233575 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44340c48d0 msgr2=0x7f44340bb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:35.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f4433fff700 1 --2- 172.21.15.203:0/3082233575 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44340c48d0 0x7f44340bb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:35.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f4433fff700 1 -- 172.21.15.203:0/3082233575 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44340c44f0 msgr2=0x7f44340ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:35.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f4433fff700 1 --2- 172.21.15.203:0/3082233575 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44340c44f0 0x7f44340ae310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:35.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f4433fff700 1 -- 172.21.15.203:0/3082233575 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44341611e0 con 0x7f4434060fc0 2021-01-06T22:08:35.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f44389d3700 1 --2- 172.21.15.203:0/3082233575 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44340c48d0 0x7f44340bb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:08:35.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f4433fff700 1 --2- 172.21.15.203:0/3082233575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4434060fc0 0x7f4434160440 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f44340c1f40 tx=0x7f4424016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:35.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f4430ff9700 1 -- 172.21.15.203:0/3082233575 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424011df0 con 0x7f4434060fc0 2021-01-06T22:08:35.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f4430ff9700 1 -- 172.21.15.203:0/3082233575 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442401d430 con 0x7f4434060fc0 2021-01-06T22:08:35.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f44391d4700 1 -- 172.21.15.203:0/3082233575 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4434161480 con 0x7f4434060fc0 2021-01-06T22:08:35.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f4430ff9700 1 -- 172.21.15.203:0/3082233575 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442401d7b0 con 0x7f4434060fc0 2021-01-06T22:08:35.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.259+0000 7f44391d4700 1 -- 172.21.15.203:0/3082233575 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44340a9970 con 0x7f4434060fc0 2021-01-06T22:08:35.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.263+0000 7f44391d4700 1 -- 172.21.15.203:0/3082233575 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44340450a0 con 0x7f4434060fc0 2021-01-06T22:08:35.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.263+0000 7f4430ff9700 1 -- 172.21.15.203:0/3082233575 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f442400f070 con 0x7f4434060fc0 2021-01-06T22:08:35.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.263+0000 7f4430ff9700 1 --2- 172.21.15.203:0/3082233575 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f44180563d0 0x7f4418058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:35.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.263+0000 7f4430ff9700 1 -- 172.21.15.203:0/3082233575 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f4424070bb0 con 0x7f4434060fc0 2021-01-06T22:08:35.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.263+0000 7f442bfff700 1 --2- 172.21.15.203:0/3082233575 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f44180563d0 0x7f4418058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:35.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.263+0000 7f442bfff700 1 --2- 172.21.15.203:0/3082233575 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f44180563d0 0x7f4418058890 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f44340a9db0 tx=0x7f441c003040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:35.267 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.271+0000 7f4430ff9700 1 -- 172.21.15.203:0/3082233575 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f4424077080 con 0x7f4434060fc0 2021-01-06T22:08:35.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.491+0000 7f44391d4700 1 -- 172.21.15.203:0/3082233575 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4434161920 con 0x7f4434060fc0 2021-01-06T22:08:35.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.491+0000 7f4430ff9700 1 -- 172.21.15.203:0/3082233575 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+3207 (secure 0 0 0) 0x7f4424070020 con 0x7f4434060fc0 2021-01-06T22:08:35.488 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:35.488 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:35.158095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":4273},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:35.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.495+0000 7f44391d4700 1 -- 172.21.15.203:0/3082233575 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f44180563d0 msgr2=0x7f4418058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:35.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.495+0000 7f44391d4700 1 --2- 172.21.15.203:0/3082233575 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f44180563d0 0x7f4418058890 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f44340a9db0 tx=0x7f441c003040).stop 2021-01-06T22:08:35.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.495+0000 7f44391d4700 1 -- 172.21.15.203:0/3082233575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4434060fc0 msgr2=0x7f4434160440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:35.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.495+0000 7f44391d4700 1 --2- 172.21.15.203:0/3082233575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4434060fc0 0x7f4434160440 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f44340c1f40 tx=0x7f4424016000).stop 2021-01-06T22:08:35.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.495+0000 7f44391d4700 1 -- 172.21.15.203:0/3082233575 shutdown_connections 2021-01-06T22:08:35.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.495+0000 7f44391d4700 1 --2- 172.21.15.203:0/3082233575 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f44180563d0 0x7f4418058890 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:35.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.495+0000 7f44391d4700 1 --2- 172.21.15.203:0/3082233575 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44340c48d0 0x7f44340bb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:35.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.495+0000 7f44391d4700 1 --2- 172.21.15.203:0/3082233575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4434060fc0 0x7f4434160440 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:35.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.495+0000 7f44391d4700 1 --2- 172.21.15.203:0/3082233575 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44340c44f0 0x7f44340ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:35.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.495+0000 7f44391d4700 1 -- 172.21.15.203:0/3082233575 >> 172.21.15.203:0/3082233575 conn(0x7f4434056cd0 msgr2=0x7f44340bd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:35.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.495+0000 7f44391d4700 1 -- 172.21.15.203:0/3082233575 shutdown_connections 2021-01-06T22:08:35.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.495+0000 7f44391d4700 1 -- 172.21.15.203:0/3082233575 wait complete. 2021-01-06T22:08:35.494 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 31 2021-01-06T22:08:35.508 INFO:tasks.mds_thrash.fs.[cephfs]:mds cluster has 2 alive and active, now stable! 2021-01-06T22:08:35.508 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=1) 2021-01-06T22:08:35.509 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-01-06T22:08:35.509 INFO:teuthology.orchestra.run:waiting for 300 2021-01-06T22:08:35.518 INFO:tasks.ceph.mds.a:Stopped 2021-01-06T22:08:35.518 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:35.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.699+0000 7f7e5b59e700 1 Processor -- start 2021-01-06T22:08:35.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.699+0000 7f7e5b59e700 1 -- start start 2021-01-06T22:08:35.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5b59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0be350 0x7f7e5c0657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:35.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5b59e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7e5c14ba00 0x7f7e5c14be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:35.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5b59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5c14c370 con 0x7f7e5c0be730 2021-01-06T22:08:35.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5b59e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e5c14ee80 con 0x7f7e5c14ba00 2021-01-06T22:08:35.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5b59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c14f120 con 0x7f7e5c0be350 2021-01-06T22:08:35.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5ad9d700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7e5c14ba00 0x7f7e5c14be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:35.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5a59c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0be350 0x7f7e5c0657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:35.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e53fff700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f7e5c0be730 0x7f7e5c0ba350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:43748/0 (socket says 172.21.15.203:43748) 2021-01-06T22:08:35.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e53fff700 1 -- 172.21.15.203:0/3661866295 learned_addr learned my addr 172.21.15.203:0/3661866295 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:35.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5a59c700 1 -- 172.21.15.203:0/3661866295 >> v2:172.21.15.91:3300/0 conn(0x7f7e5c14ba00 msgr2=0x7f7e5c14be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:35.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5a59c700 1 --2- 172.21.15.203:0/3661866295 >> v2:172.21.15.91:3300/0 conn(0x7f7e5c14ba00 0x7f7e5c14be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:35.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5a59c700 1 -- 172.21.15.203:0/3661866295 >> v1:172.21.15.91:6789/0 conn(0x7f7e5c0be730 legacy=0x7f7e5c0ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:35.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5a59c700 1 -- 172.21.15.203:0/3661866295 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c14f3c0 con 0x7f7e5c0be350 2021-01-06T22:08:35.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5a59c700 1 --2- 172.21.15.203:0/3661866295 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0be350 0x7f7e5c0657c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f7e4c00b980 tx=0x7f7e4c006de0).ready entity=mon.2 client_cookie=7ef9e5ac14c573db server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:35.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e59d9b700 1 -- 172.21.15.203:0/3661866295 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0097b0 con 0x7f7e5c0be350 2021-01-06T22:08:35.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e59d9b700 1 -- 172.21.15.203:0/3661866295 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4c009910 con 0x7f7e5c0be350 2021-01-06T22:08:35.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e59d9b700 1 -- 172.21.15.203:0/3661866295 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c009c90 con 0x7f7e5c0be350 2021-01-06T22:08:35.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5b59e700 1 -- 172.21.15.203:0/3661866295 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0be350 msgr2=0x7f7e5c0657c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:35.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5b59e700 1 --2- 172.21.15.203:0/3661866295 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0be350 0x7f7e5c0657c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f7e4c00b980 tx=0x7f7e4c006de0).stop 2021-01-06T22:08:35.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5b59e700 1 -- 172.21.15.203:0/3661866295 shutdown_connections 2021-01-06T22:08:35.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5b59e700 1 --2- 172.21.15.203:0/3661866295 >> v2:172.21.15.91:3300/0 conn(0x7f7e5c14ba00 0x7f7e5c14be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:35.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5b59e700 1 --2- 172.21.15.203:0/3661866295 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0be350 0x7f7e5c0657c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:35.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5b59e700 1 -- 172.21.15.203:0/3661866295 >> 172.21.15.203:0/3661866295 conn(0x7f7e5c067e20 msgr2=0x7f7e5c068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:35.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5b59e700 1 -- 172.21.15.203:0/3661866295 shutdown_connections 2021-01-06T22:08:35.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.703+0000 7f7e5b59e700 1 -- 172.21.15.203:0/3661866295 wait complete. 2021-01-06T22:08:35.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5b59e700 1 Processor -- start 2021-01-06T22:08:35.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5b59e700 1 -- start start 2021-01-06T22:08:35.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5b59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e5c0be350 0x7f7e5c13df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:35.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5b59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0be730 0x7f7e5c142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:35.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5b59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c14ba00 0x7f7e5c15da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:35.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5b59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c14dfe0 con 0x7f7e5c0be350 2021-01-06T22:08:35.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5b59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c14ec90 con 0x7f7e5c14ba00 2021-01-06T22:08:35.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5b59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c148a00 con 0x7f7e5c0be730 2021-01-06T22:08:35.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5a59c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e5c0be350 0x7f7e5c13df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:35.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5ad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c14ba00 0x7f7e5c15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:35.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5a59c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e5c0be350 0x7f7e5c13df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:58352/0 (socket says 172.21.15.203:58352) 2021-01-06T22:08:35.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5a59c700 1 -- 172.21.15.203:0/3905011356 learned_addr learned my addr 172.21.15.203:0/3905011356 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:35.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e53fff700 1 --2- 172.21.15.203:0/3905011356 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0be730 0x7f7e5c142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:35.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5ad9d700 1 -- 172.21.15.203:0/3905011356 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0be730 msgr2=0x7f7e5c142c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:35.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5ad9d700 1 --2- 172.21.15.203:0/3905011356 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0be730 0x7f7e5c142c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:35.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5ad9d700 1 -- 172.21.15.203:0/3905011356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e5c0be350 msgr2=0x7f7e5c13df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:35.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5ad9d700 1 --2- 172.21.15.203:0/3905011356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e5c0be350 0x7f7e5c13df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:35.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5ad9d700 1 -- 172.21.15.203:0/3905011356 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c14e210 con 0x7f7e5c14ba00 2021-01-06T22:08:35.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5ad9d700 1 --2- 172.21.15.203:0/3905011356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c14ba00 0x7f7e5c15da20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f7e5400eb80 tx=0x7f7e540091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:35.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e527fc700 1 -- 172.21.15.203:0/3905011356 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e54009480 con 0x7f7e5c14ba00 2021-01-06T22:08:35.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e527fc700 1 -- 172.21.15.203:0/3905011356 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5400c980 con 0x7f7e5c14ba00 2021-01-06T22:08:35.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5b59e700 1 -- 172.21.15.203:0/3905011356 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5c14e4b0 con 0x7f7e5c14ba00 2021-01-06T22:08:35.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e527fc700 1 -- 172.21.15.203:0/3905011356 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5400cd40 con 0x7f7e5c14ba00 2021-01-06T22:08:35.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.707+0000 7f7e5b59e700 1 -- 172.21.15.203:0/3905011356 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5c14e9f0 con 0x7f7e5c14ba00 2021-01-06T22:08:35.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.711+0000 7f7e5b59e700 1 -- 172.21.15.203:0/3905011356 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5c14f5a0 con 0x7f7e5c14ba00 2021-01-06T22:08:35.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.711+0000 7f7e527fc700 1 -- 172.21.15.203:0/3905011356 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f7e54022070 con 0x7f7e5c14ba00 2021-01-06T22:08:35.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.711+0000 7f7e527fc700 1 --2- 172.21.15.203:0/3905011356 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e400563d0 0x7f7e40058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:35.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.711+0000 7f7e527fc700 1 -- 172.21.15.203:0/3905011356 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f7e5406fbe0 con 0x7f7e5c14ba00 2021-01-06T22:08:35.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.711+0000 7f7e5a59c700 1 --2- 172.21.15.203:0/3905011356 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e400563d0 0x7f7e40058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:35.713 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.711+0000 7f7e5a59c700 1 --2- 172.21.15.203:0/3905011356 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e400563d0 0x7f7e40058890 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f7e4c006a20 tx=0x7f7e4c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:35.713 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.719+0000 7f7e527fc700 1 -- 172.21.15.203:0/3905011356 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f7e540453b0 con 0x7f7e5c14ba00 2021-01-06T22:08:35.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.923+0000 7f7e5b59e700 1 -- 172.21.15.203:0/3905011356 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e5c0450a0 con 0x7f7e5c14ba00 2021-01-06T22:08:35.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.923+0000 7f7e527fc700 1 -- 172.21.15.203:0/3905011356 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+3207 (secure 0 0 0) 0x7f7e54016070 con 0x7f7e5c14ba00 2021-01-06T22:08:35.917 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:35.917 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:35.158095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":4273},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:35.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.923+0000 7f7e5b59e700 1 -- 172.21.15.203:0/3905011356 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e400563d0 msgr2=0x7f7e40058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:35.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.923+0000 7f7e5b59e700 1 --2- 172.21.15.203:0/3905011356 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e400563d0 0x7f7e40058890 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f7e4c006a20 tx=0x7f7e4c016000).stop 2021-01-06T22:08:35.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.923+0000 7f7e5b59e700 1 -- 172.21.15.203:0/3905011356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c14ba00 msgr2=0x7f7e5c15da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:35.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.923+0000 7f7e5b59e700 1 --2- 172.21.15.203:0/3905011356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c14ba00 0x7f7e5c15da20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f7e5400eb80 tx=0x7f7e540091d0).stop 2021-01-06T22:08:35.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.923+0000 7f7e5b59e700 1 -- 172.21.15.203:0/3905011356 shutdown_connections 2021-01-06T22:08:35.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.923+0000 7f7e5b59e700 1 --2- 172.21.15.203:0/3905011356 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e400563d0 0x7f7e40058890 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:35.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.923+0000 7f7e5b59e700 1 --2- 172.21.15.203:0/3905011356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c14ba00 0x7f7e5c15da20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:35.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.923+0000 7f7e5b59e700 1 --2- 172.21.15.203:0/3905011356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e5c0be350 0x7f7e5c13df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:35.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.923+0000 7f7e5b59e700 1 --2- 172.21.15.203:0/3905011356 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0be730 0x7f7e5c142c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:35.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.923+0000 7f7e5b59e700 1 -- 172.21.15.203:0/3905011356 >> 172.21.15.203:0/3905011356 conn(0x7f7e5c067e20 msgr2=0x7f7e5c065110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:35.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.923+0000 7f7e5b59e700 1 -- 172.21.15.203:0/3905011356 shutdown_connections 2021-01-06T22:08:35.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:35.923+0000 7f7e5b59e700 1 -- 172.21.15.203:0/3905011356 wait complete. 2021-01-06T22:08:35.922 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 31 2021-01-06T22:08:35.931 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:08:37.932 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:38.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.087+0000 7fed811a0700 1 Processor -- start 2021-01-06T22:08:38.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.087+0000 7fed811a0700 1 -- start start 2021-01-06T22:08:38.083 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.087+0000 7fed811a0700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fed7c0c1430 0x7fed7c148140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:38.084 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.087+0000 7fed811a0700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7c0bb7e0 con 0x7fed7c0bb230 2021-01-06T22:08:38.084 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.087+0000 7fed811a0700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7c0bba80 con 0x7fed7c147d60 2021-01-06T22:08:38.084 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.087+0000 7fed811a0700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7c0bbd20 con 0x7fed7c0c1430 2021-01-06T22:08:38.084 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.087+0000 7fed7b7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fed7c0c1430 0x7fed7c148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:38.084 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.087+0000 7fed7bfff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fed7c147d60 0x7fed7c0671a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49112/0 (socket says 172.21.15.203:49112) 2021-01-06T22:08:38.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.087+0000 7fed7bfff700 1 -- 172.21.15.203:0/329479458 learned_addr learned my addr 172.21.15.203:0/329479458 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:38.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed7b7fe700 1 -- 172.21.15.203:0/329479458 >> v1:172.21.15.203:6789/0 conn(0x7fed7c147d60 legacy=0x7fed7c0671a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:38.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed7b7fe700 1 -- 172.21.15.203:0/329479458 >> v1:172.21.15.91:6789/0 conn(0x7fed7c0bb230 legacy=0x7fed7c0602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:38.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed7affd700 1 -- 172.21.15.203:0/329479458 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999354142 0 0) 0x7fed7c0bba80 con 0x7fed7c147d60 2021-01-06T22:08:38.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed7b7fe700 1 -- 172.21.15.203:0/329479458 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c0bbfc0 con 0x7fed7c0c1430 2021-01-06T22:08:38.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed7b7fe700 1 --2- 172.21.15.203:0/329479458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed7c0c1430 0x7fed7c148140 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fed7000b980 tx=0x7fed70006de0).ready entity=mon.1 client_cookie=55eb13e42122caa2 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:38.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed7affd700 1 -- 172.21.15.203:0/329479458 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed700097b0 con 0x7fed7c0c1430 2021-01-06T22:08:38.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed7affd700 1 -- 172.21.15.203:0/329479458 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed70009910 con 0x7fed7c0c1430 2021-01-06T22:08:38.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed7affd700 1 -- 172.21.15.203:0/329479458 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70009c90 con 0x7fed7c0c1430 2021-01-06T22:08:38.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed811a0700 1 -- 172.21.15.203:0/329479458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed7c0c1430 msgr2=0x7fed7c148140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:38.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed811a0700 1 --2- 172.21.15.203:0/329479458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed7c0c1430 0x7fed7c148140 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fed7000b980 tx=0x7fed70006de0).stop 2021-01-06T22:08:38.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed811a0700 1 -- 172.21.15.203:0/329479458 shutdown_connections 2021-01-06T22:08:38.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed811a0700 1 --2- 172.21.15.203:0/329479458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed7c0c1430 0x7fed7c148140 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:38.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed811a0700 1 -- 172.21.15.203:0/329479458 >> 172.21.15.203:0/329479458 conn(0x7fed7c0667b0 msgr2=0x7fed7c0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:38.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed811a0700 1 -- 172.21.15.203:0/329479458 shutdown_connections 2021-01-06T22:08:38.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed811a0700 1 -- 172.21.15.203:0/329479458 wait complete. 2021-01-06T22:08:38.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed811a0700 1 Processor -- start 2021-01-06T22:08:38.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.091+0000 7fed811a0700 1 -- start start 2021-01-06T22:08:38.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed811a0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed7c0bb230 0x7fed7c067340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:38.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed811a0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed7c0c1430 0x7fed7c15b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:38.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed811a0700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed7c147d60 0x7fed7c15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:38.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed811a0700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c15bf20 con 0x7fed7c147d60 2021-01-06T22:08:38.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed811a0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c15f2a0 con 0x7fed7c0c1430 2021-01-06T22:08:38.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed811a0700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c15f540 con 0x7fed7c0bb230 2021-01-06T22:08:38.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed7b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed7c0c1430 0x7fed7c15b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:38.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed7b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed7c0c1430 0x7fed7c15b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50526/0 (socket says 172.21.15.203:50526) 2021-01-06T22:08:38.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed7b7fe700 1 -- 172.21.15.203:0/945452039 learned_addr learned my addr 172.21.15.203:0/945452039 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:38.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed7bfff700 1 --2- 172.21.15.203:0/945452039 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed7c0bb230 0x7fed7c067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:38.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed8099f700 1 --2- 172.21.15.203:0/945452039 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed7c147d60 0x7fed7c15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:38.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed7b7fe700 1 -- 172.21.15.203:0/945452039 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed7c0bb230 msgr2=0x7fed7c067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:38.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed7b7fe700 1 --2- 172.21.15.203:0/945452039 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed7c0bb230 0x7fed7c067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:38.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed7b7fe700 1 -- 172.21.15.203:0/945452039 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed7c147d60 msgr2=0x7fed7c15ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:38.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed7b7fe700 1 --2- 172.21.15.203:0/945452039 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed7c147d60 0x7fed7c15ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:38.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed7b7fe700 1 -- 172.21.15.203:0/945452039 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c14f3b0 con 0x7fed7c0c1430 2021-01-06T22:08:38.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed7b7fe700 1 --2- 172.21.15.203:0/945452039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed7c0c1430 0x7fed7c15b540 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fed700118e0 tx=0x7fed70016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:38.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed63fff700 1 -- 172.21.15.203:0/945452039 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7001d430 con 0x7fed7c0c1430 2021-01-06T22:08:38.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed63fff700 1 -- 172.21.15.203:0/945452039 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7001d590 con 0x7fed7c0c1430 2021-01-06T22:08:38.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed811a0700 1 -- 172.21.15.203:0/945452039 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7c14f650 con 0x7fed7c0c1430 2021-01-06T22:08:38.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed63fff700 1 -- 172.21.15.203:0/945452039 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7001d910 con 0x7fed7c0c1430 2021-01-06T22:08:38.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed811a0700 1 -- 172.21.15.203:0/945452039 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7c14bba0 con 0x7fed7c0c1430 2021-01-06T22:08:38.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed811a0700 1 -- 172.21.15.203:0/945452039 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7c0450a0 con 0x7fed7c0c1430 2021-01-06T22:08:38.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.095+0000 7fed63fff700 1 -- 172.21.15.203:0/945452039 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fed7000f070 con 0x7fed7c0c1430 2021-01-06T22:08:38.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.099+0000 7fed63fff700 1 --2- 172.21.15.203:0/945452039 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fed5c056400 0x7fed5c0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:38.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.099+0000 7fed63fff700 1 -- 172.21.15.203:0/945452039 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fed700711c0 con 0x7fed7c0c1430 2021-01-06T22:08:38.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.099+0000 7fed7bfff700 1 --2- 172.21.15.203:0/945452039 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fed5c056400 0x7fed5c0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:38.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.099+0000 7fed7bfff700 1 --2- 172.21.15.203:0/945452039 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fed5c056400 0x7fed5c0588c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fed6c000bc0 tx=0x7fed6c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:38.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.103+0000 7fed63fff700 1 -- 172.21.15.203:0/945452039 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fed70046990 con 0x7fed7c0c1430 2021-01-06T22:08:38.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.339+0000 7fed811a0700 1 -- 172.21.15.203:0/945452039 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed7c05ca60 con 0x7fed7c0c1430 2021-01-06T22:08:38.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.339+0000 7fed63fff700 1 -- 172.21.15.203:0/945452039 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+3207 (secure 0 0 0) 0x7fed70049fb0 con 0x7fed7c0c1430 2021-01-06T22:08:38.333 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:38.333 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:35.158095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":4273},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:38.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.339+0000 7fed811a0700 1 -- 172.21.15.203:0/945452039 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fed5c056400 msgr2=0x7fed5c0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:38.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.339+0000 7fed811a0700 1 --2- 172.21.15.203:0/945452039 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fed5c056400 0x7fed5c0588c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fed6c000bc0 tx=0x7fed6c009040).stop 2021-01-06T22:08:38.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.339+0000 7fed811a0700 1 -- 172.21.15.203:0/945452039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed7c0c1430 msgr2=0x7fed7c15b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:38.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.339+0000 7fed811a0700 1 --2- 172.21.15.203:0/945452039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed7c0c1430 0x7fed7c15b540 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fed700118e0 tx=0x7fed70016000).stop 2021-01-06T22:08:38.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.339+0000 7fed811a0700 1 -- 172.21.15.203:0/945452039 shutdown_connections 2021-01-06T22:08:38.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.339+0000 7fed811a0700 1 --2- 172.21.15.203:0/945452039 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fed5c056400 0x7fed5c0588c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:38.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.339+0000 7fed811a0700 1 --2- 172.21.15.203:0/945452039 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed7c147d60 0x7fed7c15ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:38.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.339+0000 7fed811a0700 1 --2- 172.21.15.203:0/945452039 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed7c0bb230 0x7fed7c067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:38.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.339+0000 7fed811a0700 1 --2- 172.21.15.203:0/945452039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed7c0c1430 0x7fed7c15b540 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:38.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.339+0000 7fed811a0700 1 -- 172.21.15.203:0/945452039 >> 172.21.15.203:0/945452039 conn(0x7fed7c0667b0 msgr2=0x7fed7c0b9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:38.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.339+0000 7fed811a0700 1 -- 172.21.15.203:0/945452039 shutdown_connections 2021-01-06T22:08:38.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:38.339+0000 7fed811a0700 1 -- 172.21.15.203:0/945452039 wait complete. 2021-01-06T22:08:38.338 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 31 2021-01-06T22:08:38.347 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:08:40.349 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:40.480 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d20ae700 1 Processor -- start 2021-01-06T22:08:40.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d20ae700 1 -- start start 2021-01-06T22:08:40.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d20ae700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6cc1513f0 0x7fd6cc0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:40.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d20ae700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6cc0bacb0 con 0x7fd6cc14ad90 2021-01-06T22:08:40.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d20ae700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6cc152000 con 0x7fd6cc0bccb0 2021-01-06T22:08:40.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d20ae700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc0bd130 con 0x7fd6cc1513f0 2021-01-06T22:08:40.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d08ab700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fd6cc0bccb0 0x7fd6cc1517d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49130/0 (socket says 172.21.15.203:49130) 2021-01-06T22:08:40.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d08ab700 1 -- 172.21.15.203:0/1682588840 learned_addr learned my addr 172.21.15.203:0/1682588840 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:40.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d10ac700 1 --2- 172.21.15.203:0/1682588840 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6cc1513f0 0x7fd6cc0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:40.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6bbfff700 1 -- 172.21.15.203:0/1682588840 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830763488 0 0) 0x7fd6cc152000 con 0x7fd6cc0bccb0 2021-01-06T22:08:40.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6bbfff700 1 -- 172.21.15.203:0/1682588840 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6b4003230 con 0x7fd6cc0bccb0 2021-01-06T22:08:40.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6bbfff700 1 -- 172.21.15.203:0/1682588840 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2298753150 0 0) 0x7fd6b4003230 con 0x7fd6cc0bccb0 2021-01-06T22:08:40.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6bbfff700 1 -- 172.21.15.203:0/1682588840 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6cc1513f0 msgr2=0x7fd6cc0c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:40.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6bbfff700 1 --2- 172.21.15.203:0/1682588840 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6cc1513f0 0x7fd6cc0c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:40.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6bbfff700 1 -- 172.21.15.203:0/1682588840 >> v1:172.21.15.91:6789/0 conn(0x7fd6cc14ad90 legacy=0x7fd6cc0641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:40.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6bbfff700 1 -- 172.21.15.203:0/1682588840 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc067e50 con 0x7fd6cc0bccb0 2021-01-06T22:08:40.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6bbfff700 1 -- 172.21.15.203:0/1682588840 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7fd6c80014e0 con 0x7fd6cc0bccb0 2021-01-06T22:08:40.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6bbfff700 1 -- 172.21.15.203:0/1682588840 >> v1:172.21.15.203:6789/0 conn(0x7fd6cc0bccb0 legacy=0x7fd6cc1517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:40.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6bbfff700 1 --2- 172.21.15.203:0/1682588840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b4003720 0x7fd6b4003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:40.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6bbfff700 1 --2- 172.21.15.203:0/1682588840 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6cc1513f0 0x7fd6b40044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:40.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6bbfff700 1 --2- 172.21.15.203:0/1682588840 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6b40049c0 0x7fd6b4004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:40.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6bbfff700 1 -- 172.21.15.203:0/1682588840 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc0bd130 con 0x7fd6b40049c0 2021-01-06T22:08:40.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6bbfff700 1 -- 172.21.15.203:0/1682588840 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b4004170 con 0x7fd6b4003720 2021-01-06T22:08:40.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6bbfff700 1 -- 172.21.15.203:0/1682588840 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b4004330 con 0x7fd6cc1513f0 2021-01-06T22:08:40.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d10ac700 1 --2- 172.21.15.203:0/1682588840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b4003720 0x7fd6b4003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:40.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d18ad700 1 --2- 172.21.15.203:0/1682588840 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6b40049c0 0x7fd6b4004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:40.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d08ab700 1 --2- 172.21.15.203:0/1682588840 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6cc1513f0 0x7fd6b40044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:40.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d10ac700 1 -- 172.21.15.203:0/1682588840 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6cc1513f0 msgr2=0x7fd6b40044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:40.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d10ac700 1 --2- 172.21.15.203:0/1682588840 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6cc1513f0 0x7fd6b40044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:40.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d10ac700 1 -- 172.21.15.203:0/1682588840 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6b40049c0 msgr2=0x7fd6b4004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:40.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d10ac700 1 --2- 172.21.15.203:0/1682588840 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6b40049c0 0x7fd6b4004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:40.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d10ac700 1 -- 172.21.15.203:0/1682588840 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc067e50 con 0x7fd6b4003720 2021-01-06T22:08:40.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d18ad700 1 --2- 172.21.15.203:0/1682588840 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6b40049c0 0x7fd6b4004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:08:40.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.487+0000 7fd6d10ac700 1 --2- 172.21.15.203:0/1682588840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b4003720 0x7fd6b4003b40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fd6bc00bb90 tx=0x7fd6bc008040).ready entity=mon.1 client_cookie=56d9de2d97ace156 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:40.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6bbfff700 1 -- 172.21.15.203:0/1682588840 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc00b2b0 con 0x7fd6b4003720 2021-01-06T22:08:40.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6bbfff700 1 -- 172.21.15.203:0/1682588840 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6bc00ea90 con 0x7fd6b4003720 2021-01-06T22:08:40.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6bbfff700 1 -- 172.21.15.203:0/1682588840 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc00ee30 con 0x7fd6b4003720 2021-01-06T22:08:40.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d20ae700 1 -- 172.21.15.203:0/1682588840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b4003720 msgr2=0x7fd6b4003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:40.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d20ae700 1 --2- 172.21.15.203:0/1682588840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b4003720 0x7fd6b4003b40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fd6bc00bb90 tx=0x7fd6bc008040).stop 2021-01-06T22:08:40.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d18ad700 1 -- 172.21.15.203:0/1682588840 reap_dead start 2021-01-06T22:08:40.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d20ae700 1 -- 172.21.15.203:0/1682588840 shutdown_connections 2021-01-06T22:08:40.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d20ae700 1 -- 172.21.15.203:0/1682588840 >> 172.21.15.203:0/1682588840 conn(0x7fd6cc0667b0 msgr2=0x7fd6cc0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:40.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d20ae700 1 -- 172.21.15.203:0/1682588840 shutdown_connections 2021-01-06T22:08:40.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d20ae700 1 -- 172.21.15.203:0/1682588840 wait complete. 2021-01-06T22:08:40.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d20ae700 1 Processor -- start 2021-01-06T22:08:40.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d20ae700 1 -- start start 2021-01-06T22:08:40.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d20ae700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6cc060450 0x7fd6cc15d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:40.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d20ae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6cc147ac0 0x7fd6cc0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:40.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d20ae700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6cc060830 0x7fd6cc0ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:40.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d20ae700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc160bd0 con 0x7fd6cc060830 2021-01-06T22:08:40.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d20ae700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc0b2940 con 0x7fd6cc147ac0 2021-01-06T22:08:40.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d20ae700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc0b2be0 con 0x7fd6cc060450 2021-01-06T22:08:40.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d08ab700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6cc147ac0 0x7fd6cc0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:40.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d08ab700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6cc147ac0 0x7fd6cc0b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50550/0 (socket says 172.21.15.203:50550) 2021-01-06T22:08:40.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d08ab700 1 -- 172.21.15.203:0/3282654437 learned_addr learned my addr 172.21.15.203:0/3282654437 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:40.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d10ac700 1 --2- 172.21.15.203:0/3282654437 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6cc060450 0x7fd6cc15d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:40.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.491+0000 7fd6d18ad700 1 --2- 172.21.15.203:0/3282654437 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6cc060830 0x7fd6cc0ba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:40.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6d08ab700 1 -- 172.21.15.203:0/3282654437 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6cc060450 msgr2=0x7fd6cc15d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:40.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6d08ab700 1 --2- 172.21.15.203:0/3282654437 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6cc060450 0x7fd6cc15d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:40.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6d08ab700 1 -- 172.21.15.203:0/3282654437 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6cc060830 msgr2=0x7fd6cc0ba6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:40.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6d08ab700 1 --2- 172.21.15.203:0/3282654437 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6cc060830 0x7fd6cc0ba6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:40.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6d08ab700 1 -- 172.21.15.203:0/3282654437 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc0b2e80 con 0x7fd6cc147ac0 2021-01-06T22:08:40.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6d08ab700 1 --2- 172.21.15.203:0/3282654437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6cc147ac0 0x7fd6cc0b7be0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fd6c80011c0 tx=0x7fd6c8001560).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:40.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6b97fa700 1 -- 172.21.15.203:0/3282654437 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8007e00 con 0x7fd6cc147ac0 2021-01-06T22:08:40.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6b97fa700 1 -- 172.21.15.203:0/3282654437 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c800ec50 con 0x7fd6cc147ac0 2021-01-06T22:08:40.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6b97fa700 1 -- 172.21.15.203:0/3282654437 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8014430 con 0x7fd6cc147ac0 2021-01-06T22:08:40.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6d20ae700 1 -- 172.21.15.203:0/3282654437 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6cc160e00 con 0x7fd6cc147ac0 2021-01-06T22:08:40.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6d20ae700 1 -- 172.21.15.203:0/3282654437 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6cc161540 con 0x7fd6cc147ac0 2021-01-06T22:08:40.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6d20ae700 1 -- 172.21.15.203:0/3282654437 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6cc0450a0 con 0x7fd6cc147ac0 2021-01-06T22:08:40.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6b97fa700 1 -- 172.21.15.203:0/3282654437 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fd6c80147a0 con 0x7fd6cc147ac0 2021-01-06T22:08:40.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6b97fa700 1 --2- 172.21.15.203:0/3282654437 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd6ac056400 0x7fd6ac0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:40.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6b97fa700 1 -- 172.21.15.203:0/3282654437 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fd6c806f4f0 con 0x7fd6cc147ac0 2021-01-06T22:08:40.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6d10ac700 1 --2- 172.21.15.203:0/3282654437 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd6ac056400 0x7fd6ac0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:40.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.495+0000 7fd6d10ac700 1 --2- 172.21.15.203:0/3282654437 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd6ac056400 0x7fd6ac0588c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fd6cc0573a0 tx=0x7fd6bc01a000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:40.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.503+0000 7fd6b97fa700 1 -- 172.21.15.203:0/3282654437 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fd6c8044cc0 con 0x7fd6cc147ac0 2021-01-06T22:08:40.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.727+0000 7fd6d20ae700 1 -- 172.21.15.203:0/3282654437 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6cc05ca60 con 0x7fd6cc147ac0 2021-01-06T22:08:40.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.727+0000 7fd6b97fa700 1 -- 172.21.15.203:0/3282654437 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+3563 (secure 0 0 0) 0x7fd6c80482e0 con 0x7fd6cc147ac0 2021-01-06T22:08:40.723 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:40.723 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5068,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:35.158095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":4273},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:40.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.731+0000 7fd6d20ae700 1 -- 172.21.15.203:0/3282654437 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd6ac056400 msgr2=0x7fd6ac0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:40.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.731+0000 7fd6d20ae700 1 --2- 172.21.15.203:0/3282654437 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd6ac056400 0x7fd6ac0588c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fd6cc0573a0 tx=0x7fd6bc01a000).stop 2021-01-06T22:08:40.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.731+0000 7fd6d20ae700 1 -- 172.21.15.203:0/3282654437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6cc147ac0 msgr2=0x7fd6cc0b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:40.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.731+0000 7fd6d20ae700 1 --2- 172.21.15.203:0/3282654437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6cc147ac0 0x7fd6cc0b7be0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fd6c80011c0 tx=0x7fd6c8001560).stop 2021-01-06T22:08:40.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.731+0000 7fd6d20ae700 1 -- 172.21.15.203:0/3282654437 shutdown_connections 2021-01-06T22:08:40.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.731+0000 7fd6d20ae700 1 --2- 172.21.15.203:0/3282654437 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd6ac056400 0x7fd6ac0588c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:40.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.731+0000 7fd6d20ae700 1 --2- 172.21.15.203:0/3282654437 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6cc060830 0x7fd6cc0ba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:40.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.731+0000 7fd6d20ae700 1 --2- 172.21.15.203:0/3282654437 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6cc060450 0x7fd6cc15d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:40.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.731+0000 7fd6d20ae700 1 --2- 172.21.15.203:0/3282654437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6cc147ac0 0x7fd6cc0b7be0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:40.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.731+0000 7fd6d20ae700 1 -- 172.21.15.203:0/3282654437 >> 172.21.15.203:0/3282654437 conn(0x7fd6cc0667b0 msgr2=0x7fd6cc0befb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:40.727 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.731+0000 7fd6d20ae700 1 -- 172.21.15.203:0/3282654437 shutdown_connections 2021-01-06T22:08:40.727 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:40.731+0000 7fd6d20ae700 1 -- 172.21.15.203:0/3282654437 wait complete. 2021-01-06T22:08:40.727 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 32 2021-01-06T22:08:40.737 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:08:42.687 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:08:42.691 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:08:42.738 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:42.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.903+0000 7f165962a700 1 Processor -- start 2021-01-06T22:08:42.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.903+0000 7f165962a700 1 -- start start 2021-01-06T22:08:42.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f165962a700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16540be730 0x7f16540657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:42.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f165962a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f165414ba00 0x7f165414be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:42.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f165962a700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f165414c380 con 0x7f16540be350 2021-01-06T22:08:42.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f165962a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f165414ee80 con 0x7f165414ba00 2021-01-06T22:08:42.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f165962a700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165414f120 con 0x7f16540be730 2021-01-06T22:08:42.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f1658e29700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f165414ba00 0x7f165414be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:42.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f1653fff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f16540be350 0x7f16540ba350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49152/0 (socket says 172.21.15.203:49152) 2021-01-06T22:08:42.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f1658e29700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f165414ba00 0x7f165414be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50562/0 (socket says 172.21.15.203:50562) 2021-01-06T22:08:42.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f1653fff700 1 -- 172.21.15.203:0/3526847259 learned_addr learned my addr 172.21.15.203:0/3526847259 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:42.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f16537fe700 1 --2- 172.21.15.203:0/3526847259 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16540be730 0x7f16540657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:42.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f1658e29700 1 -- 172.21.15.203:0/3526847259 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16540be730 msgr2=0x7f16540657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:42.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f1658e29700 1 --2- 172.21.15.203:0/3526847259 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16540be730 0x7f16540657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:42.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f1652ffd700 1 -- 172.21.15.203:0/3526847259 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384449303 0 0) 0x7f165414c380 con 0x7f16540be350 2021-01-06T22:08:42.905 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f1658e29700 1 -- 172.21.15.203:0/3526847259 >> v1:172.21.15.203:6789/0 conn(0x7f16540be350 legacy=0x7f16540ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:42.905 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f1658e29700 1 -- 172.21.15.203:0/3526847259 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165414f3c0 con 0x7f165414ba00 2021-01-06T22:08:42.905 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f1658e29700 1 --2- 172.21.15.203:0/3526847259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f165414ba00 0x7f165414be90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f164c009c90 tx=0x7f164c0091d0).ready entity=mon.1 client_cookie=2d3e14d2d89aafb3 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:42.905 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f1652ffd700 1 -- 172.21.15.203:0/3526847259 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c00ba70 con 0x7f165414ba00 2021-01-06T22:08:42.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f1652ffd700 1 -- 172.21.15.203:0/3526847259 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f164c00bbd0 con 0x7f165414ba00 2021-01-06T22:08:42.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f1652ffd700 1 -- 172.21.15.203:0/3526847259 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c011400 con 0x7f165414ba00 2021-01-06T22:08:42.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f165962a700 1 -- 172.21.15.203:0/3526847259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f165414ba00 msgr2=0x7f165414be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:42.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f165962a700 1 --2- 172.21.15.203:0/3526847259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f165414ba00 0x7f165414be90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f164c009c90 tx=0x7f164c0091d0).stop 2021-01-06T22:08:42.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f165962a700 1 -- 172.21.15.203:0/3526847259 shutdown_connections 2021-01-06T22:08:42.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f165962a700 1 --2- 172.21.15.203:0/3526847259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f165414ba00 0x7f165414be90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:42.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f165962a700 1 --2- 172.21.15.203:0/3526847259 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16540be730 0x7f16540657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:42.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f165962a700 1 -- 172.21.15.203:0/3526847259 >> 172.21.15.203:0/3526847259 conn(0x7f1654067e20 msgr2=0x7f1654068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:42.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f165962a700 1 -- 172.21.15.203:0/3526847259 shutdown_connections 2021-01-06T22:08:42.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.907+0000 7f165962a700 1 -- 172.21.15.203:0/3526847259 wait complete. 2021-01-06T22:08:42.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.911+0000 7f165962a700 1 Processor -- start 2021-01-06T22:08:42.909 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.915+0000 7f165962a700 1 -- start start 2021-01-06T22:08:42.912 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.915+0000 7f165962a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16540be350 0x7f1654151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:42.912 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.915+0000 7f165962a700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16540be730 0x7f1654146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:42.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.915+0000 7f165962a700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f165414ba00 0x7f165415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:42.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.915+0000 7f165962a700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165414ac30 con 0x7f165414ba00 2021-01-06T22:08:42.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.915+0000 7f165962a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165414ae60 con 0x7f16540be350 2021-01-06T22:08:42.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.915+0000 7f165962a700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165414b100 con 0x7f16540be730 2021-01-06T22:08:42.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.915+0000 7f1653fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16540be350 0x7f1654151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:42.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.915+0000 7f1653fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16540be350 0x7f1654151c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50564/0 (socket says 172.21.15.203:50564) 2021-01-06T22:08:42.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.915+0000 7f1653fff700 1 -- 172.21.15.203:0/2486155266 learned_addr learned my addr 172.21.15.203:0/2486155266 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:42.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f1658e29700 1 --2- 172.21.15.203:0/2486155266 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f165414ba00 0x7f165415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:42.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f16537fe700 1 --2- 172.21.15.203:0/2486155266 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16540be730 0x7f1654146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:42.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f1653fff700 1 -- 172.21.15.203:0/2486155266 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16540be730 msgr2=0x7f1654146390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:42.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f1653fff700 1 --2- 172.21.15.203:0/2486155266 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16540be730 0x7f1654146390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:42.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f1653fff700 1 -- 172.21.15.203:0/2486155266 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f165414ba00 msgr2=0x7f165415b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:42.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f1653fff700 1 --2- 172.21.15.203:0/2486155266 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f165414ba00 0x7f165415b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:42.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f1653fff700 1 -- 172.21.15.203:0/2486155266 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165414b3a0 con 0x7f16540be350 2021-01-06T22:08:42.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f16537fe700 1 --2- 172.21.15.203:0/2486155266 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16540be730 0x7f1654146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:08:42.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f1653fff700 1 --2- 172.21.15.203:0/2486155266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16540be350 0x7f1654151c00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f164400c710 tx=0x7f1644007ab0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:42.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f163bfff700 1 -- 172.21.15.203:0/2486155266 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1644007320 con 0x7f16540be350 2021-01-06T22:08:42.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f163bfff700 1 -- 172.21.15.203:0/2486155266 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1644007480 con 0x7f16540be350 2021-01-06T22:08:42.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f165962a700 1 -- 172.21.15.203:0/2486155266 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f165414b640 con 0x7f16540be350 2021-01-06T22:08:42.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f163bfff700 1 -- 172.21.15.203:0/2486155266 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16440057f0 con 0x7f16540be350 2021-01-06T22:08:42.918 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f165962a700 1 -- 172.21.15.203:0/2486155266 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1654146a20 con 0x7f16540be350 2021-01-06T22:08:42.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f165962a700 1 -- 172.21.15.203:0/2486155266 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16540450a0 con 0x7f16540be350 2021-01-06T22:08:42.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f163bfff700 1 -- 172.21.15.203:0/2486155266 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f1644010030 con 0x7f16540be350 2021-01-06T22:08:42.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f163bfff700 1 --2- 172.21.15.203:0/2486155266 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1634056400 0x7f16340588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:42.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f163bfff700 1 -- 172.21.15.203:0/2486155266 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f164406f2d0 con 0x7f16540be350 2021-01-06T22:08:42.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f16537fe700 1 --2- 172.21.15.203:0/2486155266 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1634056400 0x7f16340588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:42.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.919+0000 7f16537fe700 1 --2- 172.21.15.203:0/2486155266 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1634056400 0x7f16340588c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f164800bb20 tx=0x7f1648008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:42.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:42.927+0000 7f163bfff700 1 -- 172.21.15.203:0/2486155266 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f1644044aa0 con 0x7f16540be350 2021-01-06T22:08:43.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:43.135+0000 7f165962a700 1 -- 172.21.15.203:0/2486155266 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16540bbdb0 con 0x7f16540be350 2021-01-06T22:08:43.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:43.135+0000 7f163bfff700 1 -- 172.21.15.203:0/2486155266 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+3563 (secure 0 0 0) 0x7f16440480c0 con 0x7f16540be350 2021-01-06T22:08:43.130 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:43.130 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5068,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:35.158095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":4273},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:43.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:43.139+0000 7f165962a700 1 -- 172.21.15.203:0/2486155266 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1634056400 msgr2=0x7f16340588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:43.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:43.139+0000 7f165962a700 1 --2- 172.21.15.203:0/2486155266 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1634056400 0x7f16340588c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f164800bb20 tx=0x7f1648008040).stop 2021-01-06T22:08:43.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:43.139+0000 7f165962a700 1 -- 172.21.15.203:0/2486155266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16540be350 msgr2=0x7f1654151c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:43.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:43.139+0000 7f165962a700 1 --2- 172.21.15.203:0/2486155266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16540be350 0x7f1654151c00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f164400c710 tx=0x7f1644007ab0).stop 2021-01-06T22:08:43.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:43.139+0000 7f165962a700 1 -- 172.21.15.203:0/2486155266 shutdown_connections 2021-01-06T22:08:43.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:43.139+0000 7f165962a700 1 --2- 172.21.15.203:0/2486155266 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1634056400 0x7f16340588c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:43.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:43.139+0000 7f165962a700 1 --2- 172.21.15.203:0/2486155266 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f165414ba00 0x7f165415b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:43.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:43.139+0000 7f165962a700 1 --2- 172.21.15.203:0/2486155266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16540be350 0x7f1654151c00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:43.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:43.139+0000 7f165962a700 1 --2- 172.21.15.203:0/2486155266 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16540be730 0x7f1654146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:43.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:43.139+0000 7f165962a700 1 -- 172.21.15.203:0/2486155266 >> 172.21.15.203:0/2486155266 conn(0x7f1654067e20 msgr2=0x7f1654064f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:43.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:43.139+0000 7f165962a700 1 -- 172.21.15.203:0/2486155266 shutdown_connections 2021-01-06T22:08:43.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:43.139+0000 7f165962a700 1 -- 172.21.15.203:0/2486155266 wait complete. 2021-01-06T22:08:43.136 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 32 2021-01-06T22:08:43.146 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:08:45.147 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:45.283 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.287+0000 7fa91565a700 1 Processor -- start 2021-01-06T22:08:45.284 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.287+0000 7fa91565a700 1 -- start start 2021-01-06T22:08:45.286 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa91565a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa910151160 0x7fa910060750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:45.286 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa91565a700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9100602b0 con 0x7fa9100633f0 2021-01-06T22:08:45.286 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa91565a700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa910151710 con 0x7fa910063010 2021-01-06T22:08:45.286 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa91565a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91014b760 con 0x7fa910151160 2021-01-06T22:08:45.287 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa90ffff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fa910063010 0x7fa91014e3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49166/0 (socket says 172.21.15.203:49166) 2021-01-06T22:08:45.287 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa90ffff700 1 -- 172.21.15.203:0/3532244425 learned_addr learned my addr 172.21.15.203:0/3532244425 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:45.287 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa914e59700 1 --2- 172.21.15.203:0/3532244425 >> v2:172.21.15.203:3300/0 conn(0x7fa910151160 0x7fa910060750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:45.287 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa90effd700 1 -- 172.21.15.203:0/3532244425 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226001574 0 0) 0x7fa910151710 con 0x7fa910063010 2021-01-06T22:08:45.288 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa90effd700 1 -- 172.21.15.203:0/3532244425 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8fc003230 con 0x7fa910063010 2021-01-06T22:08:45.288 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa914e59700 1 -- 172.21.15.203:0/3532244425 >> v1:172.21.15.203:6789/0 conn(0x7fa910063010 legacy=0x7fa91014e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:45.288 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa914e59700 1 -- 172.21.15.203:0/3532244425 >> v1:172.21.15.91:6789/0 conn(0x7fa9100633f0 legacy=0x7fa9100be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:45.288 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa914e59700 1 -- 172.21.15.203:0/3532244425 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9101519b0 con 0x7fa910151160 2021-01-06T22:08:45.289 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa914e59700 1 --2- 172.21.15.203:0/3532244425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa910151160 0x7fa910060750 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fa908009c90 tx=0x7fa9080091d0).ready entity=mon.1 client_cookie=7f745f40e46396c5 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:45.289 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa90effd700 1 -- 172.21.15.203:0/3532244425 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90800bb00 con 0x7fa910151160 2021-01-06T22:08:45.289 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa90effd700 1 -- 172.21.15.203:0/3532244425 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90800bc60 con 0x7fa910151160 2021-01-06T22:08:45.290 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa90effd700 1 -- 172.21.15.203:0/3532244425 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908011400 con 0x7fa910151160 2021-01-06T22:08:45.292 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa91565a700 1 -- 172.21.15.203:0/3532244425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa910151160 msgr2=0x7fa910060750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:45.292 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa91565a700 1 --2- 172.21.15.203:0/3532244425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa910151160 0x7fa910060750 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fa908009c90 tx=0x7fa9080091d0).stop 2021-01-06T22:08:45.292 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa91565a700 1 -- 172.21.15.203:0/3532244425 shutdown_connections 2021-01-06T22:08:45.292 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa91565a700 1 --2- 172.21.15.203:0/3532244425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa910151160 0x7fa910060750 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:45.292 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa91565a700 1 -- 172.21.15.203:0/3532244425 >> 172.21.15.203:0/3532244425 conn(0x7fa910067e20 msgr2=0x7fa910068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:45.293 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa91565a700 1 -- 172.21.15.203:0/3532244425 shutdown_connections 2021-01-06T22:08:45.293 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.291+0000 7fa91565a700 1 -- 172.21.15.203:0/3532244425 wait complete. 2021-01-06T22:08:45.293 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.295+0000 7fa91565a700 1 Processor -- start 2021-01-06T22:08:45.293 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.295+0000 7fa91565a700 1 -- start start 2021-01-06T22:08:45.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.295+0000 7fa91565a700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa910063010 0x7fa91014c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:45.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa91565a700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa9100633f0 0x7fa910160450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:45.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa91565a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa910151160 0x7fa91015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:45.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa91565a700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91015ce80 con 0x7fa9100633f0 2021-01-06T22:08:45.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa91565a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91014c960 con 0x7fa910151160 2021-01-06T22:08:45.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa91565a700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa910160c00 con 0x7fa910063010 2021-01-06T22:08:45.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa914e59700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa910151160 0x7fa91015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:45.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa90ffff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa910063010 0x7fa91014c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:45.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa914e59700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa910151160 0x7fa91015d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50582/0 (socket says 172.21.15.203:50582) 2021-01-06T22:08:45.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa914e59700 1 -- 172.21.15.203:0/1672658495 learned_addr learned my addr 172.21.15.203:0/1672658495 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:45.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa90f7fe700 1 --2- 172.21.15.203:0/1672658495 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa9100633f0 0x7fa910160450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:45.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa914e59700 1 -- 172.21.15.203:0/1672658495 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa910063010 msgr2=0x7fa91014c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:45.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa914e59700 1 --2- 172.21.15.203:0/1672658495 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa910063010 0x7fa91014c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:45.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa914e59700 1 -- 172.21.15.203:0/1672658495 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa9100633f0 msgr2=0x7fa910160450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:45.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa914e59700 1 --2- 172.21.15.203:0/1672658495 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa9100633f0 0x7fa910160450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:45.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa914e59700 1 -- 172.21.15.203:0/1672658495 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa910160ea0 con 0x7fa910151160 2021-01-06T22:08:45.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa914e59700 1 --2- 172.21.15.203:0/1672658495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa910151160 0x7fa91015d7b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fa90800b540 tx=0x7fa908017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:45.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa8f7fff700 1 -- 172.21.15.203:0/1672658495 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90801b6f0 con 0x7fa910151160 2021-01-06T22:08:45.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa8f7fff700 1 -- 172.21.15.203:0/1672658495 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90801b850 con 0x7fa910151160 2021-01-06T22:08:45.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa8f7fff700 1 -- 172.21.15.203:0/1672658495 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90801bbf0 con 0x7fa910151160 2021-01-06T22:08:45.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa91565a700 1 -- 172.21.15.203:0/1672658495 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa910161140 con 0x7fa910151160 2021-01-06T22:08:45.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa91565a700 1 -- 172.21.15.203:0/1672658495 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91015d310 con 0x7fa910151160 2021-01-06T22:08:45.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa91565a700 1 -- 172.21.15.203:0/1672658495 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91015d4d0 con 0x7fa910151160 2021-01-06T22:08:45.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.299+0000 7fa8f7fff700 1 -- 172.21.15.203:0/1672658495 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa90800f070 con 0x7fa910151160 2021-01-06T22:08:45.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.303+0000 7fa8f7fff700 1 --2- 172.21.15.203:0/1672658495 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8f00563d0 0x7fa8f0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:45.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.303+0000 7fa8f7fff700 1 -- 172.21.15.203:0/1672658495 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fa9080712a0 con 0x7fa910151160 2021-01-06T22:08:45.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.303+0000 7fa90ffff700 1 --2- 172.21.15.203:0/1672658495 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8f00563d0 0x7fa8f0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:45.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.303+0000 7fa90ffff700 1 --2- 172.21.15.203:0/1672658495 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8f00563d0 0x7fa8f0058890 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fa900005590 tx=0x7fa900009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:45.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.307+0000 7fa8f7fff700 1 -- 172.21.15.203:0/1672658495 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa908046a70 con 0x7fa910151160 2021-01-06T22:08:45.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.551+0000 7fa91565a700 1 -- 172.21.15.203:0/1672658495 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9100450a0 con 0x7fa910151160 2021-01-06T22:08:45.547 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:45.547 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5068,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:35.158095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":4273},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:45.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.551+0000 7fa8f7fff700 1 -- 172.21.15.203:0/1672658495 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+3563 (secure 0 0 0) 0x7fa90804a090 con 0x7fa910151160 2021-01-06T22:08:45.549 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.555+0000 7fa91565a700 1 -- 172.21.15.203:0/1672658495 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8f00563d0 msgr2=0x7fa8f0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:45.549 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.555+0000 7fa91565a700 1 --2- 172.21.15.203:0/1672658495 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8f00563d0 0x7fa8f0058890 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fa900005590 tx=0x7fa900009040).stop 2021-01-06T22:08:45.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.555+0000 7fa91565a700 1 -- 172.21.15.203:0/1672658495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa910151160 msgr2=0x7fa91015d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:45.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.555+0000 7fa91565a700 1 --2- 172.21.15.203:0/1672658495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa910151160 0x7fa91015d7b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fa90800b540 tx=0x7fa908017000).stop 2021-01-06T22:08:45.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.555+0000 7fa91565a700 1 -- 172.21.15.203:0/1672658495 shutdown_connections 2021-01-06T22:08:45.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.555+0000 7fa91565a700 1 --2- 172.21.15.203:0/1672658495 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa8f00563d0 0x7fa8f0058890 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:45.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.555+0000 7fa91565a700 1 --2- 172.21.15.203:0/1672658495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa910151160 0x7fa91015d7b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:45.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.555+0000 7fa91565a700 1 --2- 172.21.15.203:0/1672658495 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa910063010 0x7fa91014c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:45.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.555+0000 7fa91565a700 1 --2- 172.21.15.203:0/1672658495 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa9100633f0 0x7fa910160450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:45.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.555+0000 7fa91565a700 1 -- 172.21.15.203:0/1672658495 >> 172.21.15.203:0/1672658495 conn(0x7fa910067e20 msgr2=0x7fa91014ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:45.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.555+0000 7fa91565a700 1 -- 172.21.15.203:0/1672658495 shutdown_connections 2021-01-06T22:08:45.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:45.555+0000 7fa91565a700 1 -- 172.21.15.203:0/1672658495 wait complete. 2021-01-06T22:08:45.553 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 32 2021-01-06T22:08:45.562 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:08:47.563 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:47.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.703+0000 7fbcbb59e700 1 Processor -- start 2021-01-06T22:08:47.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.703+0000 7fbcbb59e700 1 -- start start 2021-01-06T22:08:47.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.703+0000 7fbcbb59e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbcbc0c44f0 0x7fbcbc0c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:47.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.703+0000 7fbcbb59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbcbc060fc0 0x7fbcbc150e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:47.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcbb59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcbc0c0f50 con 0x7fbcbc0c48d0 2021-01-06T22:08:47.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcbb59e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcbc066cf0 con 0x7fbcbc0c44f0 2021-01-06T22:08:47.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcbb59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc060750 con 0x7fbcbc060fc0 2021-01-06T22:08:47.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcba59c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbcbc0c44f0 0x7fbcbc0c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:47.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcba59c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbcbc0c44f0 0x7fbcbc0c1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50586/0 (socket says 172.21.15.203:50586) 2021-01-06T22:08:47.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcba59c700 1 -- 172.21.15.203:0/2773162956 learned_addr learned my addr 172.21.15.203:0/2773162956 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:47.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcbad9d700 1 --2- 172.21.15.203:0/2773162956 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbcbc060fc0 0x7fbcbc150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:47.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcba59c700 1 -- 172.21.15.203:0/2773162956 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbcbc060fc0 msgr2=0x7fbcbc150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:47.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcba59c700 1 --2- 172.21.15.203:0/2773162956 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbcbc060fc0 0x7fbcbc150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:47.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcba59c700 1 -- 172.21.15.203:0/2773162956 >> v1:172.21.15.91:6789/0 conn(0x7fbcbc0c48d0 legacy=0x7fbcbc150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:47.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcba59c700 1 -- 172.21.15.203:0/2773162956 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc0609f0 con 0x7fbcbc0c44f0 2021-01-06T22:08:47.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcba59c700 1 --2- 172.21.15.203:0/2773162956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcbc0c44f0 0x7fbcbc0c1380 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7fbcac00b9e0 tx=0x7fbcac006ea0).ready entity=mon.1 client_cookie=18a5c4bcf397f091 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:47.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcb9d9b700 1 -- 172.21.15.203:0/2773162956 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac0098d0 con 0x7fbcbc0c44f0 2021-01-06T22:08:47.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcb9d9b700 1 -- 172.21.15.203:0/2773162956 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcac009a30 con 0x7fbcbc0c44f0 2021-01-06T22:08:47.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcb9d9b700 1 -- 172.21.15.203:0/2773162956 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac009db0 con 0x7fbcbc0c44f0 2021-01-06T22:08:47.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcbb59e700 1 -- 172.21.15.203:0/2773162956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcbc0c44f0 msgr2=0x7fbcbc0c1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:47.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcbb59e700 1 --2- 172.21.15.203:0/2773162956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcbc0c44f0 0x7fbcbc0c1380 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7fbcac00b9e0 tx=0x7fbcac006ea0).stop 2021-01-06T22:08:47.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcbb59e700 1 -- 172.21.15.203:0/2773162956 shutdown_connections 2021-01-06T22:08:47.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcbb59e700 1 --2- 172.21.15.203:0/2773162956 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbcbc060fc0 0x7fbcbc150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:47.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcbb59e700 1 --2- 172.21.15.203:0/2773162956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcbc0c44f0 0x7fbcbc0c1380 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:47.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcbb59e700 1 -- 172.21.15.203:0/2773162956 >> 172.21.15.203:0/2773162956 conn(0x7fbcbc056cd0 msgr2=0x7fbcbc061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:47.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcbb59e700 1 -- 172.21.15.203:0/2773162956 shutdown_connections 2021-01-06T22:08:47.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.707+0000 7fbcbb59e700 1 -- 172.21.15.203:0/2773162956 wait complete. 2021-01-06T22:08:47.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbb59e700 1 Processor -- start 2021-01-06T22:08:47.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbb59e700 1 -- start start 2021-01-06T22:08:47.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbb59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbcbc060fc0 0x7fbcbc152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:47.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbb59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbcbc0c44f0 0x7fbcbc0c1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:47.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbb59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcbc0c48d0 0x7fbcbc151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:47.733 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbb59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc1509e0 con 0x7fbcbc060fc0 2021-01-06T22:08:47.733 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbb59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc150c10 con 0x7fbcbc0c48d0 2021-01-06T22:08:47.733 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbb59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc150eb0 con 0x7fbcbc0c44f0 2021-01-06T22:08:47.733 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcbc0c48d0 0x7fbcbc151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:47.734 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcbc0c48d0 0x7fbcbc151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50596/0 (socket says 172.21.15.203:50596) 2021-01-06T22:08:47.734 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbad9d700 1 -- 172.21.15.203:0/1290411733 learned_addr learned my addr 172.21.15.203:0/1290411733 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:47.734 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcb1d9b700 1 --2- 172.21.15.203:0/1290411733 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbcbc0c44f0 0x7fbcbc0c1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:47.734 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcba59c700 1 --2- 172.21.15.203:0/1290411733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbcbc060fc0 0x7fbcbc152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:47.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbad9d700 1 -- 172.21.15.203:0/1290411733 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbcbc0c44f0 msgr2=0x7fbcbc0c1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:47.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbad9d700 1 --2- 172.21.15.203:0/1290411733 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbcbc0c44f0 0x7fbcbc0c1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:47.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbad9d700 1 -- 172.21.15.203:0/1290411733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbcbc060fc0 msgr2=0x7fbcbc152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:47.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbad9d700 1 --2- 172.21.15.203:0/1290411733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbcbc060fc0 0x7fbcbc152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:47.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbad9d700 1 -- 172.21.15.203:0/1290411733 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc0b84c0 con 0x7fbcbc0c48d0 2021-01-06T22:08:47.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcb1d9b700 1 --2- 172.21.15.203:0/1290411733 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbcbc0c44f0 0x7fbcbc0c1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:08:47.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbad9d700 1 --2- 172.21.15.203:0/1290411733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcbc0c48d0 0x7fbcbc151150 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fbcb4006fd0 tx=0x7fbcb4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:47.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcb2ffd700 1 -- 172.21.15.203:0/1290411733 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb400c7d0 con 0x7fbcbc0c48d0 2021-01-06T22:08:47.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcb2ffd700 1 -- 172.21.15.203:0/1290411733 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb400c930 con 0x7fbcbc0c48d0 2021-01-06T22:08:47.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbb59e700 1 -- 172.21.15.203:0/1290411733 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcbc0b8760 con 0x7fbcbc0c48d0 2021-01-06T22:08:47.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcb2ffd700 1 -- 172.21.15.203:0/1290411733 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb400ccb0 con 0x7fbcbc0c48d0 2021-01-06T22:08:47.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbb59e700 1 -- 172.21.15.203:0/1290411733 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcbc0bce60 con 0x7fbcbc0c48d0 2021-01-06T22:08:47.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.711+0000 7fbcbb59e700 1 -- 172.21.15.203:0/1290411733 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcbc0b88f0 con 0x7fbcbc0c48d0 2021-01-06T22:08:47.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.715+0000 7fbcb2ffd700 1 -- 172.21.15.203:0/1290411733 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fbcb4035650 con 0x7fbcbc0c48d0 2021-01-06T22:08:47.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.715+0000 7fbcb2ffd700 1 --2- 172.21.15.203:0/1290411733 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbca00563d0 0x7fbca0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:47.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.715+0000 7fbcb2ffd700 1 -- 172.21.15.203:0/1290411733 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fbcb4091710 con 0x7fbcbc0c48d0 2021-01-06T22:08:47.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.715+0000 7fbcba59c700 1 --2- 172.21.15.203:0/1290411733 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbca00563d0 0x7fbca0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:47.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.715+0000 7fbcba59c700 1 --2- 172.21.15.203:0/1290411733 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbca00563d0 0x7fbca0058890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fbcac007fd0 tx=0x7fbcac016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:47.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.719+0000 7fbcb2ffd700 1 -- 172.21.15.203:0/1290411733 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fbcb4066e80 con 0x7fbcbc0c48d0 2021-01-06T22:08:47.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.923+0000 7fbcbb59e700 1 -- 172.21.15.203:0/1290411733 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcbc0450a0 con 0x7fbcbc0c48d0 2021-01-06T22:08:47.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.923+0000 7fbcb2ffd700 1 -- 172.21.15.203:0/1290411733 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+3563 (secure 0 0 0) 0x7fbcb406a4a0 con 0x7fbcbc0c48d0 2021-01-06T22:08:47.917 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:47.917 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5068,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:35.158095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":4273},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:47.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.927+0000 7fbcbb59e700 1 -- 172.21.15.203:0/1290411733 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbca00563d0 msgr2=0x7fbca0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:47.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.927+0000 7fbcbb59e700 1 --2- 172.21.15.203:0/1290411733 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbca00563d0 0x7fbca0058890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fbcac007fd0 tx=0x7fbcac016000).stop 2021-01-06T22:08:47.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.927+0000 7fbcbb59e700 1 -- 172.21.15.203:0/1290411733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcbc0c48d0 msgr2=0x7fbcbc151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:47.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.927+0000 7fbcbb59e700 1 --2- 172.21.15.203:0/1290411733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcbc0c48d0 0x7fbcbc151150 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fbcb4006fd0 tx=0x7fbcb4008040).stop 2021-01-06T22:08:47.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.927+0000 7fbcbb59e700 1 -- 172.21.15.203:0/1290411733 shutdown_connections 2021-01-06T22:08:47.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.927+0000 7fbcbb59e700 1 --2- 172.21.15.203:0/1290411733 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbca00563d0 0x7fbca0058890 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:47.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.927+0000 7fbcbb59e700 1 --2- 172.21.15.203:0/1290411733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcbc0c48d0 0x7fbcbc151150 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:47.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.927+0000 7fbcbb59e700 1 --2- 172.21.15.203:0/1290411733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbcbc060fc0 0x7fbcbc152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:47.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.927+0000 7fbcbb59e700 1 --2- 172.21.15.203:0/1290411733 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbcbc0c44f0 0x7fbcbc0c1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:47.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.927+0000 7fbcbb59e700 1 -- 172.21.15.203:0/1290411733 >> 172.21.15.203:0/1290411733 conn(0x7fbcbc056cd0 msgr2=0x7fbcbc14fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:47.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.927+0000 7fbcbb59e700 1 -- 172.21.15.203:0/1290411733 shutdown_connections 2021-01-06T22:08:47.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:47.927+0000 7fbcbb59e700 1 -- 172.21.15.203:0/1290411733 wait complete. 2021-01-06T22:08:47.922 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 32 2021-01-06T22:08:47.933 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:08:49.934 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:50.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.062+0000 7f9c0b495700 1 Processor -- start 2021-01-06T22:08:50.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.062+0000 7f9c0b495700 1 -- start start 2021-01-06T22:08:50.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.062+0000 7f9c0b495700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9c040c1430 0x7f9c04148140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:50.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.062+0000 7f9c0b495700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c040bb7e0 con 0x7f9c040bb230 2021-01-06T22:08:50.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.062+0000 7f9c0b495700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c040bba80 con 0x7f9c04147d60 2021-01-06T22:08:50.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.062+0000 7f9c0b495700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c040bbd20 con 0x7f9c040c1430 2021-01-06T22:08:50.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c0a493700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f9c04147d60 0x7f9c040671a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49196/0 (socket says 172.21.15.203:49196) 2021-01-06T22:08:50.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c0a493700 1 -- 172.21.15.203:0/2078016754 learned_addr learned my addr 172.21.15.203:0/2078016754 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:50.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c09c92700 1 --2- 172.21.15.203:0/2078016754 >> v2:172.21.15.91:3300/0 conn(0x7f9c040c1430 0x7f9c04148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:50.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c09491700 1 -- 172.21.15.203:0/2078016754 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1991193477 0 0) 0x7f9c040bba80 con 0x7f9c04147d60 2021-01-06T22:08:50.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c09491700 1 -- 172.21.15.203:0/2078016754 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bec003230 con 0x7f9c04147d60 2021-01-06T22:08:50.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c09491700 1 -- 172.21.15.203:0/2078016754 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587723036 0 0) 0x7f9c040bb7e0 con 0x7f9c040bb230 2021-01-06T22:08:50.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c09491700 1 -- 172.21.15.203:0/2078016754 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c040bba80 con 0x7f9c040bb230 2021-01-06T22:08:50.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c09c92700 1 -- 172.21.15.203:0/2078016754 >> v1:172.21.15.203:6789/0 conn(0x7f9c04147d60 legacy=0x7f9c040671a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:50.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c09491700 1 -- 172.21.15.203:0/2078016754 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3319608187 0 0) 0x7f9bec003230 con 0x7f9c04147d60 2021-01-06T22:08:50.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c09c92700 1 -- 172.21.15.203:0/2078016754 >> v1:172.21.15.91:6789/0 conn(0x7f9c040bb230 legacy=0x7f9c040602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:50.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c09c92700 1 -- 172.21.15.203:0/2078016754 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c040bbfc0 con 0x7f9c040c1430 2021-01-06T22:08:50.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c09c92700 1 --2- 172.21.15.203:0/2078016754 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c040c1430 0x7f9c04148140 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f9bf400b900 tx=0x7f9bf4006de0).ready entity=mon.0 client_cookie=ed3f94de9d92a94f server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:50.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c09491700 1 -- 172.21.15.203:0/2078016754 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c000014e0 con 0x7f9c040c1430 2021-01-06T22:08:50.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c09491700 1 -- 172.21.15.203:0/2078016754 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf4011400 con 0x7f9c040c1430 2021-01-06T22:08:50.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c09491700 1 -- 172.21.15.203:0/2078016754 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4011780 con 0x7f9c040c1430 2021-01-06T22:08:50.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c0b495700 1 -- 172.21.15.203:0/2078016754 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c040c1430 msgr2=0x7f9c04148140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:50.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c0b495700 1 --2- 172.21.15.203:0/2078016754 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c040c1430 0x7f9c04148140 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f9bf400b900 tx=0x7f9bf4006de0).stop 2021-01-06T22:08:50.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c0b495700 1 -- 172.21.15.203:0/2078016754 shutdown_connections 2021-01-06T22:08:50.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c0b495700 1 --2- 172.21.15.203:0/2078016754 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c040c1430 0x7f9c04148140 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:50.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c0b495700 1 -- 172.21.15.203:0/2078016754 >> 172.21.15.203:0/2078016754 conn(0x7f9c040667b0 msgr2=0x7f9c040c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:50.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c0b495700 1 -- 172.21.15.203:0/2078016754 shutdown_connections 2021-01-06T22:08:50.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.066+0000 7f9c0b495700 1 -- 172.21.15.203:0/2078016754 wait complete. 2021-01-06T22:08:50.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0b495700 1 Processor -- start 2021-01-06T22:08:50.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0b495700 1 -- start start 2021-01-06T22:08:50.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0b495700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9c040bb230 0x7f9c0415be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:50.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0b495700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c040c1430 0x7f9c041451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:50.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0b495700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c04147d60 0x7f9c0414beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:50.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0b495700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c04145f80 con 0x7f9c040c1430 2021-01-06T22:08:50.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0b495700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c04151420 con 0x7f9c04147d60 2021-01-06T22:08:50.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0b495700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0415c320 con 0x7f9c040bb230 2021-01-06T22:08:50.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0ac94700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c04147d60 0x7f9c0414beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:50.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0ac94700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c04147d60 0x7f9c0414beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50612/0 (socket says 172.21.15.203:50612) 2021-01-06T22:08:50.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0ac94700 1 -- 172.21.15.203:0/3481475923 learned_addr learned my addr 172.21.15.203:0/3481475923 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:50.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c09c92700 1 --2- 172.21.15.203:0/3481475923 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c040c1430 0x7f9c041451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:50.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0a493700 1 --2- 172.21.15.203:0/3481475923 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9c040bb230 0x7f9c0415be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:50.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0ac94700 1 -- 172.21.15.203:0/3481475923 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9c040bb230 msgr2=0x7f9c0415be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:50.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0ac94700 1 --2- 172.21.15.203:0/3481475923 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9c040bb230 0x7f9c0415be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:50.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0ac94700 1 -- 172.21.15.203:0/3481475923 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c040c1430 msgr2=0x7f9c041451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:50.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0ac94700 1 --2- 172.21.15.203:0/3481475923 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c040c1430 0x7f9c041451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:50.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0ac94700 1 -- 172.21.15.203:0/3481475923 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c04067850 con 0x7f9c04147d60 2021-01-06T22:08:50.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0ac94700 1 --2- 172.21.15.203:0/3481475923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c04147d60 0x7f9c0414beb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f9bfc00cac0 tx=0x7f9bfc007dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:50.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9bf27fc700 1 -- 172.21.15.203:0/3481475923 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc002240 con 0x7f9c04147d60 2021-01-06T22:08:50.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0b495700 1 -- 172.21.15.203:0/3481475923 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c04067af0 con 0x7f9c04147d60 2021-01-06T22:08:50.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9bf27fc700 1 -- 172.21.15.203:0/3481475923 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc0023a0 con 0x7f9c04147d60 2021-01-06T22:08:50.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9bf27fc700 1 -- 172.21.15.203:0/3481475923 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc001820 con 0x7f9c04147d60 2021-01-06T22:08:50.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.070+0000 7f9c0b495700 1 -- 172.21.15.203:0/3481475923 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c04150c40 con 0x7f9c04147d60 2021-01-06T22:08:50.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.074+0000 7f9c0b495700 1 -- 172.21.15.203:0/3481475923 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be8004f60 con 0x7f9c04147d60 2021-01-06T22:08:50.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.074+0000 7f9bf27fc700 1 -- 172.21.15.203:0/3481475923 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f9bfc019030 con 0x7f9c04147d60 2021-01-06T22:08:50.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.074+0000 7f9bf27fc700 1 --2- 172.21.15.203:0/3481475923 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9be40563d0 0x7f9be4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:50.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.074+0000 7f9bf27fc700 1 -- 172.21.15.203:0/3481475923 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f9bfc06ec60 con 0x7f9c04147d60 2021-01-06T22:08:50.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.074+0000 7f9c0a493700 1 --2- 172.21.15.203:0/3481475923 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9be40563d0 0x7f9be4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:50.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.074+0000 7f9c0a493700 1 --2- 172.21.15.203:0/3481475923 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9be40563d0 0x7f9be4058890 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f9c00004a10 tx=0x7f9c00000b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:50.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.082+0000 7f9bf27fc700 1 -- 172.21.15.203:0/3481475923 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f9bfc0444e0 con 0x7f9c04147d60 2021-01-06T22:08:50.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.298+0000 7f9c0b495700 1 -- 172.21.15.203:0/3481475923 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9be8005790 con 0x7f9c04147d60 2021-01-06T22:08:50.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.298+0000 7f9bf27fc700 1 -- 172.21.15.203:0/3481475923 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+3563 (secure 0 0 0) 0x7f9bfc047b00 con 0x7f9c04147d60 2021-01-06T22:08:50.295 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:50.295 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5068,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:35.158095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":4273},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3277561222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3277561222},{"type":"v1","addr":"172.21.15.203:6837","nonce":3277561222}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:50.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.302+0000 7f9c0b495700 1 -- 172.21.15.203:0/3481475923 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9be40563d0 msgr2=0x7f9be4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:50.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.302+0000 7f9c0b495700 1 --2- 172.21.15.203:0/3481475923 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9be40563d0 0x7f9be4058890 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f9c00004a10 tx=0x7f9c00000b80).stop 2021-01-06T22:08:50.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.302+0000 7f9c0b495700 1 -- 172.21.15.203:0/3481475923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c04147d60 msgr2=0x7f9c0414beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:50.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.302+0000 7f9c0b495700 1 --2- 172.21.15.203:0/3481475923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c04147d60 0x7f9c0414beb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f9bfc00cac0 tx=0x7f9bfc007dc0).stop 2021-01-06T22:08:50.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.302+0000 7f9c0b495700 1 -- 172.21.15.203:0/3481475923 shutdown_connections 2021-01-06T22:08:50.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.302+0000 7f9c0b495700 1 --2- 172.21.15.203:0/3481475923 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9be40563d0 0x7f9be4058890 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:50.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.302+0000 7f9c0b495700 1 --2- 172.21.15.203:0/3481475923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c04147d60 0x7f9c0414beb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:50.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.302+0000 7f9c0b495700 1 --2- 172.21.15.203:0/3481475923 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9c040bb230 0x7f9c0415be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:50.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.302+0000 7f9c0b495700 1 --2- 172.21.15.203:0/3481475923 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c040c1430 0x7f9c041451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:50.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.302+0000 7f9c0b495700 1 -- 172.21.15.203:0/3481475923 >> 172.21.15.203:0/3481475923 conn(0x7f9c040667b0 msgr2=0x7f9c040b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:50.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.302+0000 7f9c0b495700 1 -- 172.21.15.203:0/3481475923 shutdown_connections 2021-01-06T22:08:50.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:50.302+0000 7f9c0b495700 1 -- 172.21.15.203:0/3481475923 wait complete. 2021-01-06T22:08:50.300 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 32 2021-01-06T22:08:50.310 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:08:52.311 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:52.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413669c700 1 Processor -- start 2021-01-06T22:08:52.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413669c700 1 -- start start 2021-01-06T22:08:52.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413669c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4130060ba0 0x7f41300bbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:52.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413669c700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f413014dae0 0x7f413014df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:52.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413669c700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41300c11a0 con 0x7f41300bbfa0 2021-01-06T22:08:52.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413669c700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41300c1440 con 0x7f413014dae0 2021-01-06T22:08:52.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413669c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41300c16e0 con 0x7f4130060ba0 2021-01-06T22:08:52.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413569a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4130060ba0 0x7f41300bbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:52.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413569a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4130060ba0 0x7f41300bbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50640/0 (socket says 172.21.15.203:50640) 2021-01-06T22:08:52.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413569a700 1 -- 172.21.15.203:0/1973051237 learned_addr learned my addr 172.21.15.203:0/1973051237 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:52.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f4135e9b700 1 --2- 172.21.15.203:0/1973051237 >> v2:172.21.15.91:3300/0 conn(0x7f413014dae0 0x7f413014df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:52.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413569a700 1 -- 172.21.15.203:0/1973051237 >> v2:172.21.15.91:3300/0 conn(0x7f413014dae0 msgr2=0x7f413014df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:52.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413569a700 1 --2- 172.21.15.203:0/1973051237 >> v2:172.21.15.91:3300/0 conn(0x7f413014dae0 0x7f413014df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:52.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413569a700 1 -- 172.21.15.203:0/1973051237 >> v1:172.21.15.91:6789/0 conn(0x7f41300bbfa0 legacy=0x7f4130062e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:52.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413569a700 1 -- 172.21.15.203:0/1973051237 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41300c1980 con 0x7f4130060ba0 2021-01-06T22:08:52.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413569a700 1 --2- 172.21.15.203:0/1973051237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4130060ba0 0x7f41300bbab0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f4130065d60 tx=0x7f4120006ea0).ready entity=mon.1 client_cookie=ada3fa779a944c58 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:52.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f412ffff700 1 -- 172.21.15.203:0/1973051237 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41200098d0 con 0x7f4130060ba0 2021-01-06T22:08:52.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f412ffff700 1 -- 172.21.15.203:0/1973051237 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4120009a30 con 0x7f4130060ba0 2021-01-06T22:08:52.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f412ffff700 1 -- 172.21.15.203:0/1973051237 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120009db0 con 0x7f4130060ba0 2021-01-06T22:08:52.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413669c700 1 -- 172.21.15.203:0/1973051237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4130060ba0 msgr2=0x7f41300bbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:52.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413669c700 1 --2- 172.21.15.203:0/1973051237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4130060ba0 0x7f41300bbab0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f4130065d60 tx=0x7f4120006ea0).stop 2021-01-06T22:08:52.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413669c700 1 -- 172.21.15.203:0/1973051237 shutdown_connections 2021-01-06T22:08:52.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413669c700 1 --2- 172.21.15.203:0/1973051237 >> v2:172.21.15.91:3300/0 conn(0x7f413014dae0 0x7f413014df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:52.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413669c700 1 --2- 172.21.15.203:0/1973051237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4130060ba0 0x7f41300bbab0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:52.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413669c700 1 -- 172.21.15.203:0/1973051237 >> 172.21.15.203:0/1973051237 conn(0x7f4130056cd0 msgr2=0x7f41300570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:52.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413669c700 1 -- 172.21.15.203:0/1973051237 shutdown_connections 2021-01-06T22:08:52.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.446+0000 7f413669c700 1 -- 172.21.15.203:0/1973051237 wait complete. 2021-01-06T22:08:52.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f413669c700 1 Processor -- start 2021-01-06T22:08:52.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f413669c700 1 -- start start 2021-01-06T22:08:52.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f413669c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4130060ba0 0x7f4130161580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:52.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f413669c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41300bbfa0 0x7f41300b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:52.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f413669c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f413014dae0 0x7f4130150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:52.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f413669c700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41300b6a30 con 0x7f41300bbfa0 2021-01-06T22:08:52.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f413669c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4130161a70 con 0x7f413014dae0 2021-01-06T22:08:52.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f413669c700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41300b6790 con 0x7f4130060ba0 2021-01-06T22:08:52.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f4135e9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f413014dae0 0x7f4130150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:52.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f4135e9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f413014dae0 0x7f4130150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50650/0 (socket says 172.21.15.203:50650) 2021-01-06T22:08:52.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f4135e9b700 1 -- 172.21.15.203:0/754747322 learned_addr learned my addr 172.21.15.203:0/754747322 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:52.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f413569a700 1 --2- 172.21.15.203:0/754747322 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4130060ba0 0x7f4130161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:52.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f4134e99700 1 --2- 172.21.15.203:0/754747322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41300bbfa0 0x7f41300b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:52.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f4135e9b700 1 -- 172.21.15.203:0/754747322 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4130060ba0 msgr2=0x7f4130161580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:52.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f4135e9b700 1 --2- 172.21.15.203:0/754747322 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4130060ba0 0x7f4130161580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:52.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f4135e9b700 1 -- 172.21.15.203:0/754747322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41300bbfa0 msgr2=0x7f41300b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:52.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f4135e9b700 1 --2- 172.21.15.203:0/754747322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41300bbfa0 0x7f41300b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:52.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f4135e9b700 1 -- 172.21.15.203:0/754747322 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4130150240 con 0x7f413014dae0 2021-01-06T22:08:52.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f4134e99700 1 --2- 172.21.15.203:0/754747322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41300bbfa0 0x7f41300b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:08:52.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f413569a700 1 --2- 172.21.15.203:0/754747322 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4130060ba0 0x7f4130161580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:08:52.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f4135e9b700 1 --2- 172.21.15.203:0/754747322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f413014dae0 0x7f4130150ec0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f412800db30 tx=0x7f4128008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:52.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f412d7fa700 1 -- 172.21.15.203:0/754747322 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41280099b0 con 0x7f413014dae0 2021-01-06T22:08:52.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f412d7fa700 1 -- 172.21.15.203:0/754747322 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4128009b10 con 0x7f413014dae0 2021-01-06T22:08:52.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f413669c700 1 -- 172.21.15.203:0/754747322 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41301504e0 con 0x7f413014dae0 2021-01-06T22:08:52.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f412d7fa700 1 -- 172.21.15.203:0/754747322 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128013480 con 0x7f413014dae0 2021-01-06T22:08:52.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.450+0000 7f413669c700 1 -- 172.21.15.203:0/754747322 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4130150a20 con 0x7f413014dae0 2021-01-06T22:08:52.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.454+0000 7f413669c700 1 -- 172.21.15.203:0/754747322 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4130062ba0 con 0x7f413014dae0 2021-01-06T22:08:52.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.454+0000 7f412d7fa700 1 -- 172.21.15.203:0/754747322 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f4128013870 con 0x7f413014dae0 2021-01-06T22:08:52.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.454+0000 7f412d7fa700 1 --2- 172.21.15.203:0/754747322 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4110056400 0x7f41100588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:52.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.454+0000 7f412d7fa700 1 -- 172.21.15.203:0/754747322 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f412806ff20 con 0x7f413014dae0 2021-01-06T22:08:52.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.454+0000 7f413569a700 1 --2- 172.21.15.203:0/754747322 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4110056400 0x7f41100588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:52.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.454+0000 7f413569a700 1 --2- 172.21.15.203:0/754747322 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4110056400 0x7f41100588c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f412000b190 tx=0x7f4120016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:52.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.458+0000 7f412d7fa700 1 -- 172.21.15.203:0/754747322 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f4130062ba0 con 0x7f413014dae0 2021-01-06T22:08:52.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.686+0000 7f413669c700 1 -- 172.21.15.203:0/754747322 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4130062ba0 con 0x7f413014dae0 2021-01-06T22:08:52.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.686+0000 7f412d7fa700 1 -- 172.21.15.203:0/754747322 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3209 (secure 0 0 0) 0x7f4130062ba0 con 0x7f413014dae0 2021-01-06T22:08:52.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.690+0000 7f413669c700 1 -- 172.21.15.203:0/754747322 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4110056400 msgr2=0x7f41100588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:52.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.690+0000 7f413669c700 1 --2- 172.21.15.203:0/754747322 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4110056400 0x7f41100588c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f412000b190 tx=0x7f4120016000).stop 2021-01-06T22:08:52.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.690+0000 7f413669c700 1 -- 172.21.15.203:0/754747322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f413014dae0 msgr2=0x7f4130150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:52.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.690+0000 7f413669c700 1 --2- 172.21.15.203:0/754747322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f413014dae0 0x7f4130150ec0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f412800db30 tx=0x7f4128008040).stop 2021-01-06T22:08:52.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.690+0000 7f413669c700 1 -- 172.21.15.203:0/754747322 shutdown_connections 2021-01-06T22:08:52.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.690+0000 7f413669c700 1 --2- 172.21.15.203:0/754747322 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4110056400 0x7f41100588c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:52.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.690+0000 7f413669c700 1 --2- 172.21.15.203:0/754747322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f413014dae0 0x7f4130150ec0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:52.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.690+0000 7f413669c700 1 --2- 172.21.15.203:0/754747322 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4130060ba0 0x7f4130161580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:52.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.690+0000 7f413669c700 1 --2- 172.21.15.203:0/754747322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41300bbfa0 0x7f41300b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:52.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.690+0000 7f413669c700 1 -- 172.21.15.203:0/754747322 >> 172.21.15.203:0/754747322 conn(0x7f4130056cd0 msgr2=0x7f413014aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:52.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.690+0000 7f413669c700 1 -- 172.21.15.203:0/754747322 shutdown_connections 2021-01-06T22:08:52.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:52.690+0000 7f413669c700 1 -- 172.21.15.203:0/754747322 wait complete. 2021-01-06T22:08:52.862 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 34 2021-01-06T22:08:52.862 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:52.863 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:52.414430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:resolve","state_seq":6,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:52.863 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2021-01-06T22:08:52.863 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2021-01-06T22:08:52.864 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:53.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf412e0700 1 Processor -- start 2021-01-06T22:08:53.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf412e0700 1 -- start start 2021-01-06T22:08:53.008 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf412e0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf3c1478c0 0x7fdf3c0bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:53.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf412e0700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3c0c16b0 con 0x7fdf3c14e310 2021-01-06T22:08:53.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf412e0700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3c0bb2d0 con 0x7fdf3c064380 2021-01-06T22:08:53.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf412e0700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c14a560 con 0x7fdf3c1478c0 2021-01-06T22:08:53.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3bfff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fdf3c064380 0x7fdf3c0c0fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49248/0 (socket says 172.21.15.203:49248) 2021-01-06T22:08:53.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3bfff700 1 -- 172.21.15.203:0/139653362 learned_addr learned my addr 172.21.15.203:0/139653362 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:53.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf40adf700 1 --2- 172.21.15.203:0/139653362 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf3c1478c0 0x7fdf3c0bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:53.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 -- 172.21.15.203:0/139653362 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932563418 0 0) 0x7fdf3c0bb2d0 con 0x7fdf3c064380 2021-01-06T22:08:53.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 -- 172.21.15.203:0/139653362 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf28003230 con 0x7fdf3c064380 2021-01-06T22:08:53.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 -- 172.21.15.203:0/139653362 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3429528205 0 0) 0x7fdf2c0010d0 con 0x7fdf3c14e310 2021-01-06T22:08:53.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 -- 172.21.15.203:0/139653362 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf3c0bb2d0 con 0x7fdf3c14e310 2021-01-06T22:08:53.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 -- 172.21.15.203:0/139653362 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2876318419 0 0) 0x7fdf28003230 con 0x7fdf3c064380 2021-01-06T22:08:53.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 -- 172.21.15.203:0/139653362 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf3c1478c0 msgr2=0x7fdf3c0bb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:53.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 --2- 172.21.15.203:0/139653362 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf3c1478c0 0x7fdf3c0bb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:53.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 -- 172.21.15.203:0/139653362 >> v1:172.21.15.91:6789/0 conn(0x7fdf3c14e310 legacy=0x7fdf3c1471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:53.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 -- 172.21.15.203:0/139653362 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c14a800 con 0x7fdf3c064380 2021-01-06T22:08:53.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 -- 172.21.15.203:0/139653362 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7fdf30001500 con 0x7fdf3c064380 2021-01-06T22:08:53.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 -- 172.21.15.203:0/139653362 >> v1:172.21.15.203:6789/0 conn(0x7fdf3c064380 legacy=0x7fdf3c0c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:53.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 --2- 172.21.15.203:0/139653362 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf3c1478c0 0x7fdf280038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:53.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 --2- 172.21.15.203:0/139653362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf28003da0 0x7fdf280044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:53.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 --2- 172.21.15.203:0/139653362 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf280049c0 0x7fdf28004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:53.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 -- 172.21.15.203:0/139653362 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c14a560 con 0x7fdf280049c0 2021-01-06T22:08:53.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 -- 172.21.15.203:0/139653362 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28005330 con 0x7fdf28003da0 2021-01-06T22:08:53.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.010+0000 7fdf3affd700 1 -- 172.21.15.203:0/139653362 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2800d960 con 0x7fdf3c1478c0 2021-01-06T22:08:53.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf3b7fe700 1 --2- 172.21.15.203:0/139653362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf28003da0 0x7fdf280044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:53.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf3bfff700 1 --2- 172.21.15.203:0/139653362 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf3c1478c0 0x7fdf280038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:53.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf40adf700 1 --2- 172.21.15.203:0/139653362 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf280049c0 0x7fdf28004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:53.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf3b7fe700 1 -- 172.21.15.203:0/139653362 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf3c1478c0 msgr2=0x7fdf280038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:53.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf3b7fe700 1 --2- 172.21.15.203:0/139653362 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf3c1478c0 0x7fdf280038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:53.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf3b7fe700 1 -- 172.21.15.203:0/139653362 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf280049c0 msgr2=0x7fdf28004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:53.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf3b7fe700 1 --2- 172.21.15.203:0/139653362 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf280049c0 0x7fdf28004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:53.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf3b7fe700 1 -- 172.21.15.203:0/139653362 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf28003230 con 0x7fdf28003da0 2021-01-06T22:08:53.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf3b7fe700 1 --2- 172.21.15.203:0/139653362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf28003da0 0x7fdf280044d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fdf2c007c20 tx=0x7fdf2c001b40).ready entity=mon.1 client_cookie=452b8d7947fd1d86 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:53.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf3affd700 1 -- 172.21.15.203:0/139653362 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c00c9c0 con 0x7fdf28003da0 2021-01-06T22:08:53.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf3affd700 1 -- 172.21.15.203:0/139653362 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c00cb20 con 0x7fdf28003da0 2021-01-06T22:08:53.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf3affd700 1 -- 172.21.15.203:0/139653362 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c012400 con 0x7fdf28003da0 2021-01-06T22:08:53.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf412e0700 1 -- 172.21.15.203:0/139653362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf28003da0 msgr2=0x7fdf280044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:53.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf412e0700 1 --2- 172.21.15.203:0/139653362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf28003da0 0x7fdf280044d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fdf2c007c20 tx=0x7fdf2c001b40).stop 2021-01-06T22:08:53.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf40adf700 1 -- 172.21.15.203:0/139653362 reap_dead start 2021-01-06T22:08:53.021 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf412e0700 1 -- 172.21.15.203:0/139653362 shutdown_connections 2021-01-06T22:08:53.021 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf412e0700 1 -- 172.21.15.203:0/139653362 >> 172.21.15.203:0/139653362 conn(0x7fdf3c0667b0 msgr2=0x7fdf3c0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:53.021 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf412e0700 1 -- 172.21.15.203:0/139653362 shutdown_connections 2021-01-06T22:08:53.021 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.014+0000 7fdf412e0700 1 -- 172.21.15.203:0/139653362 wait complete. 2021-01-06T22:08:53.021 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf412e0700 1 Processor -- start 2021-01-06T22:08:53.022 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf412e0700 1 -- start start 2021-01-06T22:08:53.022 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf412e0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c0b6ab0 0x7fdf3c0b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:53.022 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf412e0700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf3c0b6e90 0x7fdf3c0b7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:53.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf412e0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf3c0b91e0 0x7fdf3c0b9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:53.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf412e0700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c067dd0 con 0x7fdf3c0b6e90 2021-01-06T22:08:53.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf412e0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0b5f20 con 0x7fdf3c0b6ab0 2021-01-06T22:08:53.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf412e0700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0b9b50 con 0x7fdf3c0b91e0 2021-01-06T22:08:53.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf3bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c0b6ab0 0x7fdf3c0b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:53.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf3bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c0b6ab0 0x7fdf3c0b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50666/0 (socket says 172.21.15.203:50666) 2021-01-06T22:08:53.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf3bfff700 1 -- 172.21.15.203:0/3794089125 learned_addr learned my addr 172.21.15.203:0/3794089125 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:53.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf3b7fe700 1 --2- 172.21.15.203:0/3794089125 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf3c0b6e90 0x7fdf3c0b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:53.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf40adf700 1 --2- 172.21.15.203:0/3794089125 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf3c0b91e0 0x7fdf3c0b9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:53.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf3bfff700 1 -- 172.21.15.203:0/3794089125 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf3c0b91e0 msgr2=0x7fdf3c0b9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:53.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf3bfff700 1 --2- 172.21.15.203:0/3794089125 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf3c0b91e0 0x7fdf3c0b9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:53.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf3bfff700 1 -- 172.21.15.203:0/3794089125 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf3c0b6e90 msgr2=0x7fdf3c0b7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:53.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf3bfff700 1 --2- 172.21.15.203:0/3794089125 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf3c0b6e90 0x7fdf3c0b7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:53.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf3bfff700 1 -- 172.21.15.203:0/3794089125 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c159d90 con 0x7fdf3c0b6ab0 2021-01-06T22:08:53.026 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf3bfff700 1 --2- 172.21.15.203:0/3794089125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c0b6ab0 0x7fdf3c0b46a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fdf30004bc0 tx=0x7fdf3c0b7f00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:53.026 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf23fff700 1 -- 172.21.15.203:0/3794089125 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3000f9e0 con 0x7fdf3c0b6ab0 2021-01-06T22:08:53.026 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf23fff700 1 -- 172.21.15.203:0/3794089125 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3000fb40 con 0x7fdf3c0b6ab0 2021-01-06T22:08:53.026 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf412e0700 1 -- 172.21.15.203:0/3794089125 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3c15a030 con 0x7fdf3c0b6ab0 2021-01-06T22:08:53.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf23fff700 1 -- 172.21.15.203:0/3794089125 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3000fee0 con 0x7fdf3c0b6ab0 2021-01-06T22:08:53.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.018+0000 7fdf412e0700 1 -- 172.21.15.203:0/3794089125 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3c0b63f0 con 0x7fdf3c0b6ab0 2021-01-06T22:08:53.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.022+0000 7fdf412e0700 1 -- 172.21.15.203:0/3794089125 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3c0450a0 con 0x7fdf3c0b6ab0 2021-01-06T22:08:53.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.022+0000 7fdf23fff700 1 -- 172.21.15.203:0/3794089125 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fdf3001b030 con 0x7fdf3c0b6ab0 2021-01-06T22:08:53.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.022+0000 7fdf23fff700 1 --2- 172.21.15.203:0/3794089125 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdf1c056400 0x7fdf1c0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:53.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.022+0000 7fdf23fff700 1 -- 172.21.15.203:0/3794089125 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fdf3006f390 con 0x7fdf3c0b6ab0 2021-01-06T22:08:53.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.022+0000 7fdf3b7fe700 1 --2- 172.21.15.203:0/3794089125 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdf1c056400 0x7fdf1c0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:53.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.022+0000 7fdf3b7fe700 1 --2- 172.21.15.203:0/3794089125 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdf1c056400 0x7fdf1c0588c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fdf3c05e2c0 tx=0x7fdf2c017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:53.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.026+0000 7fdf23fff700 1 -- 172.21.15.203:0/3794089125 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fdf30044b90 con 0x7fdf3c0b6ab0 2021-01-06T22:08:53.234 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.237+0000 7fdf412e0700 1 -- 172.21.15.203:0/3794089125 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf3c05ca60 con 0x7fdf3c0b6ab0 2021-01-06T22:08:53.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.237+0000 7fdf23fff700 1 -- 172.21.15.203:0/3794089125 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3209 (secure 0 0 0) 0x7fdf30019030 con 0x7fdf3c0b6ab0 2021-01-06T22:08:53.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.241+0000 7fdf412e0700 1 -- 172.21.15.203:0/3794089125 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdf1c056400 msgr2=0x7fdf1c0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:53.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.241+0000 7fdf412e0700 1 --2- 172.21.15.203:0/3794089125 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdf1c056400 0x7fdf1c0588c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fdf3c05e2c0 tx=0x7fdf2c017000).stop 2021-01-06T22:08:53.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.241+0000 7fdf412e0700 1 -- 172.21.15.203:0/3794089125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c0b6ab0 msgr2=0x7fdf3c0b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:53.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.241+0000 7fdf412e0700 1 --2- 172.21.15.203:0/3794089125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c0b6ab0 0x7fdf3c0b46a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fdf30004bc0 tx=0x7fdf3c0b7f00).stop 2021-01-06T22:08:53.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.241+0000 7fdf412e0700 1 -- 172.21.15.203:0/3794089125 shutdown_connections 2021-01-06T22:08:53.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.241+0000 7fdf412e0700 1 --2- 172.21.15.203:0/3794089125 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdf1c056400 0x7fdf1c0588c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:53.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.241+0000 7fdf412e0700 1 --2- 172.21.15.203:0/3794089125 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf3c0b91e0 0x7fdf3c0b9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:53.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.241+0000 7fdf412e0700 1 --2- 172.21.15.203:0/3794089125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c0b6ab0 0x7fdf3c0b46a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:53.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.241+0000 7fdf412e0700 1 --2- 172.21.15.203:0/3794089125 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf3c0b6e90 0x7fdf3c0b7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:53.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.241+0000 7fdf412e0700 1 -- 172.21.15.203:0/3794089125 >> 172.21.15.203:0/3794089125 conn(0x7fdf3c0667b0 msgr2=0x7fdf3c14cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:53.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.241+0000 7fdf412e0700 1 -- 172.21.15.203:0/3794089125 shutdown_connections 2021-01-06T22:08:53.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:53.241+0000 7fdf412e0700 1 -- 172.21.15.203:0/3794089125 wait complete. 2021-01-06T22:08:53.336 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 34 2021-01-06T22:08:53.336 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:53.336 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:52.414430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:resolve","state_seq":6,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:53.337 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 34, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 34, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_4274": { "addr": "172.21.15.91:6837/3888667247", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6835", "nonce": 3888667247, "type": "v2" }, { "addr": "172.21.15.91:6837", "nonce": 3888667247, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4274, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 2 }, "gid_5068": { "addr": "172.21.15.203:6835/1477297469", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6834", "nonce": 1477297469, "type": "v2" }, { "addr": "172.21.15.203:6835", "nonce": 1477297469, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5068, "incarnation": 33, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:resolve", "state_seq": 6 } }, "last_failure": 0, "last_failure_osd_epoch": 26, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2021-01-06T22:08:52.414430+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 4274, "mds_1": 5068 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.91:6836/2428882934", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6834", "nonce": 2428882934, "type": "v2" }, { "addr": "172.21.15.91:6836", "nonce": 2428882934, "type": "v1" } ] }, "epoch": 26, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4724, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2021-01-06T22:08:55.338 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:55.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.473+0000 7fa601de0700 1 Processor -- start 2021-01-06T22:08:55.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.473+0000 7fa601de0700 1 -- start start 2021-01-06T22:08:55.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.473+0000 7fa601de0700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa5fc148d70 0x7fa5fc0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:55.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.473+0000 7fa601de0700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa5fc149150 0x7fa5fc14bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:55.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.473+0000 7fa601de0700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5fc14f0a0 con 0x7fa5fc0602b0 2021-01-06T22:08:55.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.473+0000 7fa601de0700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5fc1495a0 con 0x7fa5fc149150 2021-01-06T22:08:55.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.473+0000 7fa601de0700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5fc14c3c0 con 0x7fa5fc148d70 2021-01-06T22:08:55.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.473+0000 7fa5fbfff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fa5fc0602b0 0x7fa5fc14fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49270/0 (socket says 172.21.15.203:49270) 2021-01-06T22:08:55.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.473+0000 7fa5fbfff700 1 -- 172.21.15.203:0/904265976 learned_addr learned my addr 172.21.15.203:0/904265976 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:55.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.473+0000 7fa600dde700 1 --2- 172.21.15.203:0/904265976 >> v2:172.21.15.203:3300/0 conn(0x7fa5fc148d70 0x7fa5fc0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:55.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.473+0000 7fa6015df700 1 --2- 172.21.15.203:0/904265976 >> v2:172.21.15.91:3300/0 conn(0x7fa5fc149150 0x7fa5fc14bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:55.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa600dde700 1 -- 172.21.15.203:0/904265976 >> v2:172.21.15.91:3300/0 conn(0x7fa5fc149150 msgr2=0x7fa5fc14bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:55.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa600dde700 1 --2- 172.21.15.203:0/904265976 >> v2:172.21.15.91:3300/0 conn(0x7fa5fc149150 0x7fa5fc14bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:55.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa600dde700 1 -- 172.21.15.203:0/904265976 >> v1:172.21.15.203:6789/0 conn(0x7fa5fc0602b0 legacy=0x7fa5fc14fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:55.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa600dde700 1 -- 172.21.15.203:0/904265976 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc14c660 con 0x7fa5fc148d70 2021-01-06T22:08:55.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa600dde700 1 --2- 172.21.15.203:0/904265976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5fc148d70 0x7fa5fc0c0fa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fa5fc061ee0 tx=0x7fa5f0006de0).ready entity=mon.1 client_cookie=7e6e28c56ab6a385 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:55.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa5fb7fe700 1 -- 172.21.15.203:0/904265976 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0009750 con 0x7fa5fc148d70 2021-01-06T22:08:55.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa5fb7fe700 1 -- 172.21.15.203:0/904265976 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f00098b0 con 0x7fa5fc148d70 2021-01-06T22:08:55.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa5fb7fe700 1 -- 172.21.15.203:0/904265976 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0009c30 con 0x7fa5fc148d70 2021-01-06T22:08:55.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa601de0700 1 -- 172.21.15.203:0/904265976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5fc148d70 msgr2=0x7fa5fc0c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:55.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa601de0700 1 --2- 172.21.15.203:0/904265976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5fc148d70 0x7fa5fc0c0fa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fa5fc061ee0 tx=0x7fa5f0006de0).stop 2021-01-06T22:08:55.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa601de0700 1 -- 172.21.15.203:0/904265976 shutdown_connections 2021-01-06T22:08:55.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa601de0700 1 --2- 172.21.15.203:0/904265976 >> v2:172.21.15.91:3300/0 conn(0x7fa5fc149150 0x7fa5fc14bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:55.480 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa601de0700 1 --2- 172.21.15.203:0/904265976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5fc148d70 0x7fa5fc0c0fa0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:55.480 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa601de0700 1 -- 172.21.15.203:0/904265976 >> 172.21.15.203:0/904265976 conn(0x7fa5fc0667b0 msgr2=0x7fa5fc0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:55.480 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa601de0700 1 -- 172.21.15.203:0/904265976 shutdown_connections 2021-01-06T22:08:55.480 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa601de0700 1 -- 172.21.15.203:0/904265976 wait complete. 2021-01-06T22:08:55.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa601de0700 1 Processor -- start 2021-01-06T22:08:55.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.477+0000 7fa601de0700 1 -- start start 2021-01-06T22:08:55.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa601de0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5fc0602b0 0x7fa5fc15f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:55.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa601de0700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5fc148d70 0x7fa5fc15be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:55.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa601de0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5fc149150 0x7fa5fc151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:55.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa601de0700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc0b5550 con 0x7fa5fc148d70 2021-01-06T22:08:55.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa601de0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc067370 con 0x7fa5fc0602b0 2021-01-06T22:08:55.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa601de0700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc0b5780 con 0x7fa5fc149150 2021-01-06T22:08:55.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa600dde700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5fc0602b0 0x7fa5fc15f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:55.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa600dde700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5fc0602b0 0x7fa5fc15f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50682/0 (socket says 172.21.15.203:50682) 2021-01-06T22:08:55.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa600dde700 1 -- 172.21.15.203:0/3986264232 learned_addr learned my addr 172.21.15.203:0/3986264232 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:55.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa5fbfff700 1 --2- 172.21.15.203:0/3986264232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5fc148d70 0x7fa5fc15be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:55.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa6015df700 1 --2- 172.21.15.203:0/3986264232 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5fc149150 0x7fa5fc151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:55.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa600dde700 1 -- 172.21.15.203:0/3986264232 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5fc149150 msgr2=0x7fa5fc151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:55.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa600dde700 1 --2- 172.21.15.203:0/3986264232 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5fc149150 0x7fa5fc151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:55.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa600dde700 1 -- 172.21.15.203:0/3986264232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5fc148d70 msgr2=0x7fa5fc15be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:55.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa600dde700 1 --2- 172.21.15.203:0/3986264232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5fc148d70 0x7fa5fc15be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:55.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa600dde700 1 -- 172.21.15.203:0/3986264232 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc0b5a20 con 0x7fa5fc0602b0 2021-01-06T22:08:55.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa6015df700 1 --2- 172.21.15.203:0/3986264232 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5fc149150 0x7fa5fc151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:08:55.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa600dde700 1 --2- 172.21.15.203:0/3986264232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5fc0602b0 0x7fa5fc15f590 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fa5fc14c500 tx=0x7fa5f0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:55.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa5f8ff9700 1 -- 172.21.15.203:0/3986264232 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f001c420 con 0x7fa5fc0602b0 2021-01-06T22:08:55.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa5f8ff9700 1 -- 172.21.15.203:0/3986264232 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f001c580 con 0x7fa5fc0602b0 2021-01-06T22:08:55.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa5f8ff9700 1 -- 172.21.15.203:0/3986264232 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f001c900 con 0x7fa5fc0602b0 2021-01-06T22:08:55.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa601de0700 1 -- 172.21.15.203:0/3986264232 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5fc0b5cc0 con 0x7fa5fc0602b0 2021-01-06T22:08:55.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa601de0700 1 -- 172.21.15.203:0/3986264232 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5fc15b9e0 con 0x7fa5fc0602b0 2021-01-06T22:08:55.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.481+0000 7fa601de0700 1 -- 172.21.15.203:0/3986264232 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5fc15bba0 con 0x7fa5fc0602b0 2021-01-06T22:08:55.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.485+0000 7fa5f8ff9700 1 -- 172.21.15.203:0/3986264232 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa5f000f070 con 0x7fa5fc0602b0 2021-01-06T22:08:55.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.485+0000 7fa5f8ff9700 1 --2- 172.21.15.203:0/3986264232 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa5dc0563d0 0x7fa5dc058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:55.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.485+0000 7fa5f8ff9700 1 -- 172.21.15.203:0/3986264232 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fa5f00710d0 con 0x7fa5fc0602b0 2021-01-06T22:08:55.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.485+0000 7fa5fbfff700 1 --2- 172.21.15.203:0/3986264232 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa5dc0563d0 0x7fa5dc058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:55.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.485+0000 7fa5fbfff700 1 --2- 172.21.15.203:0/3986264232 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa5dc0563d0 0x7fa5dc058890 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fa5fc068320 tx=0x7fa5e8007b70).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:55.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.489+0000 7fa5f8ff9700 1 -- 172.21.15.203:0/3986264232 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa5f0046740 con 0x7fa5fc0602b0 2021-01-06T22:08:55.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.713+0000 7fa601de0700 1 -- 172.21.15.203:0/3986264232 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5fc057340 con 0x7fa5fc0602b0 2021-01-06T22:08:55.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.713+0000 7fa5f8ff9700 1 -- 172.21.15.203:0/3986264232 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3208 (secure 0 0 0) 0x7fa5f0049d60 con 0x7fa5fc0602b0 2021-01-06T22:08:55.712 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:55.712 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:55.422280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:55.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.717+0000 7fa601de0700 1 -- 172.21.15.203:0/3986264232 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa5dc0563d0 msgr2=0x7fa5dc058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:55.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.717+0000 7fa601de0700 1 --2- 172.21.15.203:0/3986264232 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa5dc0563d0 0x7fa5dc058890 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fa5fc068320 tx=0x7fa5e8007b70).stop 2021-01-06T22:08:55.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.717+0000 7fa601de0700 1 -- 172.21.15.203:0/3986264232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5fc0602b0 msgr2=0x7fa5fc15f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:55.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.717+0000 7fa601de0700 1 --2- 172.21.15.203:0/3986264232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5fc0602b0 0x7fa5fc15f590 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fa5fc14c500 tx=0x7fa5f0016000).stop 2021-01-06T22:08:55.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.717+0000 7fa601de0700 1 -- 172.21.15.203:0/3986264232 shutdown_connections 2021-01-06T22:08:55.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.717+0000 7fa601de0700 1 --2- 172.21.15.203:0/3986264232 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa5dc0563d0 0x7fa5dc058890 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:55.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.717+0000 7fa601de0700 1 --2- 172.21.15.203:0/3986264232 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5fc149150 0x7fa5fc151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:55.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.717+0000 7fa601de0700 1 --2- 172.21.15.203:0/3986264232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5fc0602b0 0x7fa5fc15f590 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:55.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.717+0000 7fa601de0700 1 --2- 172.21.15.203:0/3986264232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5fc148d70 0x7fa5fc15be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:55.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.717+0000 7fa601de0700 1 -- 172.21.15.203:0/3986264232 >> 172.21.15.203:0/3986264232 conn(0x7fa5fc0667b0 msgr2=0x7fa5fc0bf200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:55.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.717+0000 7fa601de0700 1 -- 172.21.15.203:0/3986264232 shutdown_connections 2021-01-06T22:08:55.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:55.717+0000 7fa601de0700 1 -- 172.21.15.203:0/3986264232 wait complete. 2021-01-06T22:08:55.719 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 37 2021-01-06T22:08:55.728 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=1, replacing gid=4273 2021-01-06T22:08:55.728 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.a 2021-01-06T22:08:57.730 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2021-01-06T22:08:57.730 INFO:tasks.ceph.mds.a:Restarting daemon 2021-01-06T22:08:57.731 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-01-06T22:08:57.733 INFO:tasks.ceph.mds.a:Started 2021-01-06T22:08:57.734 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:08:57.814 INFO:tasks.ceph.mds.a.smithi203.stdout:starting mds.a at 2021-01-06T22:08:57.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.857+0000 7fe88c923700 1 Processor -- start 2021-01-06T22:08:57.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.857+0000 7fe88c923700 1 -- start start 2021-01-06T22:08:57.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.857+0000 7fe88c923700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe888063010 0x7fe88814e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:57.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.857+0000 7fe88c923700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe8880633f0 0x7fe888064b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:57.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.857+0000 7fe88c923700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8880be990 con 0x7fe8880be1b0 2021-01-06T22:08:57.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.857+0000 7fe88c923700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe88814ec60 con 0x7fe888063010 2021-01-06T22:08:57.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.857+0000 7fe88c923700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8880be6f0 con 0x7fe8880633f0 2021-01-06T22:08:57.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.857+0000 7fe887fff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fe8880be1b0 0x7fe888147790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49300/0 (socket says 172.21.15.203:49300) 2021-01-06T22:08:57.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.857+0000 7fe887fff700 1 -- 172.21.15.203:0/2786929780 learned_addr learned my addr 172.21.15.203:0/2786929780 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:08:57.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.857+0000 7fe8877fe700 1 --2- 172.21.15.203:0/2786929780 >> v2:172.21.15.91:3300/0 conn(0x7fe888063010 0x7fe88814e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:57.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.857+0000 7fe886ffd700 1 --2- 172.21.15.203:0/2786929780 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe8880633f0 0x7fe888064b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:57.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe8867fc700 1 -- 172.21.15.203:0/2786929780 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652375593 0 0) 0x7fe8880be990 con 0x7fe8880be1b0 2021-01-06T22:08:57.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe8867fc700 1 -- 172.21.15.203:0/2786929780 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe874003230 con 0x7fe8880be1b0 2021-01-06T22:08:57.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe8877fe700 1 -- 172.21.15.203:0/2786929780 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe8880633f0 msgr2=0x7fe888064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:08:57.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe8877fe700 1 --2- 172.21.15.203:0/2786929780 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe8880633f0 0x7fe888064b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:57.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe8877fe700 1 -- 172.21.15.203:0/2786929780 >> v1:172.21.15.203:6789/0 conn(0x7fe8880be1b0 legacy=0x7fe888147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:57.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe8877fe700 1 -- 172.21.15.203:0/2786929780 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8880c1050 con 0x7fe888063010 2021-01-06T22:08:57.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe8877fe700 1 --2- 172.21.15.203:0/2786929780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe888063010 0x7fe88814e770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fe87800b980 tx=0x7fe878006de0).ready entity=mon.0 client_cookie=1f29f9f17d0935ed server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:57.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe8867fc700 1 -- 172.21.15.203:0/2786929780 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe878009640 con 0x7fe888063010 2021-01-06T22:08:57.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe8867fc700 1 -- 172.21.15.203:0/2786929780 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8780097a0 con 0x7fe888063010 2021-01-06T22:08:57.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe8867fc700 1 -- 172.21.15.203:0/2786929780 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe878009b20 con 0x7fe888063010 2021-01-06T22:08:57.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe88c923700 1 -- 172.21.15.203:0/2786929780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe888063010 msgr2=0x7fe88814e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:57.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe88c923700 1 --2- 172.21.15.203:0/2786929780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe888063010 0x7fe88814e770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fe87800b980 tx=0x7fe878006de0).stop 2021-01-06T22:08:57.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe88c923700 1 -- 172.21.15.203:0/2786929780 shutdown_connections 2021-01-06T22:08:57.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe88c923700 1 --2- 172.21.15.203:0/2786929780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe888063010 0x7fe88814e770 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:57.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe88c923700 1 --2- 172.21.15.203:0/2786929780 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe8880633f0 0x7fe888064b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:57.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe88c923700 1 -- 172.21.15.203:0/2786929780 >> 172.21.15.203:0/2786929780 conn(0x7fe888067e20 msgr2=0x7fe888068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:57.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe88c923700 1 -- 172.21.15.203:0/2786929780 shutdown_connections 2021-01-06T22:08:57.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.861+0000 7fe88c923700 1 -- 172.21.15.203:0/2786929780 wait complete. 2021-01-06T22:08:57.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe88c923700 1 Processor -- start 2021-01-06T22:08:57.878 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe88c923700 1 -- start start 2021-01-06T22:08:57.878 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe88c923700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe888063010 0x7fe888160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:57.879 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe88c923700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8880633f0 0x7fe88815f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:57.879 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe88c923700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8880be1b0 0x7fe88815f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:57.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe88c923700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88815fcc0 con 0x7fe8880be1b0 2021-01-06T22:08:57.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe88c923700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8881614b0 con 0x7fe8880633f0 2021-01-06T22:08:57.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe88c923700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe888161750 con 0x7fe888063010 2021-01-06T22:08:57.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe886ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8880633f0 0x7fe88815f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:57.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe886ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8880633f0 0x7fe88815f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50710/0 (socket says 172.21.15.203:50710) 2021-01-06T22:08:57.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe886ffd700 1 -- 172.21.15.203:0/38124797 learned_addr learned my addr 172.21.15.203:0/38124797 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:08:57.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe8877fe700 1 --2- 172.21.15.203:0/38124797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe888063010 0x7fe888160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:57.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe887fff700 1 --2- 172.21.15.203:0/38124797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8880be1b0 0x7fe88815f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:57.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe886ffd700 1 -- 172.21.15.203:0/38124797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe888063010 msgr2=0x7fe888160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:57.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe886ffd700 1 --2- 172.21.15.203:0/38124797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe888063010 0x7fe888160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:57.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe886ffd700 1 -- 172.21.15.203:0/38124797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8880be1b0 msgr2=0x7fe88815f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:57.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe886ffd700 1 --2- 172.21.15.203:0/38124797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8880be1b0 0x7fe88815f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:57.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe886ffd700 1 -- 172.21.15.203:0/38124797 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88814d1f0 con 0x7fe8880633f0 2021-01-06T22:08:57.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe886ffd700 1 --2- 172.21.15.203:0/38124797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8880633f0 0x7fe88815f2e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7fe87c00b220 tx=0x7fe87c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:57.886 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe86f7fe700 1 -- 172.21.15.203:0/38124797 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c010440 con 0x7fe8880633f0 2021-01-06T22:08:57.886 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe86f7fe700 1 -- 172.21.15.203:0/38124797 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87c0105a0 con 0x7fe8880633f0 2021-01-06T22:08:57.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe88c923700 1 -- 172.21.15.203:0/38124797 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88814d490 con 0x7fe8880633f0 2021-01-06T22:08:57.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe86f7fe700 1 -- 172.21.15.203:0/38124797 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c010920 con 0x7fe8880633f0 2021-01-06T22:08:57.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.865+0000 7fe88c923700 1 -- 172.21.15.203:0/38124797 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88814b960 con 0x7fe8880633f0 2021-01-06T22:08:57.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.869+0000 7fe88c923700 1 -- 172.21.15.203:0/38124797 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88814d930 con 0x7fe8880633f0 2021-01-06T22:08:57.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.869+0000 7fe86f7fe700 1 -- 172.21.15.203:0/38124797 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fe87c01f070 con 0x7fe8880633f0 2021-01-06T22:08:57.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.869+0000 7fe86f7fe700 1 --2- 172.21.15.203:0/38124797 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe8680563d0 0x7fe868058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:08:57.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.869+0000 7fe86f7fe700 1 -- 172.21.15.203:0/38124797 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fe87c06fc60 con 0x7fe8880633f0 2021-01-06T22:08:57.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.869+0000 7fe8877fe700 1 --2- 172.21.15.203:0/38124797 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe8680563d0 0x7fe868058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:08:57.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.869+0000 7fe8877fe700 1 --2- 172.21.15.203:0/38124797 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe8680563d0 0x7fe868058890 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fe87800b130 tx=0x7fe878016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:08:57.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:57.876+0000 7fe86f7fe700 1 -- 172.21.15.203:0/38124797 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fe87c0453e0 con 0x7fe8880633f0 2021-01-06T22:08:58.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:58.104+0000 7fe88c923700 1 -- 172.21.15.203:0/38124797 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8880450a0 con 0x7fe8880633f0 2021-01-06T22:08:58.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:58.104+0000 7fe86f7fe700 1 -- 172.21.15.203:0/38124797 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3208 (secure 0 0 0) 0x7fe87c048a00 con 0x7fe8880633f0 2021-01-06T22:08:58.105 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:08:58.105 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:55.422280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:08:58.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:58.108+0000 7fe88c923700 1 -- 172.21.15.203:0/38124797 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe8680563d0 msgr2=0x7fe868058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:58.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:58.108+0000 7fe88c923700 1 --2- 172.21.15.203:0/38124797 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe8680563d0 0x7fe868058890 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fe87800b130 tx=0x7fe878016000).stop 2021-01-06T22:08:58.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:58.108+0000 7fe88c923700 1 -- 172.21.15.203:0/38124797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8880633f0 msgr2=0x7fe88815f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:08:58.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:58.108+0000 7fe88c923700 1 --2- 172.21.15.203:0/38124797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8880633f0 0x7fe88815f2e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7fe87c00b220 tx=0x7fe87c008040).stop 2021-01-06T22:08:58.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:58.108+0000 7fe88c923700 1 -- 172.21.15.203:0/38124797 shutdown_connections 2021-01-06T22:08:58.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:58.108+0000 7fe88c923700 1 --2- 172.21.15.203:0/38124797 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe8680563d0 0x7fe868058890 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:58.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:58.108+0000 7fe88c923700 1 --2- 172.21.15.203:0/38124797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8880be1b0 0x7fe88815f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:58.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:58.108+0000 7fe88c923700 1 --2- 172.21.15.203:0/38124797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe888063010 0x7fe888160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:58.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:58.108+0000 7fe88c923700 1 --2- 172.21.15.203:0/38124797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8880633f0 0x7fe88815f2e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:08:58.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:58.108+0000 7fe88c923700 1 -- 172.21.15.203:0/38124797 >> 172.21.15.203:0/38124797 conn(0x7fe888067e20 msgr2=0x7fe8880bc350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:08:58.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:58.108+0000 7fe88c923700 1 -- 172.21.15.203:0/38124797 shutdown_connections 2021-01-06T22:08:58.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:08:58.108+0000 7fe88c923700 1 -- 172.21.15.203:0/38124797 wait complete. 2021-01-06T22:08:58.110 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 37 2021-01-06T22:08:58.119 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2021-01-06T22:09:00.120 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:09:00.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.252+0000 7f7dfb59e700 1 Processor -- start 2021-01-06T22:09:00.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.252+0000 7f7dfb59e700 1 -- start start 2021-01-06T22:09:00.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.252+0000 7f7dfb59e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7dfc147e00 0x7f7dfc068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:00.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.252+0000 7f7dfb59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dfc0b9310 con 0x7f7dfc1481e0 2021-01-06T22:09:00.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.252+0000 7f7dfb59e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dfc057340 con 0x7f7dfc067e60 2021-01-06T22:09:00.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.252+0000 7f7dfb59e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dfc0ba400 con 0x7f7dfc147e00 2021-01-06T22:09:00.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.252+0000 7f7dfad9d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f7dfc067e60 0x7f7dfc14df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49324/0 (socket says 172.21.15.203:49324) 2021-01-06T22:09:00.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.252+0000 7f7dfad9d700 1 -- 172.21.15.203:0/1891404835 learned_addr learned my addr 172.21.15.203:0/1891404835 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:09:00.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfa59c700 1 --2- 172.21.15.203:0/1891404835 >> v2:172.21.15.91:3300/0 conn(0x7f7dfc147e00 0x7f7dfc068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:00.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 -- 172.21.15.203:0/1891404835 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147264193 0 0) 0x7f7dfc057340 con 0x7f7dfc067e60 2021-01-06T22:09:00.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 -- 172.21.15.203:0/1891404835 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7de0003230 con 0x7f7dfc067e60 2021-01-06T22:09:00.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 -- 172.21.15.203:0/1891404835 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2735933207 0 0) 0x7f7de0003230 con 0x7f7dfc067e60 2021-01-06T22:09:00.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 -- 172.21.15.203:0/1891404835 >> v2:172.21.15.91:3300/0 conn(0x7f7dfc147e00 msgr2=0x7f7dfc068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:09:00.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 --2- 172.21.15.203:0/1891404835 >> v2:172.21.15.91:3300/0 conn(0x7f7dfc147e00 0x7f7dfc068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:00.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 -- 172.21.15.203:0/1891404835 >> v1:172.21.15.91:6789/0 conn(0x7f7dfc1481e0 legacy=0x7f7dfc067750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:00.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 -- 172.21.15.203:0/1891404835 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dfc0b8b30 con 0x7f7dfc067e60 2021-01-06T22:09:00.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 -- 172.21.15.203:0/1891404835 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f7df4001860 con 0x7f7dfc067e60 2021-01-06T22:09:00.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 -- 172.21.15.203:0/1891404835 >> v1:172.21.15.203:6789/0 conn(0x7f7dfc067e60 legacy=0x7f7dfc14df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:00.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 --2- 172.21.15.203:0/1891404835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7de0003720 0x7f7de0003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:00.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 --2- 172.21.15.203:0/1891404835 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7de00044d0 0x7f7de0004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:00.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 --2- 172.21.15.203:0/1891404835 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7de0004e40 0x7f7de000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:00.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 -- 172.21.15.203:0/1891404835 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc0ba400 con 0x7f7de0004e40 2021-01-06T22:09:00.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 -- 172.21.15.203:0/1891404835 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de00041c0 con 0x7f7de0003720 2021-01-06T22:09:00.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 -- 172.21.15.203:0/1891404835 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de000bc10 con 0x7f7de00044d0 2021-01-06T22:09:00.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfa59c700 1 --2- 172.21.15.203:0/1891404835 >> v2:172.21.15.91:3300/0 conn(0x7f7dfc147e00 0x7f7dfc068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:09:00.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfa59c700 1 --2- 172.21.15.203:0/1891404835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7de0003720 0x7f7de0003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:00.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfad9d700 1 --2- 172.21.15.203:0/1891404835 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7de0004e40 0x7f7de000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:00.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df9d9b700 1 --2- 172.21.15.203:0/1891404835 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7de00044d0 0x7f7de0004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:00.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfa59c700 1 -- 172.21.15.203:0/1891404835 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7de00044d0 msgr2=0x7f7de0004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:09:00.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfa59c700 1 --2- 172.21.15.203:0/1891404835 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7de00044d0 0x7f7de0004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:00.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfa59c700 1 -- 172.21.15.203:0/1891404835 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7de0004e40 msgr2=0x7f7de000b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:09:00.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfa59c700 1 --2- 172.21.15.203:0/1891404835 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7de0004e40 0x7f7de000b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:00.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfa59c700 1 -- 172.21.15.203:0/1891404835 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de0003230 con 0x7f7de0003720 2021-01-06T22:09:00.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfad9d700 1 -- 172.21.15.203:0/1891404835 reap_dead start 2021-01-06T22:09:00.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfa59c700 1 --2- 172.21.15.203:0/1891404835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7de0003720 0x7f7de0003b00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f7dec00bdf0 tx=0x7f7dec008040).ready entity=mon.1 client_cookie=94337739ace546ae server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:09:00.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 -- 172.21.15.203:0/1891404835 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec00b4f0 con 0x7f7de0003720 2021-01-06T22:09:00.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 -- 172.21.15.203:0/1891404835 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dec00ec40 con 0x7f7de0003720 2021-01-06T22:09:00.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7df959a700 1 -- 172.21.15.203:0/1891404835 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec01a470 con 0x7f7de0003720 2021-01-06T22:09:00.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfb59e700 1 -- 172.21.15.203:0/1891404835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7de0003720 msgr2=0x7f7de0003b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:00.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfb59e700 1 --2- 172.21.15.203:0/1891404835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7de0003720 0x7f7de0003b00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f7dec00bdf0 tx=0x7f7dec008040).stop 2021-01-06T22:09:00.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfb59e700 1 -- 172.21.15.203:0/1891404835 shutdown_connections 2021-01-06T22:09:00.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfb59e700 1 --2- 172.21.15.203:0/1891404835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7de0003720 0x7f7de0003b00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:00.267 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfb59e700 1 -- 172.21.15.203:0/1891404835 >> 172.21.15.203:0/1891404835 conn(0x7f7dfc0bd3d0 msgr2=0x7f7dfc0bd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:09:00.267 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfb59e700 1 -- 172.21.15.203:0/1891404835 shutdown_connections 2021-01-06T22:09:00.267 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.256+0000 7f7dfb59e700 1 -- 172.21.15.203:0/1891404835 wait complete. 2021-01-06T22:09:00.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfb59e700 1 Processor -- start 2021-01-06T22:09:00.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfb59e700 1 -- start start 2021-01-06T22:09:00.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfb59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7de0003720 0x7f7dfc144020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:00.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfb59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dfc15cfb0 0x7f7dfc144510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:00.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfb59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc141fb0 0x7f7dfc13d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:00.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfb59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc144a00 con 0x7f7dfc15cfb0 2021-01-06T22:09:00.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfb59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc0b4830 con 0x7f7dfc141fb0 2021-01-06T22:09:00.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfb59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc0b4ad0 con 0x7f7de0003720 2021-01-06T22:09:00.272 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc141fb0 0x7f7dfc13d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:00.272 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc141fb0 0x7f7dfc13d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50742/0 (socket says 172.21.15.203:50742) 2021-01-06T22:09:00.272 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfad9d700 1 -- 172.21.15.203:0/3140010659 learned_addr learned my addr 172.21.15.203:0/3140010659 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:09:00.272 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfa59c700 1 --2- 172.21.15.203:0/3140010659 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7de0003720 0x7f7dfc144020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:00.273 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7df9d9b700 1 --2- 172.21.15.203:0/3140010659 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dfc15cfb0 0x7f7dfc144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:00.273 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfad9d700 1 -- 172.21.15.203:0/3140010659 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7de0003720 msgr2=0x7f7dfc144020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:00.273 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfad9d700 1 --2- 172.21.15.203:0/3140010659 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7de0003720 0x7f7dfc144020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:00.273 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfad9d700 1 -- 172.21.15.203:0/3140010659 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dfc15cfb0 msgr2=0x7f7dfc144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:00.274 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfad9d700 1 --2- 172.21.15.203:0/3140010659 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dfc15cfb0 0x7f7dfc144510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:00.274 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.260+0000 7f7dfad9d700 1 -- 172.21.15.203:0/3140010659 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dfc143640 con 0x7f7dfc141fb0 2021-01-06T22:09:00.274 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.264+0000 7f7dfad9d700 1 --2- 172.21.15.203:0/3140010659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc141fb0 0x7f7dfc13d120 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f7dfc05c7e0 tx=0x7f7dfc0671a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:09:00.274 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.264+0000 7f7dea7fc700 1 -- 172.21.15.203:0/3140010659 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df400fb80 con 0x7f7dfc141fb0 2021-01-06T22:09:00.274 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.264+0000 7f7dea7fc700 1 -- 172.21.15.203:0/3140010659 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df400fce0 con 0x7f7dfc141fb0 2021-01-06T22:09:00.275 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.264+0000 7f7dfb59e700 1 -- 172.21.15.203:0/3140010659 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dfc1438e0 con 0x7f7dfc141fb0 2021-01-06T22:09:00.275 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.264+0000 7f7dea7fc700 1 -- 172.21.15.203:0/3140010659 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df4012580 con 0x7f7dfc141fb0 2021-01-06T22:09:00.276 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.264+0000 7f7dfb59e700 1 -- 172.21.15.203:0/3140010659 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dfc147240 con 0x7f7dfc141fb0 2021-01-06T22:09:00.276 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.264+0000 7f7dfb59e700 1 -- 172.21.15.203:0/3140010659 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dfc0450a0 con 0x7f7dfc141fb0 2021-01-06T22:09:00.276 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.264+0000 7f7dea7fc700 1 -- 172.21.15.203:0/3140010659 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f7df401b030 con 0x7f7dfc141fb0 2021-01-06T22:09:00.276 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.264+0000 7f7dea7fc700 1 --2- 172.21.15.203:0/3140010659 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7dd80563d0 0x7f7dd8058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:00.276 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.264+0000 7f7dea7fc700 1 -- 172.21.15.203:0/3140010659 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f7df406ee90 con 0x7f7dfc141fb0 2021-01-06T22:09:00.277 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.264+0000 7f7dfa59c700 1 --2- 172.21.15.203:0/3140010659 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7dd80563d0 0x7f7dd8058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:00.277 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.264+0000 7f7dfa59c700 1 --2- 172.21.15.203:0/3140010659 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7dd80563d0 0x7f7dd8058890 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f7dec007fd0 tx=0x7f7dec01a000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:09:00.277 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.272+0000 7f7dea7fc700 1 -- 172.21.15.203:0/3140010659 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f7df4044610 con 0x7f7dfc141fb0 2021-01-06T22:09:00.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.492+0000 7f7dfb59e700 1 -- 172.21.15.203:0/3140010659 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7dfc0bcfc0 con 0x7f7dfc141fb0 2021-01-06T22:09:00.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.492+0000 7f7dea7fc700 1 -- 172.21.15.203:0/3140010659 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3208 (secure 0 0 0) 0x7f7df4019020 con 0x7f7dfc141fb0 2021-01-06T22:09:00.493 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:09:00.493 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:55.422280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:09:00.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.496+0000 7f7dfb59e700 1 -- 172.21.15.203:0/3140010659 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7dd80563d0 msgr2=0x7f7dd8058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:00.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.496+0000 7f7dfb59e700 1 --2- 172.21.15.203:0/3140010659 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7dd80563d0 0x7f7dd8058890 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f7dec007fd0 tx=0x7f7dec01a000).stop 2021-01-06T22:09:00.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.496+0000 7f7dfb59e700 1 -- 172.21.15.203:0/3140010659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc141fb0 msgr2=0x7f7dfc13d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:00.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.496+0000 7f7dfb59e700 1 --2- 172.21.15.203:0/3140010659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc141fb0 0x7f7dfc13d120 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f7dfc05c7e0 tx=0x7f7dfc0671a0).stop 2021-01-06T22:09:00.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.496+0000 7f7dfb59e700 1 -- 172.21.15.203:0/3140010659 shutdown_connections 2021-01-06T22:09:00.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.496+0000 7f7dfb59e700 1 --2- 172.21.15.203:0/3140010659 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7dd80563d0 0x7f7dd8058890 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:00.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.496+0000 7f7dfb59e700 1 --2- 172.21.15.203:0/3140010659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc141fb0 0x7f7dfc13d120 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:00.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.496+0000 7f7dfb59e700 1 --2- 172.21.15.203:0/3140010659 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7de0003720 0x7f7dfc144020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:00.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.496+0000 7f7dfb59e700 1 --2- 172.21.15.203:0/3140010659 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dfc15cfb0 0x7f7dfc144510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:00.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.496+0000 7f7dfb59e700 1 -- 172.21.15.203:0/3140010659 >> 172.21.15.203:0/3140010659 conn(0x7f7dfc0bd3d0 msgr2=0x7f7dfc0be8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:09:00.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.496+0000 7f7dfb59e700 1 -- 172.21.15.203:0/3140010659 shutdown_connections 2021-01-06T22:09:00.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:00.496+0000 7f7dfb59e700 1 -- 172.21.15.203:0/3140010659 wait complete. 2021-01-06T22:09:00.498 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 37 2021-01-06T22:09:00.508 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2021-01-06T22:09:02.509 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:09:02.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578cffc700 1 Processor -- start 2021-01-06T22:09:02.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578cffc700 1 -- start start 2021-01-06T22:09:02.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578cffc700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5788151560 0x7f57880bf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:02.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578cffc700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5788152460 con 0x7f5788064bc0 2021-01-06T22:09:02.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578cffc700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f578814e740 con 0x7f57880bf100 2021-01-06T22:09:02.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578cffc700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578814e9e0 con 0x7f5788151560 2021-01-06T22:09:02.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578659c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f57880bf100 0x7f578814df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49342/0 (socket says 172.21.15.203:49342) 2021-01-06T22:09:02.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578659c700 1 -- 172.21.15.203:0/790204731 learned_addr learned my addr 172.21.15.203:0/790204731 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:09:02.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f5786d9d700 1 --2- 172.21.15.203:0/790204731 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5788151560 0x7f57880bf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:02.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578559a700 1 -- 172.21.15.203:0/790204731 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123876596 0 0) 0x7f578814e740 con 0x7f57880bf100 2021-01-06T22:09:02.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578559a700 1 -- 172.21.15.203:0/790204731 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f576c003230 con 0x7f57880bf100 2021-01-06T22:09:02.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578559a700 1 -- 172.21.15.203:0/790204731 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3664400203 0 0) 0x7f576c003230 con 0x7f57880bf100 2021-01-06T22:09:02.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578559a700 1 -- 172.21.15.203:0/790204731 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5788151560 msgr2=0x7f57880bf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:09:02.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578559a700 1 --2- 172.21.15.203:0/790204731 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5788151560 0x7f57880bf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:02.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578559a700 1 -- 172.21.15.203:0/790204731 >> v1:172.21.15.91:6789/0 conn(0x7f5788064bc0 legacy=0x7f5788147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:02.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578559a700 1 -- 172.21.15.203:0/790204731 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578814ec80 con 0x7f57880bf100 2021-01-06T22:09:02.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578559a700 1 -- 172.21.15.203:0/790204731 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f57780014e0 con 0x7f57880bf100 2021-01-06T22:09:02.649 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578559a700 1 -- 172.21.15.203:0/790204731 >> v1:172.21.15.203:6789/0 conn(0x7f57880bf100 legacy=0x7f578814df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:02.649 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578559a700 1 --2- 172.21.15.203:0/790204731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f576c003720 0x7f576c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:02.650 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578559a700 1 --2- 172.21.15.203:0/790204731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0044d0 0x7f576c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:02.650 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578559a700 1 --2- 172.21.15.203:0/790204731 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5788151560 0x7f576c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:02.650 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578559a700 1 -- 172.21.15.203:0/790204731 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578814e9e0 con 0x7f576c003720 2021-01-06T22:09:02.651 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578559a700 1 -- 172.21.15.203:0/790204731 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c004290 con 0x7f576c0044d0 2021-01-06T22:09:02.651 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578559a700 1 -- 172.21.15.203:0/790204731 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576c00b720 con 0x7f5788151560 2021-01-06T22:09:02.652 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f5785d9b700 1 --2- 172.21.15.203:0/790204731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0044d0 0x7f576c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:02.652 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578659c700 1 --2- 172.21.15.203:0/790204731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f576c003720 0x7f576c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:02.653 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f5786d9d700 1 --2- 172.21.15.203:0/790204731 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5788151560 0x7f576c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:02.653 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f5785d9b700 1 -- 172.21.15.203:0/790204731 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5788151560 msgr2=0x7f576c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:09:02.653 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f5785d9b700 1 --2- 172.21.15.203:0/790204731 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5788151560 0x7f576c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:02.654 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f5785d9b700 1 -- 172.21.15.203:0/790204731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f576c003720 msgr2=0x7f576c003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:09:02.654 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f5785d9b700 1 --2- 172.21.15.203:0/790204731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f576c003720 0x7f576c003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:02.655 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f5785d9b700 1 -- 172.21.15.203:0/790204731 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c003230 con 0x7f576c0044d0 2021-01-06T22:09:02.655 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f5786d9d700 1 --2- 172.21.15.203:0/790204731 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5788151560 0x7f576c004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:09:02.656 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.632+0000 7f578659c700 1 --2- 172.21.15.203:0/790204731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f576c003720 0x7f576c003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:09:02.656 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f5785d9b700 1 --2- 172.21.15.203:0/790204731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0044d0 0x7f576c004950 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f5788064fc0 tx=0x7f577c007b80).ready entity=mon.1 client_cookie=f8a5f108ca5318ca server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:09:02.656 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578559a700 1 -- 172.21.15.203:0/790204731 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c007020 con 0x7f576c0044d0 2021-01-06T22:09:02.657 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578559a700 1 -- 172.21.15.203:0/790204731 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577c007180 con 0x7f576c0044d0 2021-01-06T22:09:02.657 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578559a700 1 -- 172.21.15.203:0/790204731 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c00c830 con 0x7f576c0044d0 2021-01-06T22:09:02.657 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578cffc700 1 -- 172.21.15.203:0/790204731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0044d0 msgr2=0x7f576c004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:02.657 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578cffc700 1 --2- 172.21.15.203:0/790204731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0044d0 0x7f576c004950 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f5788064fc0 tx=0x7f577c007b80).stop 2021-01-06T22:09:02.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f5786d9d700 1 -- 172.21.15.203:0/790204731 reap_dead start 2021-01-06T22:09:02.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578cffc700 1 -- 172.21.15.203:0/790204731 shutdown_connections 2021-01-06T22:09:02.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578cffc700 1 -- 172.21.15.203:0/790204731 >> 172.21.15.203:0/790204731 conn(0x7f57880602b0 msgr2=0x7f57880606c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:09:02.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578cffc700 1 -- 172.21.15.203:0/790204731 shutdown_connections 2021-01-06T22:09:02.659 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578cffc700 1 -- 172.21.15.203:0/790204731 wait complete. 2021-01-06T22:09:02.659 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578cffc700 1 Processor -- start 2021-01-06T22:09:02.659 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578cffc700 1 -- start start 2021-01-06T22:09:02.659 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578cffc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0044d0 0x7f578814bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:02.660 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578cffc700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57880b32f0 0x7f5788143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:02.660 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578cffc700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f57880b3770 0x7f5788144370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:02.660 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578cffc700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578815dc90 con 0x7f57880b32f0 2021-01-06T22:09:02.660 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578cffc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578815df30 con 0x7f576c0044d0 2021-01-06T22:09:02.661 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578cffc700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578815e1d0 con 0x7f57880b3770 2021-01-06T22:09:02.661 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0044d0 0x7f578814bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:02.661 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0044d0 0x7f578814bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50760/0 (socket says 172.21.15.203:50760) 2021-01-06T22:09:02.662 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.636+0000 7f578659c700 1 -- 172.21.15.203:0/167626797 learned_addr learned my addr 172.21.15.203:0/167626797 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:09:02.662 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f5786d9d700 1 --2- 172.21.15.203:0/167626797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f57880b3770 0x7f5788144370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:02.662 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f5785d9b700 1 --2- 172.21.15.203:0/167626797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57880b32f0 0x7f5788143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:02.662 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f578659c700 1 -- 172.21.15.203:0/167626797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f57880b3770 msgr2=0x7f5788144370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:02.663 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f578659c700 1 --2- 172.21.15.203:0/167626797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f57880b3770 0x7f5788144370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:02.663 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f578659c700 1 -- 172.21.15.203:0/167626797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57880b32f0 msgr2=0x7f5788143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:02.663 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f578659c700 1 --2- 172.21.15.203:0/167626797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57880b32f0 0x7f5788143e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:02.663 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f578659c700 1 -- 172.21.15.203:0/167626797 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578815e470 con 0x7f576c0044d0 2021-01-06T22:09:02.664 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f5785d9b700 1 --2- 172.21.15.203:0/167626797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57880b32f0 0x7f5788143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:09:02.664 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f5786d9d700 1 --2- 172.21.15.203:0/167626797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f57880b3770 0x7f5788144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:09:02.664 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f578659c700 1 --2- 172.21.15.203:0/167626797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0044d0 0x7f578814bc40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f5778005b00 tx=0x7f578814e8d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:09:02.664 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f57767fc700 1 -- 172.21.15.203:0/167626797 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577800e890 con 0x7f576c0044d0 2021-01-06T22:09:02.665 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f57767fc700 1 -- 172.21.15.203:0/167626797 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577800e9f0 con 0x7f576c0044d0 2021-01-06T22:09:02.665 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f57767fc700 1 -- 172.21.15.203:0/167626797 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577800ed70 con 0x7f576c0044d0 2021-01-06T22:09:02.665 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f578cffc700 1 -- 172.21.15.203:0/167626797 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578815e710 con 0x7f576c0044d0 2021-01-06T22:09:02.665 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f578cffc700 1 -- 172.21.15.203:0/167626797 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578815d8e0 con 0x7f576c0044d0 2021-01-06T22:09:02.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f578cffc700 1 -- 172.21.15.203:0/167626797 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f578805d600 con 0x7f576c0044d0 2021-01-06T22:09:02.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f57767fc700 1 -- 172.21.15.203:0/167626797 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f5778014640 con 0x7f576c0044d0 2021-01-06T22:09:02.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f57767fc700 1 --2- 172.21.15.203:0/167626797 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5764056400 0x7f57640588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:02.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f57767fc700 1 -- 172.21.15.203:0/167626797 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f577806f350 con 0x7f576c0044d0 2021-01-06T22:09:02.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f5785d9b700 1 --2- 172.21.15.203:0/167626797 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5764056400 0x7f57640588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:02.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.640+0000 7f5785d9b700 1 --2- 172.21.15.203:0/167626797 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5764056400 0x7f57640588c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f578814b2b0 tx=0x7f577c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:09:02.668 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.648+0000 7f57767fc700 1 -- 172.21.15.203:0/167626797 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f5778044a70 con 0x7f576c0044d0 2021-01-06T22:09:02.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.904+0000 7f578cffc700 1 -- 172.21.15.203:0/167626797 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57880612c0 con 0x7f576c0044d0 2021-01-06T22:09:02.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.904+0000 7f57767fc700 1 -- 172.21.15.203:0/167626797 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3561 (secure 0 0 0) 0x7f5778048090 con 0x7f576c0044d0 2021-01-06T22:09:02.904 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:09:02.904 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4859,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:55.422280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:09:02.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.908+0000 7f578cffc700 1 -- 172.21.15.203:0/167626797 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5764056400 msgr2=0x7f57640588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:02.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.908+0000 7f578cffc700 1 --2- 172.21.15.203:0/167626797 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5764056400 0x7f57640588c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f578814b2b0 tx=0x7f577c015000).stop 2021-01-06T22:09:02.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.908+0000 7f578cffc700 1 -- 172.21.15.203:0/167626797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0044d0 msgr2=0x7f578814bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:02.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.908+0000 7f578cffc700 1 --2- 172.21.15.203:0/167626797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0044d0 0x7f578814bc40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f5778005b00 tx=0x7f578814e8d0).stop 2021-01-06T22:09:02.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.908+0000 7f578cffc700 1 -- 172.21.15.203:0/167626797 shutdown_connections 2021-01-06T22:09:02.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.908+0000 7f578cffc700 1 --2- 172.21.15.203:0/167626797 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5764056400 0x7f57640588c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:02.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.908+0000 7f578cffc700 1 --2- 172.21.15.203:0/167626797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f57880b3770 0x7f5788144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:02.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.908+0000 7f578cffc700 1 --2- 172.21.15.203:0/167626797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0044d0 0x7f578814bc40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:02.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.908+0000 7f578cffc700 1 --2- 172.21.15.203:0/167626797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57880b32f0 0x7f5788143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:02.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.908+0000 7f578cffc700 1 -- 172.21.15.203:0/167626797 >> 172.21.15.203:0/167626797 conn(0x7f57880602b0 msgr2=0x7f57880c19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:09:02.909 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.908+0000 7f578cffc700 1 -- 172.21.15.203:0/167626797 shutdown_connections 2021-01-06T22:09:02.909 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:02.908+0000 7f578cffc700 1 -- 172.21.15.203:0/167626797 wait complete. 2021-01-06T22:09:02.909 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 38 2021-01-06T22:09:02.922 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2021-01-06T22:09:02.922 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 52 secs before thrashing 2021-01-06T22:09:12.727 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:09:12.729 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:09:42.827 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:09:42.830 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:09:54.923 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:09:55.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.054+0000 7f8ff7fff700 1 Processor -- start 2021-01-06T22:09:55.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.054+0000 7f8ff7fff700 1 -- start start 2021-01-06T22:09:55.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff7fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ff8064380 0x7f8ff80c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:55.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff7fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8ff814e310 0x7f8ff81471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:55.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff7fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8ff8064760 0x7f8ff81476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:55.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff7fff700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff8147b90 con 0x7f8ff8064760 2021-01-06T22:09:55.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff7fff700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff80bb2d0 con 0x7f8ff814e310 2021-01-06T22:09:55.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff7fff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff814a560 con 0x7f8ff8064380 2021-01-06T22:09:55.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff67fc700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8ff814e310 0x7f8ff81471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:55.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff67fc700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8ff814e310 0x7f8ff81471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50820/0 (socket says 172.21.15.203:50820) 2021-01-06T22:09:55.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff67fc700 1 -- 172.21.15.203:0/1092998169 learned_addr learned my addr 172.21.15.203:0/1092998169 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:09:55.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff6ffd700 1 --2- 172.21.15.203:0/1092998169 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ff8064380 0x7f8ff80c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:55.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff77fe700 1 --2- 172.21.15.203:0/1092998169 >> v2:172.21.15.91:3300/0 conn(0x7f8ff8064760 0x7f8ff81476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:55.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff67fc700 1 -- 172.21.15.203:0/1092998169 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ff8064380 msgr2=0x7f8ff80c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:09:55.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff67fc700 1 --2- 172.21.15.203:0/1092998169 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ff8064380 0x7f8ff80c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:55.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff67fc700 1 -- 172.21.15.203:0/1092998169 >> v2:172.21.15.91:3300/0 conn(0x7f8ff8064760 msgr2=0x7f8ff81476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:09:55.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff67fc700 1 --2- 172.21.15.203:0/1092998169 >> v2:172.21.15.91:3300/0 conn(0x7f8ff8064760 0x7f8ff81476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:55.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff67fc700 1 -- 172.21.15.203:0/1092998169 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff814a800 con 0x7f8ff814e310 2021-01-06T22:09:55.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff67fc700 1 --2- 172.21.15.203:0/1092998169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff814e310 0x7f8ff81471b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f8ff8066e10 tx=0x7f8fec006ea0).ready entity=mon.1 client_cookie=63c5f7a7a4f5fb81 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:09:55.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff5ffb700 1 -- 172.21.15.203:0/1092998169 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0098a0 con 0x7f8ff814e310 2021-01-06T22:09:55.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff5ffb700 1 -- 172.21.15.203:0/1092998169 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec009a00 con 0x7f8ff814e310 2021-01-06T22:09:55.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff5ffb700 1 -- 172.21.15.203:0/1092998169 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec009d80 con 0x7f8ff814e310 2021-01-06T22:09:55.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff7fff700 1 -- 172.21.15.203:0/1092998169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff814e310 msgr2=0x7f8ff81471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:55.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff7fff700 1 --2- 172.21.15.203:0/1092998169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff814e310 0x7f8ff81471b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f8ff8066e10 tx=0x7f8fec006ea0).stop 2021-01-06T22:09:55.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff7fff700 1 -- 172.21.15.203:0/1092998169 shutdown_connections 2021-01-06T22:09:55.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff7fff700 1 --2- 172.21.15.203:0/1092998169 >> v2:172.21.15.91:3300/0 conn(0x7f8ff8064760 0x7f8ff81476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:55.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff7fff700 1 --2- 172.21.15.203:0/1092998169 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ff8064380 0x7f8ff80c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:55.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff7fff700 1 --2- 172.21.15.203:0/1092998169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff814e310 0x7f8ff81471b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:55.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff7fff700 1 -- 172.21.15.203:0/1092998169 >> 172.21.15.203:0/1092998169 conn(0x7f8ff80667b0 msgr2=0x7f8ff80c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:09:55.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff7fff700 1 -- 172.21.15.203:0/1092998169 shutdown_connections 2021-01-06T22:09:55.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.058+0000 7f8ff7fff700 1 -- 172.21.15.203:0/1092998169 wait complete. 2021-01-06T22:09:55.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff7fff700 1 Processor -- start 2021-01-06T22:09:55.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff7fff700 1 -- start start 2021-01-06T22:09:55.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff8064380 0x7f8ff81492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:55.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff7fff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ff8064760 0x7f8ff814fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:55.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff7fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ff814e310 0x7f8ff815c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:55.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff7fff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff8061870 con 0x7f8ff8064760 2021-01-06T22:09:55.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff7fff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff80b5d20 con 0x7f8ff8064380 2021-01-06T22:09:55.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff7fff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff80b5fc0 con 0x7f8ff814e310 2021-01-06T22:09:55.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff6ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff8064380 0x7f8ff81492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:55.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff6ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff8064380 0x7f8ff81492a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50824/0 (socket says 172.21.15.203:50824) 2021-01-06T22:09:55.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff6ffd700 1 -- 172.21.15.203:0/2436568736 learned_addr learned my addr 172.21.15.203:0/2436568736 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:09:55.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff77fe700 1 --2- 172.21.15.203:0/2436568736 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ff814e310 0x7f8ff815c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:55.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff67fc700 1 --2- 172.21.15.203:0/2436568736 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ff8064760 0x7f8ff814fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:55.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff6ffd700 1 -- 172.21.15.203:0/2436568736 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ff814e310 msgr2=0x7f8ff815c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:55.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff6ffd700 1 --2- 172.21.15.203:0/2436568736 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ff814e310 0x7f8ff815c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:55.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff6ffd700 1 -- 172.21.15.203:0/2436568736 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ff8064760 msgr2=0x7f8ff814fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:55.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff6ffd700 1 --2- 172.21.15.203:0/2436568736 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ff8064760 0x7f8ff814fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:55.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff6ffd700 1 -- 172.21.15.203:0/2436568736 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff8159250 con 0x7f8ff8064380 2021-01-06T22:09:55.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff6ffd700 1 --2- 172.21.15.203:0/2436568736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff8064380 0x7f8ff81492a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f8fe800eb20 tx=0x7f8fe8006de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:09:55.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8fdeffd700 1 -- 172.21.15.203:0/2436568736 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800c9b0 con 0x7f8ff8064380 2021-01-06T22:09:55.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8fdeffd700 1 -- 172.21.15.203:0/2436568736 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe800cb10 con 0x7f8ff8064380 2021-01-06T22:09:55.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff7fff700 1 -- 172.21.15.203:0/2436568736 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff81594f0 con 0x7f8ff8064380 2021-01-06T22:09:55.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8fdeffd700 1 -- 172.21.15.203:0/2436568736 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800ce90 con 0x7f8ff8064380 2021-01-06T22:09:55.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.062+0000 7f8ff7fff700 1 -- 172.21.15.203:0/2436568736 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff8159a30 con 0x7f8ff8064380 2021-01-06T22:09:55.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.066+0000 7f8ff7fff700 1 -- 172.21.15.203:0/2436568736 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff8159bf0 con 0x7f8ff8064380 2021-01-06T22:09:55.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.066+0000 7f8fdeffd700 1 -- 172.21.15.203:0/2436568736 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f8fe8012600 con 0x7f8ff8064380 2021-01-06T22:09:55.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.066+0000 7f8fdeffd700 1 --2- 172.21.15.203:0/2436568736 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8fd8056400 0x7f8fd80588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:55.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.066+0000 7f8fdeffd700 1 -- 172.21.15.203:0/2436568736 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f8fe806fa10 con 0x7f8ff8064380 2021-01-06T22:09:55.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.066+0000 7f8ff67fc700 1 --2- 172.21.15.203:0/2436568736 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8fd8056400 0x7f8fd80588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:55.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.066+0000 7f8ff67fc700 1 --2- 172.21.15.203:0/2436568736 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8fd8056400 0x7f8fd80588c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f8ff80bd300 tx=0x7f8fec016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:09:55.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.070+0000 7f8fdeffd700 1 -- 172.21.15.203:0/2436568736 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f8fe8045130 con 0x7f8ff8064380 2021-01-06T22:09:55.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.294+0000 7f8ff7fff700 1 -- 172.21.15.203:0/2436568736 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ff805ca60 con 0x7f8ff8064380 2021-01-06T22:09:55.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.294+0000 7f8fdeffd700 1 -- 172.21.15.203:0/2436568736 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3561 (secure 0 0 0) 0x7f8fe8022250 con 0x7f8ff8064380 2021-01-06T22:09:55.297 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:09:55.298 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4859,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:55.422280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:09:55.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.298+0000 7f8ff7fff700 1 -- 172.21.15.203:0/2436568736 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8fd8056400 msgr2=0x7f8fd80588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:55.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.298+0000 7f8ff7fff700 1 --2- 172.21.15.203:0/2436568736 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8fd8056400 0x7f8fd80588c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f8ff80bd300 tx=0x7f8fec016000).stop 2021-01-06T22:09:55.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.298+0000 7f8ff7fff700 1 -- 172.21.15.203:0/2436568736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff8064380 msgr2=0x7f8ff81492a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:55.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.298+0000 7f8ff7fff700 1 --2- 172.21.15.203:0/2436568736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff8064380 0x7f8ff81492a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f8fe800eb20 tx=0x7f8fe8006de0).stop 2021-01-06T22:09:55.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.298+0000 7f8ff7fff700 1 -- 172.21.15.203:0/2436568736 shutdown_connections 2021-01-06T22:09:55.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.298+0000 7f8ff7fff700 1 --2- 172.21.15.203:0/2436568736 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8fd8056400 0x7f8fd80588c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:55.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.298+0000 7f8ff7fff700 1 --2- 172.21.15.203:0/2436568736 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ff814e310 0x7f8ff815c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:55.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.298+0000 7f8ff7fff700 1 --2- 172.21.15.203:0/2436568736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff8064380 0x7f8ff81492a0 secure :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0x7f8fe800eb20 tx=0x7f8fe8006de0).stop 2021-01-06T22:09:55.304 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.298+0000 7f8ff7fff700 1 --2- 172.21.15.203:0/2436568736 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ff8064760 0x7f8ff814fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:55.304 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.298+0000 7f8ff7fff700 1 -- 172.21.15.203:0/2436568736 >> 172.21.15.203:0/2436568736 conn(0x7f8ff80667b0 msgr2=0x7f8ff80ba050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:09:55.304 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.298+0000 7f8ff7fff700 1 -- 172.21.15.203:0/2436568736 shutdown_connections 2021-01-06T22:09:55.304 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.302+0000 7f8ff7fff700 1 -- 172.21.15.203:0/2436568736 wait complete. 2021-01-06T22:09:55.305 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 38 2021-01-06T22:09:55.315 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=0) 2021-01-06T22:09:55.316 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2021-01-06T22:09:55.316 INFO:teuthology.orchestra.run:waiting for 300 2021-01-06T22:09:55.334 INFO:tasks.ceph.mds.f:Stopped 2021-01-06T22:09:55.335 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:09:55.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.458+0000 7f728f59e700 1 Processor -- start 2021-01-06T22:09:55.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.458+0000 7f728f59e700 1 -- start start 2021-01-06T22:09:55.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.458+0000 7f728f59e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f72901513f0 0x7f72900c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:55.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.458+0000 7f728f59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72900bacb0 con 0x7f72900bccb0 2021-01-06T22:09:55.462 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.458+0000 7f728f59e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7290152000 con 0x7f729014ad90 2021-01-06T22:09:55.462 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.458+0000 7f728f59e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72900bd160 con 0x7f72901513f0 2021-01-06T22:09:55.462 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.458+0000 7f728ed9d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f729014ad90 0x7f72900641e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49434/0 (socket says 172.21.15.203:49434) 2021-01-06T22:09:55.462 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.458+0000 7f728ed9d700 1 -- 172.21.15.203:0/3905585906 learned_addr learned my addr 172.21.15.203:0/3905585906 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:09:55.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.458+0000 7f728e59c700 1 --2- 172.21.15.203:0/3905585906 >> v2:172.21.15.203:3300/0 conn(0x7f72901513f0 0x7f72900c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:55.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.458+0000 7f728e59c700 1 -- 172.21.15.203:0/3905585906 >> v1:172.21.15.203:6789/0 conn(0x7f729014ad90 legacy=0x7f72900641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:55.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.458+0000 7f728e59c700 1 -- 172.21.15.203:0/3905585906 >> v1:172.21.15.91:6789/0 conn(0x7f72900bccb0 legacy=0x7f72901517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:55.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.458+0000 7f728e59c700 1 -- 172.21.15.203:0/3905585906 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7290067e50 con 0x7f72901513f0 2021-01-06T22:09:55.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.458+0000 7f728e59c700 1 --2- 172.21.15.203:0/3905585906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72901513f0 0x7f72900c0fa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f728000b980 tx=0x7f7280006de0).ready entity=mon.1 client_cookie=7a14d28371bfe9fb server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:09:55.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728d59a700 1 -- 172.21.15.203:0/3905585906 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72800097b0 con 0x7f72901513f0 2021-01-06T22:09:55.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728d59a700 1 -- 172.21.15.203:0/3905585906 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7280009910 con 0x7f72901513f0 2021-01-06T22:09:55.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728d59a700 1 -- 172.21.15.203:0/3905585906 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7280009c90 con 0x7f72901513f0 2021-01-06T22:09:55.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728f59e700 1 -- 172.21.15.203:0/3905585906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72901513f0 msgr2=0x7f72900c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:55.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728f59e700 1 --2- 172.21.15.203:0/3905585906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72901513f0 0x7f72900c0fa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f728000b980 tx=0x7f7280006de0).stop 2021-01-06T22:09:55.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728f59e700 1 -- 172.21.15.203:0/3905585906 shutdown_connections 2021-01-06T22:09:55.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728f59e700 1 --2- 172.21.15.203:0/3905585906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72901513f0 0x7f72900c0fa0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:55.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728f59e700 1 -- 172.21.15.203:0/3905585906 >> 172.21.15.203:0/3905585906 conn(0x7f72900667b0 msgr2=0x7f72900c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:09:55.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728f59e700 1 -- 172.21.15.203:0/3905585906 shutdown_connections 2021-01-06T22:09:55.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728f59e700 1 -- 172.21.15.203:0/3905585906 wait complete. 2021-01-06T22:09:55.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728f59e700 1 Processor -- start 2021-01-06T22:09:55.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728f59e700 1 -- start start 2021-01-06T22:09:55.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728f59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72900bccb0 0x7f72900b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:55.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728f59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729014ad90 0x7f7290061130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:55.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728f59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7290160d20 0x7f7290145fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:55.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728f59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72900b1e00 con 0x7f7290160d20 2021-01-06T22:09:55.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728f59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72900c1030 con 0x7f729014ad90 2021-01-06T22:09:55.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728f59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72900b2b90 con 0x7f72900bccb0 2021-01-06T22:09:55.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728dd9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729014ad90 0x7f7290061130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:55.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728ed9d700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7290160d20 0x7f7290145fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:55.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728e59c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72900bccb0 0x7f72900b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:55.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728dd9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729014ad90 0x7f7290061130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50846/0 (socket says 172.21.15.203:50846) 2021-01-06T22:09:55.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.462+0000 7f728dd9b700 1 -- 172.21.15.203:0/1185093613 learned_addr learned my addr 172.21.15.203:0/1185093613 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:09:55.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f728dd9b700 1 -- 172.21.15.203:0/1185093613 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72900bccb0 msgr2=0x7f72900b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:55.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f728dd9b700 1 --2- 172.21.15.203:0/1185093613 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72900bccb0 0x7f72900b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:55.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f728dd9b700 1 -- 172.21.15.203:0/1185093613 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7290160d20 msgr2=0x7f7290145fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:55.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f728dd9b700 1 --2- 172.21.15.203:0/1185093613 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7290160d20 0x7f7290145fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:55.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f728dd9b700 1 -- 172.21.15.203:0/1185093613 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729015e210 con 0x7f729014ad90 2021-01-06T22:09:55.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f728dd9b700 1 --2- 172.21.15.203:0/1185093613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729014ad90 0x7f7290061130 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f728400c710 tx=0x7f7284007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:09:55.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f727e7fc700 1 -- 172.21.15.203:0/1185093613 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7284007020 con 0x7f729014ad90 2021-01-06T22:09:55.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f727e7fc700 1 -- 172.21.15.203:0/1185093613 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7284007180 con 0x7f729014ad90 2021-01-06T22:09:55.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f728f59e700 1 -- 172.21.15.203:0/1185093613 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729015e4b0 con 0x7f729014ad90 2021-01-06T22:09:55.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f727e7fc700 1 -- 172.21.15.203:0/1185093613 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728400e560 con 0x7f729014ad90 2021-01-06T22:09:55.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f728f59e700 1 -- 172.21.15.203:0/1185093613 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72900b1b60 con 0x7f729014ad90 2021-01-06T22:09:55.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f728f59e700 1 -- 172.21.15.203:0/1185093613 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72900450a0 con 0x7f729014ad90 2021-01-06T22:09:55.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f727e7fc700 1 -- 172.21.15.203:0/1185093613 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f7284010070 con 0x7f729014ad90 2021-01-06T22:09:55.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f727e7fc700 1 --2- 172.21.15.203:0/1185093613 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f726c0563d0 0x7f726c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:55.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f727e7fc700 1 -- 172.21.15.203:0/1185093613 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f728406e800 con 0x7f729014ad90 2021-01-06T22:09:55.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f728e59c700 1 --2- 172.21.15.203:0/1185093613 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f726c0563d0 0x7f726c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:55.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.466+0000 7f728e59c700 1 --2- 172.21.15.203:0/1185093613 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f726c0563d0 0x7f726c058890 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f729005c9b0 tx=0x7f7280016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:09:55.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.474+0000 7f727e7fc700 1 -- 172.21.15.203:0/1185093613 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f7284043670 con 0x7f729014ad90 2021-01-06T22:09:55.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.694+0000 7f728f59e700 1 -- 172.21.15.203:0/1185093613 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72900bacb0 con 0x7f729014ad90 2021-01-06T22:09:55.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.694+0000 7f727e7fc700 1 -- 172.21.15.203:0/1185093613 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3561 (secure 0 0 0) 0x7f72900bacb0 con 0x7f729014ad90 2021-01-06T22:09:55.696 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:09:55.696 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4859,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:55.422280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:09:55.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.694+0000 7f728f59e700 1 -- 172.21.15.203:0/1185093613 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f726c0563d0 msgr2=0x7f726c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:55.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.694+0000 7f728f59e700 1 --2- 172.21.15.203:0/1185093613 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f726c0563d0 0x7f726c058890 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f729005c9b0 tx=0x7f7280016000).stop 2021-01-06T22:09:55.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.694+0000 7f728f59e700 1 -- 172.21.15.203:0/1185093613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729014ad90 msgr2=0x7f7290061130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:55.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.694+0000 7f728f59e700 1 --2- 172.21.15.203:0/1185093613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729014ad90 0x7f7290061130 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f728400c710 tx=0x7f7284007c40).stop 2021-01-06T22:09:55.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.694+0000 7f728f59e700 1 -- 172.21.15.203:0/1185093613 shutdown_connections 2021-01-06T22:09:55.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.694+0000 7f728f59e700 1 --2- 172.21.15.203:0/1185093613 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f726c0563d0 0x7f726c058890 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0x7f729005c9b0 tx=0x7f7280016000).stop 2021-01-06T22:09:55.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.694+0000 7f728f59e700 1 --2- 172.21.15.203:0/1185093613 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7290160d20 0x7f7290145fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:55.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.694+0000 7f728f59e700 1 --2- 172.21.15.203:0/1185093613 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72900bccb0 0x7f72900b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:55.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.694+0000 7f728f59e700 1 --2- 172.21.15.203:0/1185093613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729014ad90 0x7f7290061130 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:55.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.694+0000 7f728f59e700 1 -- 172.21.15.203:0/1185093613 >> 172.21.15.203:0/1185093613 conn(0x7f72900667b0 msgr2=0x7f72900641e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:09:55.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.694+0000 7f728f59e700 1 -- 172.21.15.203:0/1185093613 shutdown_connections 2021-01-06T22:09:55.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:55.694+0000 7f728f59e700 1 -- 172.21.15.203:0/1185093613 wait complete. 2021-01-06T22:09:55.701 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 38 2021-01-06T22:09:55.711 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:09:57.712 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:09:57.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.846+0000 7f9e19caf700 1 Processor -- start 2021-01-06T22:09:57.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.846+0000 7f9e19caf700 1 -- start start 2021-01-06T22:09:57.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e19caf700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9e1414e310 0x7f9e141471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:57.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e19caf700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9e14064760 0x7f9e141476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:57.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e19caf700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e140c16b0 con 0x7f9e14064380 2021-01-06T22:09:57.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e19caf700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e14147b90 con 0x7f9e14064760 2021-01-06T22:09:57.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e19caf700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e140bb2d0 con 0x7f9e1414e310 2021-01-06T22:09:57.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e13fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9e1414e310 0x7f9e141471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:57.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e13fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9e1414e310 0x7f9e141471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50864/0 (socket says 172.21.15.203:50864) 2021-01-06T22:09:57.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e13fff700 1 -- 172.21.15.203:0/4286864641 learned_addr learned my addr 172.21.15.203:0/4286864641 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:09:57.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e194ae700 1 --2- 172.21.15.203:0/4286864641 >> v2:172.21.15.91:3300/0 conn(0x7f9e14064760 0x7f9e141476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:57.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e13fff700 1 -- 172.21.15.203:0/4286864641 >> v2:172.21.15.91:3300/0 conn(0x7f9e14064760 msgr2=0x7f9e141476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:09:57.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e13fff700 1 --2- 172.21.15.203:0/4286864641 >> v2:172.21.15.91:3300/0 conn(0x7f9e14064760 0x7f9e141476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:57.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e13fff700 1 -- 172.21.15.203:0/4286864641 >> v1:172.21.15.91:6789/0 conn(0x7f9e14064380 legacy=0x7f9e140c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:57.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e137fe700 1 -- 172.21.15.203:0/4286864641 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382670383 0 0) 0x7f9e04001450 con 0x7f9e14064380 2021-01-06T22:09:57.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e13fff700 1 -- 172.21.15.203:0/4286864641 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1414a560 con 0x7f9e1414e310 2021-01-06T22:09:57.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e13fff700 1 --2- 172.21.15.203:0/4286864641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e1414e310 0x7f9e141471b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f9e14061c90 tx=0x7f9e08006f30).ready entity=mon.1 client_cookie=db712917d6c09a85 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:09:57.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e137fe700 1 -- 172.21.15.203:0/4286864641 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0800cb60 con 0x7f9e1414e310 2021-01-06T22:09:57.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e137fe700 1 -- 172.21.15.203:0/4286864641 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0800ccc0 con 0x7f9e1414e310 2021-01-06T22:09:57.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e137fe700 1 -- 172.21.15.203:0/4286864641 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08010620 con 0x7f9e1414e310 2021-01-06T22:09:57.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e19caf700 1 -- 172.21.15.203:0/4286864641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e1414e310 msgr2=0x7f9e141471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:57.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.850+0000 7f9e19caf700 1 --2- 172.21.15.203:0/4286864641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e1414e310 0x7f9e141471b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f9e14061c90 tx=0x7f9e08006f30).stop 2021-01-06T22:09:57.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.854+0000 7f9e19caf700 1 -- 172.21.15.203:0/4286864641 shutdown_connections 2021-01-06T22:09:57.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.854+0000 7f9e19caf700 1 --2- 172.21.15.203:0/4286864641 >> v2:172.21.15.91:3300/0 conn(0x7f9e14064760 0x7f9e141476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:57.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.854+0000 7f9e19caf700 1 --2- 172.21.15.203:0/4286864641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e1414e310 0x7f9e141471b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:57.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.854+0000 7f9e19caf700 1 -- 172.21.15.203:0/4286864641 >> 172.21.15.203:0/4286864641 conn(0x7f9e140667b0 msgr2=0x7f9e140c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:09:57.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.854+0000 7f9e19caf700 1 -- 172.21.15.203:0/4286864641 shutdown_connections 2021-01-06T22:09:57.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.854+0000 7f9e19caf700 1 -- 172.21.15.203:0/4286864641 wait complete. 2021-01-06T22:09:57.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.854+0000 7f9e19caf700 1 Processor -- start 2021-01-06T22:09:57.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.854+0000 7f9e19caf700 1 -- start start 2021-01-06T22:09:57.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e19caf700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e14064380 0x7f9e14149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:57.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e19caf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e14064760 0x7f9e141462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:57.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e19caf700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e1414e310 0x7f9e141467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:57.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e19caf700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e140b5c40 con 0x7f9e14064380 2021-01-06T22:09:57.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e19caf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e141499f0 con 0x7f9e14064760 2021-01-06T22:09:57.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e19caf700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1415a010 con 0x7f9e1414e310 2021-01-06T22:09:57.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e13fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e14064760 0x7f9e141462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:57.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e13fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e14064760 0x7f9e141462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50870/0 (socket says 172.21.15.203:50870) 2021-01-06T22:09:57.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e13fff700 1 -- 172.21.15.203:0/2056520333 learned_addr learned my addr 172.21.15.203:0/2056520333 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:09:57.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e194ae700 1 --2- 172.21.15.203:0/2056520333 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e1414e310 0x7f9e141467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:57.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e18cad700 1 --2- 172.21.15.203:0/2056520333 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e14064380 0x7f9e14149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:57.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e13fff700 1 -- 172.21.15.203:0/2056520333 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e1414e310 msgr2=0x7f9e141467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:57.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e13fff700 1 --2- 172.21.15.203:0/2056520333 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e1414e310 0x7f9e141467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:57.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e13fff700 1 -- 172.21.15.203:0/2056520333 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e14064380 msgr2=0x7f9e14149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:57.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e13fff700 1 --2- 172.21.15.203:0/2056520333 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e14064380 0x7f9e14149120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:57.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e13fff700 1 -- 172.21.15.203:0/2056520333 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e140b5e70 con 0x7f9e14064760 2021-01-06T22:09:57.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e18cad700 1 --2- 172.21.15.203:0/2056520333 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e14064380 0x7f9e14149120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:09:57.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e13fff700 1 --2- 172.21.15.203:0/2056520333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e14064760 0x7f9e141462d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f9e140bb410 tx=0x7f9e140b7f00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:09:57.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e10ff9700 1 -- 172.21.15.203:0/2056520333 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08019630 con 0x7f9e14064760 2021-01-06T22:09:57.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e19caf700 1 -- 172.21.15.203:0/2056520333 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e140b6110 con 0x7f9e14064760 2021-01-06T22:09:57.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e10ff9700 1 -- 172.21.15.203:0/2056520333 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e08019790 con 0x7f9e14064760 2021-01-06T22:09:57.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e10ff9700 1 -- 172.21.15.203:0/2056520333 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08019b30 con 0x7f9e14064760 2021-01-06T22:09:57.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.858+0000 7f9e19caf700 1 -- 172.21.15.203:0/2056520333 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e140b66d0 con 0x7f9e14064760 2021-01-06T22:09:57.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.862+0000 7f9e19caf700 1 -- 172.21.15.203:0/2056520333 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e140450a0 con 0x7f9e14064760 2021-01-06T22:09:57.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.862+0000 7f9e10ff9700 1 -- 172.21.15.203:0/2056520333 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f9e0800e040 con 0x7f9e14064760 2021-01-06T22:09:57.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.862+0000 7f9e10ff9700 1 --2- 172.21.15.203:0/2056520333 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9df40563d0 0x7f9df4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:09:57.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.862+0000 7f9e10ff9700 1 -- 172.21.15.203:0/2056520333 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f9e08070720 con 0x7f9e14064760 2021-01-06T22:09:57.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.862+0000 7f9e18cad700 1 --2- 172.21.15.203:0/2056520333 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9df40563d0 0x7f9df4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:09:57.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.862+0000 7f9e18cad700 1 --2- 172.21.15.203:0/2056520333 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9df40563d0 0x7f9df4058890 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f9e04001c40 tx=0x7f9e04009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:09:57.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:57.866+0000 7f9e10ff9700 1 -- 172.21.15.203:0/2056520333 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f9e08045f20 con 0x7f9e14064760 2021-01-06T22:09:58.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:58.070+0000 7f9e19caf700 1 -- 172.21.15.203:0/2056520333 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e14061f70 con 0x7f9e14064760 2021-01-06T22:09:58.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:58.074+0000 7f9e10ff9700 1 -- 172.21.15.203:0/2056520333 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3561 (secure 0 0 0) 0x7f9e08014090 con 0x7f9e14064760 2021-01-06T22:09:58.076 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:09:58.076 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4859,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:55.422280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:09:58.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:58.074+0000 7f9e19caf700 1 -- 172.21.15.203:0/2056520333 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9df40563d0 msgr2=0x7f9df4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:58.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:58.074+0000 7f9e19caf700 1 --2- 172.21.15.203:0/2056520333 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9df40563d0 0x7f9df4058890 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f9e04001c40 tx=0x7f9e04009040).stop 2021-01-06T22:09:58.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:58.074+0000 7f9e19caf700 1 -- 172.21.15.203:0/2056520333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e14064760 msgr2=0x7f9e141462d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:09:58.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:58.074+0000 7f9e19caf700 1 --2- 172.21.15.203:0/2056520333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e14064760 0x7f9e141462d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f9e140bb410 tx=0x7f9e140b7f00).stop 2021-01-06T22:09:58.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:58.074+0000 7f9e19caf700 1 -- 172.21.15.203:0/2056520333 shutdown_connections 2021-01-06T22:09:58.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:58.074+0000 7f9e19caf700 1 --2- 172.21.15.203:0/2056520333 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9df40563d0 0x7f9df4058890 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:58.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:58.074+0000 7f9e19caf700 1 --2- 172.21.15.203:0/2056520333 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e1414e310 0x7f9e141467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:58.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:58.074+0000 7f9e19caf700 1 --2- 172.21.15.203:0/2056520333 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e14064380 0x7f9e14149120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:58.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:58.074+0000 7f9e19caf700 1 --2- 172.21.15.203:0/2056520333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e14064760 0x7f9e141462d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:09:58.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:58.074+0000 7f9e19caf700 1 -- 172.21.15.203:0/2056520333 >> 172.21.15.203:0/2056520333 conn(0x7f9e140667b0 msgr2=0x7f9e14067b70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:09:58.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:58.074+0000 7f9e19caf700 1 -- 172.21.15.203:0/2056520333 shutdown_connections 2021-01-06T22:09:58.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:09:58.074+0000 7f9e19caf700 1 -- 172.21.15.203:0/2056520333 wait complete. 2021-01-06T22:09:58.081 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 38 2021-01-06T22:09:58.090 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:10:00.092 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:10:00.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.214+0000 7fe489b1c700 1 Processor -- start 2021-01-06T22:10:00.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.214+0000 7fe489b1c700 1 -- start start 2021-01-06T22:10:00.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.214+0000 7fe489b1c700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe4840a5e50 0x7fe4840ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:00.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.214+0000 7fe489b1c700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4840646b0 con 0x7fe4840bb7d0 2021-01-06T22:10:00.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.214+0000 7fe489b1c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe484152cc0 con 0x7fe4840bb3f0 2021-01-06T22:10:00.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.214+0000 7fe489b1c700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4840670c0 con 0x7fe4840a5e50 2021-01-06T22:10:00.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.214+0000 7fe488b1a700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fe4840bb3f0 0x7fe4840669b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49476/0 (socket says 172.21.15.203:49476) 2021-01-06T22:10:00.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.214+0000 7fe488b1a700 1 -- 172.21.15.203:0/3563562910 learned_addr learned my addr 172.21.15.203:0/3563562910 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:10:00.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.214+0000 7fe47bfff700 1 --2- 172.21.15.203:0/3563562910 >> v2:172.21.15.91:3300/0 conn(0x7fe4840a5e50 0x7fe4840ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:00.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.214+0000 7fe47b7fe700 1 -- 172.21.15.203:0/3563562910 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898220981 0 0) 0x7fe484152cc0 con 0x7fe4840bb3f0 2021-01-06T22:10:00.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.214+0000 7fe47bfff700 1 -- 172.21.15.203:0/3563562910 >> v1:172.21.15.203:6789/0 conn(0x7fe4840bb3f0 legacy=0x7fe4840669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:00.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.214+0000 7fe47bfff700 1 -- 172.21.15.203:0/3563562910 >> v1:172.21.15.91:6789/0 conn(0x7fe4840bb7d0 legacy=0x7fe48414eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:00.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.214+0000 7fe47bfff700 1 -- 172.21.15.203:0/3563562910 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe484064450 con 0x7fe4840a5e50 2021-01-06T22:10:00.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.214+0000 7fe47bfff700 1 --2- 172.21.15.203:0/3563562910 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4840a5e50 0x7fe4840ba9d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fe47400b980 tx=0x7fe474006de0).ready entity=mon.0 client_cookie=3817aaad532edb7d server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:00.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.218+0000 7fe47b7fe700 1 -- 172.21.15.203:0/3563562910 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4740097b0 con 0x7fe4840a5e50 2021-01-06T22:10:00.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.218+0000 7fe47b7fe700 1 -- 172.21.15.203:0/3563562910 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe474009910 con 0x7fe4840a5e50 2021-01-06T22:10:00.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.218+0000 7fe47b7fe700 1 -- 172.21.15.203:0/3563562910 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe474009c90 con 0x7fe4840a5e50 2021-01-06T22:10:00.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.218+0000 7fe489b1c700 1 -- 172.21.15.203:0/3563562910 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4840a5e50 msgr2=0x7fe4840ba9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:00.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.218+0000 7fe489b1c700 1 --2- 172.21.15.203:0/3563562910 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4840a5e50 0x7fe4840ba9d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fe47400b980 tx=0x7fe474006de0).stop 2021-01-06T22:10:00.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.218+0000 7fe489b1c700 1 -- 172.21.15.203:0/3563562910 shutdown_connections 2021-01-06T22:10:00.224 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.218+0000 7fe489b1c700 1 --2- 172.21.15.203:0/3563562910 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4840a5e50 0x7fe4840ba9d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:00.224 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.218+0000 7fe489b1c700 1 -- 172.21.15.203:0/3563562910 >> 172.21.15.203:0/3563562910 conn(0x7fe484061530 msgr2=0x7fe484063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:00.224 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.218+0000 7fe489b1c700 1 -- 172.21.15.203:0/3563562910 shutdown_connections 2021-01-06T22:10:00.224 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.218+0000 7fe489b1c700 1 -- 172.21.15.203:0/3563562910 wait complete. 2021-01-06T22:10:00.224 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.218+0000 7fe489b1c700 1 Processor -- start 2021-01-06T22:10:00.225 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.218+0000 7fe489b1c700 1 -- start start 2021-01-06T22:10:00.225 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe489b1c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4840a5e50 0x7fe484151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:00.225 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe489b1c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4840bb3f0 0x7fe484162dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:00.225 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe489b1c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4840bb7d0 0x7fe4841632c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:00.225 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe489b1c700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4841628c0 con 0x7fe4840bb7d0 2021-01-06T22:10:00.226 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe489b1c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe484161a10 con 0x7fe4840bb3f0 2021-01-06T22:10:00.226 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe489b1c700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe484161c40 con 0x7fe4840a5e50 2021-01-06T22:10:00.226 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe47bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4840bb3f0 0x7fe484162dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:00.226 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe47bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4840bb3f0 0x7fe484162dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50888/0 (socket says 172.21.15.203:50888) 2021-01-06T22:10:00.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe47bfff700 1 -- 172.21.15.203:0/417354192 learned_addr learned my addr 172.21.15.203:0/417354192 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:10:00.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe488b1a700 1 --2- 172.21.15.203:0/417354192 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4840a5e50 0x7fe484151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:00.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe48931b700 1 --2- 172.21.15.203:0/417354192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4840bb7d0 0x7fe4841632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:00.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe47bfff700 1 -- 172.21.15.203:0/417354192 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4840a5e50 msgr2=0x7fe484151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:00.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe47bfff700 1 --2- 172.21.15.203:0/417354192 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4840a5e50 0x7fe484151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:00.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe47bfff700 1 -- 172.21.15.203:0/417354192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4840bb7d0 msgr2=0x7fe4841632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:00.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe47bfff700 1 --2- 172.21.15.203:0/417354192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4840bb7d0 0x7fe4841632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:00.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe47bfff700 1 -- 172.21.15.203:0/417354192 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe484161ee0 con 0x7fe4840bb3f0 2021-01-06T22:10:00.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe47bfff700 1 --2- 172.21.15.203:0/417354192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4840bb3f0 0x7fe484162dd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fe474010000 tx=0x7fe474016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:00.230 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe478ff9700 1 -- 172.21.15.203:0/417354192 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe474020070 con 0x7fe4840bb3f0 2021-01-06T22:10:00.230 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe478ff9700 1 -- 172.21.15.203:0/417354192 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47401c420 con 0x7fe4840bb3f0 2021-01-06T22:10:00.230 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe489b1c700 1 -- 172.21.15.203:0/417354192 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe484162180 con 0x7fe4840bb3f0 2021-01-06T22:10:00.230 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe478ff9700 1 -- 172.21.15.203:0/417354192 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47401c7c0 con 0x7fe4840bb3f0 2021-01-06T22:10:00.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe489b1c700 1 -- 172.21.15.203:0/417354192 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4840bc1b0 con 0x7fe4840bb3f0 2021-01-06T22:10:00.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe489b1c700 1 -- 172.21.15.203:0/417354192 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4840450a0 con 0x7fe4840bb3f0 2021-01-06T22:10:00.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe478ff9700 1 -- 172.21.15.203:0/417354192 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fe47400f070 con 0x7fe4840bb3f0 2021-01-06T22:10:00.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe478ff9700 1 --2- 172.21.15.203:0/417354192 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe4640563d0 0x7fe464058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:00.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe478ff9700 1 -- 172.21.15.203:0/417354192 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fe474070840 con 0x7fe4840bb3f0 2021-01-06T22:10:00.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.222+0000 7fe488b1a700 1 --2- 172.21.15.203:0/417354192 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe4640563d0 0x7fe464058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:00.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.226+0000 7fe488b1a700 1 --2- 172.21.15.203:0/417354192 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe4640563d0 0x7fe464058890 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fe4800011a0 tx=0x7fe4800079f0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:00.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.230+0000 7fe478ff9700 1 -- 172.21.15.203:0/417354192 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fe474045fc0 con 0x7fe4840bb3f0 2021-01-06T22:10:00.433 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.430+0000 7fe489b1c700 1 -- 172.21.15.203:0/417354192 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe484057980 con 0x7fe4840bb3f0 2021-01-06T22:10:00.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.434+0000 7fe478ff9700 1 -- 172.21.15.203:0/417354192 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3561 (secure 0 0 0) 0x7fe4740495e0 con 0x7fe4840bb3f0 2021-01-06T22:10:00.434 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:10:00.434 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4859,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:55.422280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:10:00.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.434+0000 7fe489b1c700 1 -- 172.21.15.203:0/417354192 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe4640563d0 msgr2=0x7fe464058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:00.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.434+0000 7fe489b1c700 1 --2- 172.21.15.203:0/417354192 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe4640563d0 0x7fe464058890 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fe4800011a0 tx=0x7fe4800079f0).stop 2021-01-06T22:10:00.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.434+0000 7fe489b1c700 1 -- 172.21.15.203:0/417354192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4840bb3f0 msgr2=0x7fe484162dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:00.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.434+0000 7fe489b1c700 1 --2- 172.21.15.203:0/417354192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4840bb3f0 0x7fe484162dd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fe474010000 tx=0x7fe474016000).stop 2021-01-06T22:10:00.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.434+0000 7fe489b1c700 1 -- 172.21.15.203:0/417354192 shutdown_connections 2021-01-06T22:10:00.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.434+0000 7fe489b1c700 1 --2- 172.21.15.203:0/417354192 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe4640563d0 0x7fe464058890 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:00.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.434+0000 7fe489b1c700 1 --2- 172.21.15.203:0/417354192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4840bb7d0 0x7fe4841632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:00.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.434+0000 7fe489b1c700 1 --2- 172.21.15.203:0/417354192 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4840a5e50 0x7fe484151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:00.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.434+0000 7fe489b1c700 1 --2- 172.21.15.203:0/417354192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4840bb3f0 0x7fe484162dd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:00.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.434+0000 7fe489b1c700 1 -- 172.21.15.203:0/417354192 >> 172.21.15.203:0/417354192 conn(0x7fe484061530 msgr2=0x7fe4840bef80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:00.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.434+0000 7fe489b1c700 1 -- 172.21.15.203:0/417354192 shutdown_connections 2021-01-06T22:10:00.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:00.434+0000 7fe489b1c700 1 -- 172.21.15.203:0/417354192 wait complete. 2021-01-06T22:10:00.439 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 38 2021-01-06T22:10:00.449 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:10:02.450 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:10:02.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa3a2f6700 1 Processor -- start 2021-01-06T22:10:02.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa3a2f6700 1 -- start start 2021-01-06T22:10:02.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa3a2f6700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faa340c04c0 0x7faa340680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:02.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa3a2f6700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa340c08a0 0x7faa34063ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:02.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa3a2f6700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa341531f0 con 0x7faa34155b60 2021-01-06T22:10:02.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa3a2f6700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa341522e0 con 0x7faa340c04c0 2021-01-06T22:10:02.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa3a2f6700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa34063a10 con 0x7faa340c08a0 2021-01-06T22:10:02.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa33fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faa340c04c0 0x7faa340680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:02.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa337fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa340c08a0 0x7faa34063ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:02.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa33fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faa340c04c0 0x7faa340680b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:58746/0 (socket says 172.21.15.203:58746) 2021-01-06T22:10:02.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa33fff700 1 -- 172.21.15.203:0/1417329043 learned_addr learned my addr 172.21.15.203:0/1417329043 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:10:02.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa337fe700 1 -- 172.21.15.203:0/1417329043 >> v2:172.21.15.91:3300/0 conn(0x7faa340c04c0 msgr2=0x7faa340680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:10:02.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa337fe700 1 --2- 172.21.15.203:0/1417329043 >> v2:172.21.15.91:3300/0 conn(0x7faa340c04c0 0x7faa340680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:02.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa337fe700 1 -- 172.21.15.203:0/1417329043 >> v1:172.21.15.91:6789/0 conn(0x7faa34155b60 legacy=0x7faa3414f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:02.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa337fe700 1 -- 172.21.15.203:0/1417329043 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa341524d0 con 0x7faa340c08a0 2021-01-06T22:10:02.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa32ffd700 1 -- 172.21.15.203:0/1417329043 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701609042 0 0) 0x7faa341531f0 con 0x7faa34155b60 2021-01-06T22:10:02.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa33fff700 1 --2- 172.21.15.203:0/1417329043 >> v2:172.21.15.91:3300/0 conn(0x7faa340c04c0 0x7faa340680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:10:02.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa337fe700 1 --2- 172.21.15.203:0/1417329043 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa340c08a0 0x7faa34063ee0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7faa24009c90 tx=0x7faa240091d0).ready entity=mon.2 client_cookie=83a09a6c3b7439c7 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:02.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa32ffd700 1 -- 172.21.15.203:0/1417329043 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2400bb00 con 0x7faa340c08a0 2021-01-06T22:10:02.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa32ffd700 1 -- 172.21.15.203:0/1417329043 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2400bc60 con 0x7faa340c08a0 2021-01-06T22:10:02.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.578+0000 7faa32ffd700 1 -- 172.21.15.203:0/1417329043 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa24011400 con 0x7faa340c08a0 2021-01-06T22:10:02.590 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.582+0000 7faa3a2f6700 1 -- 172.21.15.203:0/1417329043 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa340c08a0 msgr2=0x7faa34063ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:02.590 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.582+0000 7faa3a2f6700 1 --2- 172.21.15.203:0/1417329043 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa340c08a0 0x7faa34063ee0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7faa24009c90 tx=0x7faa240091d0).stop 2021-01-06T22:10:02.590 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.582+0000 7faa3a2f6700 1 -- 172.21.15.203:0/1417329043 shutdown_connections 2021-01-06T22:10:02.590 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.582+0000 7faa3a2f6700 1 --2- 172.21.15.203:0/1417329043 >> v2:172.21.15.91:3300/0 conn(0x7faa340c04c0 0x7faa340680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:02.590 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.582+0000 7faa3a2f6700 1 --2- 172.21.15.203:0/1417329043 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa340c08a0 0x7faa34063ee0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:02.591 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.582+0000 7faa3a2f6700 1 -- 172.21.15.203:0/1417329043 >> 172.21.15.203:0/1417329043 conn(0x7faa34056cd0 msgr2=0x7faa340626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:02.591 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.582+0000 7faa3a2f6700 1 -- 172.21.15.203:0/1417329043 shutdown_connections 2021-01-06T22:10:02.591 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.582+0000 7faa3a2f6700 1 -- 172.21.15.203:0/1417329043 wait complete. 2021-01-06T22:10:02.593 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.582+0000 7faa3a2f6700 1 Processor -- start 2021-01-06T22:10:02.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.582+0000 7faa3a2f6700 1 -- start start 2021-01-06T22:10:02.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa3a2f6700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa340c04c0 0x7faa3414d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:02.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa3a2f6700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa340c08a0 0x7faa3414e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:02.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa3a2f6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa34155b60 0x7faa340bedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:02.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa3a2f6700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa34165e80 con 0x7faa340c08a0 2021-01-06T22:10:02.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa3a2f6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa34166120 con 0x7faa34155b60 2021-01-06T22:10:02.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa3a2f6700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa34161de0 con 0x7faa340c04c0 2021-01-06T22:10:02.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa38893700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa34155b60 0x7faa340bedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:02.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa38893700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa34155b60 0x7faa340bedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50908/0 (socket says 172.21.15.203:50908) 2021-01-06T22:10:02.596 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa38893700 1 -- 172.21.15.203:0/1987661776 learned_addr learned my addr 172.21.15.203:0/1987661776 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:10:02.596 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa337fe700 1 --2- 172.21.15.203:0/1987661776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa340c08a0 0x7faa3414e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:02.596 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa33fff700 1 --2- 172.21.15.203:0/1987661776 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa340c04c0 0x7faa3414d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:02.596 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa38893700 1 -- 172.21.15.203:0/1987661776 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa340c04c0 msgr2=0x7faa3414d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:02.596 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa38893700 1 --2- 172.21.15.203:0/1987661776 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa340c04c0 0x7faa3414d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:02.597 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa38893700 1 -- 172.21.15.203:0/1987661776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa340c08a0 msgr2=0x7faa3414e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:02.597 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa38893700 1 --2- 172.21.15.203:0/1987661776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa340c08a0 0x7faa3414e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:02.597 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa38893700 1 -- 172.21.15.203:0/1987661776 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa34162080 con 0x7faa34155b60 2021-01-06T22:10:02.597 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa38893700 1 --2- 172.21.15.203:0/1987661776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa34155b60 0x7faa340bedf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7faa2c00c730 tx=0x7faa2c007a30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:02.597 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa2bfff700 1 -- 172.21.15.203:0/1987661776 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c00e490 con 0x7faa34155b60 2021-01-06T22:10:02.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa2bfff700 1 -- 172.21.15.203:0/1987661776 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2c00e5f0 con 0x7faa34155b60 2021-01-06T22:10:02.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa2bfff700 1 -- 172.21.15.203:0/1987661776 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c00e990 con 0x7faa34155b60 2021-01-06T22:10:02.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa3a2f6700 1 -- 172.21.15.203:0/1987661776 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa34162320 con 0x7faa34155b60 2021-01-06T22:10:02.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa3a2f6700 1 -- 172.21.15.203:0/1987661776 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa3414c6c0 con 0x7faa34155b60 2021-01-06T22:10:02.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.586+0000 7faa3a2f6700 1 -- 172.21.15.203:0/1987661776 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa3405cfe0 con 0x7faa34155b60 2021-01-06T22:10:02.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.590+0000 7faa2bfff700 1 -- 172.21.15.203:0/1987661776 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7faa2c019030 con 0x7faa34155b60 2021-01-06T22:10:02.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.590+0000 7faa2bfff700 1 --2- 172.21.15.203:0/1987661776 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faa0c0563d0 0x7faa0c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:02.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.590+0000 7faa33fff700 1 --2- 172.21.15.203:0/1987661776 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faa0c0563d0 0x7faa0c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:02.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.590+0000 7faa2bfff700 1 -- 172.21.15.203:0/1987661776 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6331+0+0 (secure 0 0 0) 0x7faa2c06eba0 con 0x7faa34155b60 2021-01-06T22:10:02.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.590+0000 7faa33fff700 1 --2- 172.21.15.203:0/1987661776 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faa0c0563d0 0x7faa0c058890 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7faa1c0069e0 tx=0x7faa1c006ea0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:02.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.594+0000 7faa2bfff700 1 -- 172.21.15.203:0/1987661776 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7faa2c044320 con 0x7faa34155b60 2021-01-06T22:10:02.798 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.798+0000 7faa3a2f6700 1 -- 172.21.15.203:0/1987661776 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa34064e60 con 0x7faa34155b60 2021-01-06T22:10:02.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.798+0000 7faa2bfff700 1 -- 172.21.15.203:0/1987661776 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3561 (secure 0 0 0) 0x7faa2c017070 con 0x7faa34155b60 2021-01-06T22:10:02.800 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:10:02.800 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4859,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:55.422280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:10:02.801 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.798+0000 7faa3a2f6700 1 -- 172.21.15.203:0/1987661776 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faa0c0563d0 msgr2=0x7faa0c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:02.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.798+0000 7faa3a2f6700 1 --2- 172.21.15.203:0/1987661776 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faa0c0563d0 0x7faa0c058890 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7faa1c0069e0 tx=0x7faa1c006ea0).stop 2021-01-06T22:10:02.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.798+0000 7faa3a2f6700 1 -- 172.21.15.203:0/1987661776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa34155b60 msgr2=0x7faa340bedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:02.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.798+0000 7faa3a2f6700 1 --2- 172.21.15.203:0/1987661776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa34155b60 0x7faa340bedf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7faa2c00c730 tx=0x7faa2c007a30).stop 2021-01-06T22:10:02.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.798+0000 7faa3a2f6700 1 -- 172.21.15.203:0/1987661776 shutdown_connections 2021-01-06T22:10:02.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.798+0000 7faa3a2f6700 1 --2- 172.21.15.203:0/1987661776 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faa0c0563d0 0x7faa0c058890 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:02.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.798+0000 7faa3a2f6700 1 --2- 172.21.15.203:0/1987661776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa34155b60 0x7faa340bedf0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:02.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.798+0000 7faa3a2f6700 1 --2- 172.21.15.203:0/1987661776 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa340c04c0 0x7faa3414d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:02.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.798+0000 7faa3a2f6700 1 --2- 172.21.15.203:0/1987661776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa340c08a0 0x7faa3414e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:02.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.798+0000 7faa3a2f6700 1 -- 172.21.15.203:0/1987661776 >> 172.21.15.203:0/1987661776 conn(0x7faa34056cd0 msgr2=0x7faa341514d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:02.804 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.798+0000 7faa3a2f6700 1 -- 172.21.15.203:0/1987661776 shutdown_connections 2021-01-06T22:10:02.804 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:02.798+0000 7faa3a2f6700 1 -- 172.21.15.203:0/1987661776 wait complete. 2021-01-06T22:10:02.805 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 38 2021-01-06T22:10:02.814 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:10:04.815 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:10:04.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.942+0000 7fa43fbd2700 1 Processor -- start 2021-01-06T22:10:04.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.942+0000 7fa43fbd2700 1 -- start start 2021-01-06T22:10:04.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.942+0000 7fa43fbd2700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4380b9c00 0x7fa43814c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:04.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43fbd2700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa4380b9fe0 0x7fa4380bbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:04.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43fbd2700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4380c0320 con 0x7fa4380bc0e0 2021-01-06T22:10:04.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43fbd2700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4380c11d0 con 0x7fa4380b9fe0 2021-01-06T22:10:04.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43fbd2700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa438068040 con 0x7fa4380b9c00 2021-01-06T22:10:04.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43d16d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa4380b9fe0 0x7fa4380bbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:04.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43d16d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa4380b9fe0 0x7fa4380bbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50912/0 (socket says 172.21.15.203:50912) 2021-01-06T22:10:04.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43d16d700 1 -- 172.21.15.203:0/3596466711 learned_addr learned my addr 172.21.15.203:0/3596466711 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:10:04.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43d96e700 1 --2- 172.21.15.203:0/3596466711 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4380b9c00 0x7fa43814c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:04.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43d16d700 1 -- 172.21.15.203:0/3596466711 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4380b9c00 msgr2=0x7fa43814c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:10:04.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43d16d700 1 --2- 172.21.15.203:0/3596466711 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4380b9c00 0x7fa43814c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:04.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43d16d700 1 -- 172.21.15.203:0/3596466711 >> v1:172.21.15.91:6789/0 conn(0x7fa4380bc0e0 legacy=0x7fa438067540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:10:04.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43d16d700 1 -- 172.21.15.203:0/3596466711 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4380c0550 con 0x7fa4380b9fe0 2021-01-06T22:10:04.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43d96e700 1 --2- 172.21.15.203:0/3596466711 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4380b9c00 0x7fa43814c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:10:04.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43d16d700 1 --2- 172.21.15.203:0/3596466711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4380b9fe0 0x7fa4380bbbf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fa42800b9e0 tx=0x7fa428006ea0).ready entity=mon.1 client_cookie=8138a53ad6ef0280 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:04.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43c96c700 1 -- 172.21.15.203:0/3596466711 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428009960 con 0x7fa4380b9fe0 2021-01-06T22:10:04.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43c96c700 1 -- 172.21.15.203:0/3596466711 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa428009ac0 con 0x7fa4380b9fe0 2021-01-06T22:10:04.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43c96c700 1 -- 172.21.15.203:0/3596466711 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428009e40 con 0x7fa4380b9fe0 2021-01-06T22:10:04.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43fbd2700 1 -- 172.21.15.203:0/3596466711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4380b9fe0 msgr2=0x7fa4380bbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:04.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43fbd2700 1 --2- 172.21.15.203:0/3596466711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4380b9fe0 0x7fa4380bbbf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fa42800b9e0 tx=0x7fa428006ea0).stop 2021-01-06T22:10:04.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43fbd2700 1 -- 172.21.15.203:0/3596466711 shutdown_connections 2021-01-06T22:10:04.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43fbd2700 1 --2- 172.21.15.203:0/3596466711 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4380b9c00 0x7fa43814c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:04.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43fbd2700 1 --2- 172.21.15.203:0/3596466711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4380b9fe0 0x7fa4380bbbf0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:04.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43fbd2700 1 -- 172.21.15.203:0/3596466711 >> 172.21.15.203:0/3596466711 conn(0x7fa438056cd0 msgr2=0x7fa4380626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:04.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43fbd2700 1 -- 172.21.15.203:0/3596466711 shutdown_connections 2021-01-06T22:10:04.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.946+0000 7fa43fbd2700 1 -- 172.21.15.203:0/3596466711 wait complete. 2021-01-06T22:10:04.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43fbd2700 1 Processor -- start 2021-01-06T22:10:04.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43fbd2700 1 -- start start 2021-01-06T22:10:04.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43fbd2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4380b9c00 0x7fa43815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:04.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43fbd2700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4380b9fe0 0x7fa43815d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:04.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43fbd2700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4380bc0e0 0x7fa43815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:04.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43fbd2700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43815d750 con 0x7fa4380bc0e0 2021-01-06T22:10:04.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43fbd2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4380ae550 con 0x7fa4380b9c00 2021-01-06T22:10:04.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43fbd2700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4380ae7b0 con 0x7fa4380b9fe0 2021-01-06T22:10:04.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43d96e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4380b9c00 0x7fa43815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:04.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43d96e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4380b9c00 0x7fa43815cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50918/0 (socket says 172.21.15.203:50918) 2021-01-06T22:10:04.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43d96e700 1 -- 172.21.15.203:0/1325443888 learned_addr learned my addr 172.21.15.203:0/1325443888 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:10:04.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43d16d700 1 --2- 172.21.15.203:0/1325443888 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4380b9fe0 0x7fa43815d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:04.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43e16f700 1 --2- 172.21.15.203:0/1325443888 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4380bc0e0 0x7fa43815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:04.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43d96e700 1 -- 172.21.15.203:0/1325443888 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4380b9fe0 msgr2=0x7fa43815d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:04.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43d96e700 1 --2- 172.21.15.203:0/1325443888 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4380b9fe0 0x7fa43815d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:04.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43d96e700 1 -- 172.21.15.203:0/1325443888 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4380bc0e0 msgr2=0x7fa43815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:04.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43d96e700 1 --2- 172.21.15.203:0/1325443888 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4380bc0e0 0x7fa43815efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:04.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43d96e700 1 -- 172.21.15.203:0/1325443888 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4380aea50 con 0x7fa4380b9c00 2021-01-06T22:10:04.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43d96e700 1 --2- 172.21.15.203:0/1325443888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4380b9c00 0x7fa43815cd70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fa43400db90 tx=0x7fa434008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:04.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa425ffb700 1 -- 172.21.15.203:0/1325443888 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa434009790 con 0x7fa4380b9c00 2021-01-06T22:10:04.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa425ffb700 1 -- 172.21.15.203:0/1325443888 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4340098f0 con 0x7fa4380b9c00 2021-01-06T22:10:04.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43fbd2700 1 -- 172.21.15.203:0/1325443888 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4380affb0 con 0x7fa4380b9c00 2021-01-06T22:10:04.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa425ffb700 1 -- 172.21.15.203:0/1325443888 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa434009c90 con 0x7fa4380b9c00 2021-01-06T22:10:04.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.950+0000 7fa43fbd2700 1 -- 172.21.15.203:0/1325443888 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4380b04f0 con 0x7fa4380b9c00 2021-01-06T22:10:04.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.954+0000 7fa43fbd2700 1 -- 172.21.15.203:0/1325443888 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4380c0730 con 0x7fa4380b9c00 2021-01-06T22:10:04.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.954+0000 7fa425ffb700 1 -- 172.21.15.203:0/1325443888 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa434013480 con 0x7fa4380b9c00 2021-01-06T22:10:04.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.954+0000 7fa425ffb700 1 --2- 172.21.15.203:0/1325443888 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa4180563d0 0x7fa418058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:04.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.954+0000 7fa425ffb700 1 -- 172.21.15.203:0/1325443888 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fa43406f910 con 0x7fa4380b9c00 2021-01-06T22:10:04.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.954+0000 7fa43d16d700 1 --2- 172.21.15.203:0/1325443888 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa4180563d0 0x7fa418058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:04.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.954+0000 7fa43d16d700 1 --2- 172.21.15.203:0/1325443888 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa4180563d0 0x7fa418058890 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fa438063000 tx=0x7fa428016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:04.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:04.962+0000 7fa425ffb700 1 -- 172.21.15.203:0/1325443888 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa434045030 con 0x7fa4380b9c00 2021-01-06T22:10:05.170 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:05.170+0000 7fa43fbd2700 1 -- 172.21.15.203:0/1325443888 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa43805cfe0 con 0x7fa4380b9c00 2021-01-06T22:10:05.171 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:05.170+0000 7fa425ffb700 1 -- 172.21.15.203:0/1325443888 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3561 (secure 0 0 0) 0x7fa434048650 con 0x7fa4380b9c00 2021-01-06T22:10:05.171 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:10:05.171 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4859,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:55.422280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:10:05.173 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:05.170+0000 7fa43fbd2700 1 -- 172.21.15.203:0/1325443888 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa4180563d0 msgr2=0x7fa418058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:05.173 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:05.170+0000 7fa43fbd2700 1 --2- 172.21.15.203:0/1325443888 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa4180563d0 0x7fa418058890 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fa438063000 tx=0x7fa428016000).stop 2021-01-06T22:10:05.174 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:05.170+0000 7fa43fbd2700 1 -- 172.21.15.203:0/1325443888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4380b9c00 msgr2=0x7fa43815cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:05.174 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:05.170+0000 7fa43fbd2700 1 --2- 172.21.15.203:0/1325443888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4380b9c00 0x7fa43815cd70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fa43400db90 tx=0x7fa434008040).stop 2021-01-06T22:10:05.174 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:05.170+0000 7fa43fbd2700 1 -- 172.21.15.203:0/1325443888 shutdown_connections 2021-01-06T22:10:05.175 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:05.170+0000 7fa43fbd2700 1 --2- 172.21.15.203:0/1325443888 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa4180563d0 0x7fa418058890 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:05.175 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:05.170+0000 7fa43fbd2700 1 --2- 172.21.15.203:0/1325443888 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4380bc0e0 0x7fa43815efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:05.175 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:05.170+0000 7fa43fbd2700 1 --2- 172.21.15.203:0/1325443888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4380b9c00 0x7fa43815cd70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:05.176 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:05.170+0000 7fa43fbd2700 1 --2- 172.21.15.203:0/1325443888 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4380b9fe0 0x7fa43815d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:05.176 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:05.170+0000 7fa43fbd2700 1 -- 172.21.15.203:0/1325443888 >> 172.21.15.203:0/1325443888 conn(0x7fa438056cd0 msgr2=0x7fa4380638b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:05.176 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:05.170+0000 7fa43fbd2700 1 -- 172.21.15.203:0/1325443888 shutdown_connections 2021-01-06T22:10:05.176 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:05.170+0000 7fa43fbd2700 1 -- 172.21.15.203:0/1325443888 wait complete. 2021-01-06T22:10:05.177 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 38 2021-01-06T22:10:05.187 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:10:07.187 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:10:07.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e627e0700 1 Processor -- start 2021-01-06T22:10:07.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e627e0700 1 -- start start 2021-01-06T22:10:07.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e627e0700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7e5c0c04c0 0x7f7e5c0680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:07.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e627e0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0c08a0 0x7f7e5c063ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:07.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e627e0700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5c1531f0 con 0x7f7e5c155b60 2021-01-06T22:10:07.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e627e0700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e5c1522e0 con 0x7f7e5c0c04c0 2021-01-06T22:10:07.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e627e0700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c063a10 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e60d7d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f7e5c155b60 0x7f7e5c14f310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49520/0 (socket says 172.21.15.203:49520) 2021-01-06T22:10:07.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e60d7d700 1 -- 172.21.15.203:0/1383519352 learned_addr learned my addr 172.21.15.203:0/1383519352 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:10:07.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e5bfff700 1 --2- 172.21.15.203:0/1383519352 >> v2:172.21.15.203:3300/0 conn(0x7f7e5c0c04c0 0x7f7e5c0680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:07.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e5b7fe700 1 --2- 172.21.15.203:0/1383519352 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0c08a0 0x7f7e5c063ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:07.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e5affd700 1 -- 172.21.15.203:0/1383519352 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312341416 0 0) 0x7f7e5c1531f0 con 0x7f7e5c155b60 2021-01-06T22:10:07.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e5affd700 1 -- 172.21.15.203:0/1383519352 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e40003230 con 0x7f7e5c155b60 2021-01-06T22:10:07.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e5affd700 1 -- 172.21.15.203:0/1383519352 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1876109573 0 0) 0x7f7e40003230 con 0x7f7e5c155b60 2021-01-06T22:10:07.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e5b7fe700 1 -- 172.21.15.203:0/1383519352 >> v2:172.21.15.203:3300/0 conn(0x7f7e5c0c04c0 msgr2=0x7f7e5c0680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:10:07.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e5b7fe700 1 --2- 172.21.15.203:0/1383519352 >> v2:172.21.15.203:3300/0 conn(0x7f7e5c0c04c0 0x7f7e5c0680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:07.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e5b7fe700 1 -- 172.21.15.203:0/1383519352 >> v1:172.21.15.203:6789/0 conn(0x7f7e5c155b60 legacy=0x7f7e5c14f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:07.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e5b7fe700 1 -- 172.21.15.203:0/1383519352 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c1524d0 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.398+0000 7f7e5bfff700 1 --2- 172.21.15.203:0/1383519352 >> v2:172.21.15.203:3300/0 conn(0x7f7e5c0c04c0 0x7f7e5c0680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:10:07.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.402+0000 7f7e5b7fe700 1 --2- 172.21.15.203:0/1383519352 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0c08a0 0x7f7e5c063ee0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f7e4400b980 tx=0x7f7e44006de0).ready entity=mon.2 client_cookie=f959ca19866349c7 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:07.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.402+0000 7f7e5affd700 1 -- 172.21.15.203:0/1383519352 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e540014e0 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.402+0000 7f7e5affd700 1 -- 172.21.15.203:0/1383519352 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e44011400 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.402+0000 7f7e5affd700 1 -- 172.21.15.203:0/1383519352 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e44011780 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.402+0000 7f7e627e0700 1 -- 172.21.15.203:0/1383519352 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0c08a0 msgr2=0x7f7e5c063ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:07.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.402+0000 7f7e627e0700 1 --2- 172.21.15.203:0/1383519352 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0c08a0 0x7f7e5c063ee0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f7e4400b980 tx=0x7f7e44006de0).stop 2021-01-06T22:10:07.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.402+0000 7f7e627e0700 1 -- 172.21.15.203:0/1383519352 shutdown_connections 2021-01-06T22:10:07.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.402+0000 7f7e627e0700 1 --2- 172.21.15.203:0/1383519352 >> v2:172.21.15.203:3300/0 conn(0x7f7e5c0c04c0 0x7f7e5c0680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:07.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.402+0000 7f7e627e0700 1 --2- 172.21.15.203:0/1383519352 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c0c08a0 0x7f7e5c063ee0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:07.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.402+0000 7f7e627e0700 1 -- 172.21.15.203:0/1383519352 >> 172.21.15.203:0/1383519352 conn(0x7f7e5c056cd0 msgr2=0x7f7e5c0626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:07.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.402+0000 7f7e627e0700 1 -- 172.21.15.203:0/1383519352 shutdown_connections 2021-01-06T22:10:07.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.402+0000 7f7e627e0700 1 -- 172.21.15.203:0/1383519352 wait complete. 2021-01-06T22:10:07.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.402+0000 7f7e627e0700 1 Processor -- start 2021-01-06T22:10:07.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.402+0000 7f7e627e0700 1 -- start start 2021-01-06T22:10:07.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e627e0700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e5c0c04c0 0x7f7e5c0b93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:07.417 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e627e0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c0c08a0 0x7f7e5c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:07.417 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e627e0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c155b60 0x7f7e5c14bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:07.417 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e627e0700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c14eb60 con 0x7f7e5c0c04c0 2021-01-06T22:10:07.418 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e627e0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c165c40 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.418 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e627e0700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c166110 con 0x7f7e5c155b60 2021-01-06T22:10:07.418 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e5b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c0c08a0 0x7f7e5c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:07.419 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e5b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c0c08a0 0x7f7e5c0b89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50930/0 (socket says 172.21.15.203:50930) 2021-01-06T22:10:07.419 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e5b7fe700 1 -- 172.21.15.203:0/375746006 learned_addr learned my addr 172.21.15.203:0/375746006 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:10:07.419 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e5bfff700 1 --2- 172.21.15.203:0/375746006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e5c0c04c0 0x7f7e5c0b93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:07.420 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e60d7d700 1 --2- 172.21.15.203:0/375746006 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c155b60 0x7f7e5c14bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:07.420 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e5b7fe700 1 -- 172.21.15.203:0/375746006 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c155b60 msgr2=0x7f7e5c14bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:07.420 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e5b7fe700 1 --2- 172.21.15.203:0/375746006 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c155b60 0x7f7e5c14bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:07.421 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e5b7fe700 1 -- 172.21.15.203:0/375746006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e5c0c04c0 msgr2=0x7f7e5c0b93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:07.421 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e5b7fe700 1 --2- 172.21.15.203:0/375746006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e5c0c04c0 0x7f7e5c0b93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:07.421 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e5b7fe700 1 -- 172.21.15.203:0/375746006 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c14e490 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e5bfff700 1 --2- 172.21.15.203:0/375746006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e5c0c04c0 0x7f7e5c0b93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:10:07.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e60d7d700 1 --2- 172.21.15.203:0/375746006 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c155b60 0x7f7e5c14bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:10:07.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e5b7fe700 1 --2- 172.21.15.203:0/375746006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c0c08a0 0x7f7e5c0b89e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f7e5c063b50 tx=0x7f7e44016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e53fff700 1 -- 172.21.15.203:0/375746006 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e44010bc0 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e53fff700 1 -- 172.21.15.203:0/375746006 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e44010d20 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e627e0700 1 -- 172.21.15.203:0/375746006 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5c165e70 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.424 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e53fff700 1 -- 172.21.15.203:0/375746006 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4401d640 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.424 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e627e0700 1 -- 172.21.15.203:0/375746006 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5c14d950 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.424 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e627e0700 1 -- 172.21.15.203:0/375746006 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5c05cfe0 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.424 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e53fff700 1 -- 172.21.15.203:0/375746006 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f7e4400f070 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.426 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e53fff700 1 --2- 172.21.15.203:0/375746006 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e340563d0 0x7f7e34058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:07.426 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e53fff700 1 -- 172.21.15.203:0/375746006 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f7e44070ba0 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.427 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.406+0000 7f7e5bfff700 1 --2- 172.21.15.203:0/375746006 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e340563d0 0x7f7e34058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:07.427 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.410+0000 7f7e5bfff700 1 --2- 172.21.15.203:0/375746006 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e340563d0 0x7f7e34058890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f7e4c009f30 tx=0x7f7e4c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:07.428 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.414+0000 7f7e53fff700 1 -- 172.21.15.203:0/375746006 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f7e44046320 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.618+0000 7f7e627e0700 1 -- 172.21.15.203:0/375746006 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e5c064e60 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.618+0000 7f7e53fff700 1 -- 172.21.15.203:0/375746006 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3561 (secure 0 0 0) 0x7f7e44049940 con 0x7f7e5c0c08a0 2021-01-06T22:10:07.621 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:10:07.621 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4859,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:55.422280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:10:07.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.622+0000 7f7e627e0700 1 -- 172.21.15.203:0/375746006 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e340563d0 msgr2=0x7f7e34058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:07.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.622+0000 7f7e627e0700 1 --2- 172.21.15.203:0/375746006 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e340563d0 0x7f7e34058890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f7e4c009f30 tx=0x7f7e4c008040).stop 2021-01-06T22:10:07.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.622+0000 7f7e627e0700 1 -- 172.21.15.203:0/375746006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c0c08a0 msgr2=0x7f7e5c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:07.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.622+0000 7f7e627e0700 1 --2- 172.21.15.203:0/375746006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c0c08a0 0x7f7e5c0b89e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f7e5c063b50 tx=0x7f7e44016000).stop 2021-01-06T22:10:07.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.622+0000 7f7e627e0700 1 -- 172.21.15.203:0/375746006 shutdown_connections 2021-01-06T22:10:07.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.622+0000 7f7e627e0700 1 --2- 172.21.15.203:0/375746006 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7e340563d0 0x7f7e34058890 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:07.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.622+0000 7f7e627e0700 1 --2- 172.21.15.203:0/375746006 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e5c155b60 0x7f7e5c14bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:07.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.622+0000 7f7e627e0700 1 --2- 172.21.15.203:0/375746006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e5c0c04c0 0x7f7e5c0b93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:07.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.622+0000 7f7e627e0700 1 --2- 172.21.15.203:0/375746006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e5c0c08a0 0x7f7e5c0b89e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:07.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.622+0000 7f7e627e0700 1 -- 172.21.15.203:0/375746006 >> 172.21.15.203:0/375746006 conn(0x7f7e5c056cd0 msgr2=0x7f7e5c0626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:07.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.622+0000 7f7e627e0700 1 -- 172.21.15.203:0/375746006 shutdown_connections 2021-01-06T22:10:07.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:07.622+0000 7f7e627e0700 1 -- 172.21.15.203:0/375746006 wait complete. 2021-01-06T22:10:07.629 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 38 2021-01-06T22:10:07.642 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:10:09.643 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:10:09.774 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.769+0000 7f301bfff700 1 Processor -- start 2021-01-06T22:10:09.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.769+0000 7f301bfff700 1 -- start start 2021-01-06T22:10:09.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301bfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f301c0a5680 0x7f301c14fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:09.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301bfff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f301c060fe0 0x7f301c064340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:09.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301bfff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f301c067660 con 0x7f301c0a5a60 2021-01-06T22:10:09.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301bfff700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f301c064ab0 con 0x7f301c060fe0 2021-01-06T22:10:09.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301bfff700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f301c1500f0 con 0x7f301c0a5680 2021-01-06T22:10:09.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301affd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f301c0a5680 0x7f301c14fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:09.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301affd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f301c0a5680 0x7f301c14fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50942/0 (socket says 172.21.15.203:50942) 2021-01-06T22:10:09.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301affd700 1 -- 172.21.15.203:0/2924701059 learned_addr learned my addr 172.21.15.203:0/2924701059 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:10:09.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301b7fe700 1 --2- 172.21.15.203:0/2924701059 >> v2:172.21.15.91:3300/0 conn(0x7f301c060fe0 0x7f301c064340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:09.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301affd700 1 -- 172.21.15.203:0/2924701059 >> v2:172.21.15.91:3300/0 conn(0x7f301c060fe0 msgr2=0x7f301c064340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:10:09.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301affd700 1 --2- 172.21.15.203:0/2924701059 >> v2:172.21.15.91:3300/0 conn(0x7f301c060fe0 0x7f301c064340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:09.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301affd700 1 -- 172.21.15.203:0/2924701059 >> v1:172.21.15.91:6789/0 conn(0x7f301c0a5a60 legacy=0x7f301c0608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:09.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301affd700 1 -- 172.21.15.203:0/2924701059 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c064830 con 0x7f301c0a5680 2021-01-06T22:10:09.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301affd700 1 --2- 172.21.15.203:0/2924701059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f301c0a5680 0x7f301c14fc00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f301c0c1040 tx=0x7f300c006ea0).ready entity=mon.1 client_cookie=ee0dda35e6b0cad server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:09.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f3019ffb700 1 -- 172.21.15.203:0/2924701059 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c0098a0 con 0x7f301c0a5680 2021-01-06T22:10:09.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f3019ffb700 1 -- 172.21.15.203:0/2924701059 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300c009a00 con 0x7f301c0a5680 2021-01-06T22:10:09.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f3019ffb700 1 -- 172.21.15.203:0/2924701059 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c009d80 con 0x7f301c0a5680 2021-01-06T22:10:09.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301bfff700 1 -- 172.21.15.203:0/2924701059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f301c0a5680 msgr2=0x7f301c14fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:09.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301bfff700 1 --2- 172.21.15.203:0/2924701059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f301c0a5680 0x7f301c14fc00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f301c0c1040 tx=0x7f300c006ea0).stop 2021-01-06T22:10:09.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301bfff700 1 -- 172.21.15.203:0/2924701059 shutdown_connections 2021-01-06T22:10:09.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301bfff700 1 --2- 172.21.15.203:0/2924701059 >> v2:172.21.15.91:3300/0 conn(0x7f301c060fe0 0x7f301c064340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:09.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301bfff700 1 --2- 172.21.15.203:0/2924701059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f301c0a5680 0x7f301c14fc00 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:09.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301bfff700 1 -- 172.21.15.203:0/2924701059 >> 172.21.15.203:0/2924701059 conn(0x7f301c061530 msgr2=0x7f301c063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:09.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301bfff700 1 -- 172.21.15.203:0/2924701059 shutdown_connections 2021-01-06T22:10:09.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.773+0000 7f301bfff700 1 -- 172.21.15.203:0/2924701059 wait complete. 2021-01-06T22:10:09.787 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301bfff700 1 Processor -- start 2021-01-06T22:10:09.787 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301bfff700 1 -- start start 2021-01-06T22:10:09.787 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301bfff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f301c060fe0 0x7f301c143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:09.788 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f301c0a5680 0x7f301c0b9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:09.788 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301bfff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f301c0a5a60 0x7f301c140400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:09.788 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301bfff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c0b8760 con 0x7f301c060fe0 2021-01-06T22:10:09.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301bfff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c143240 con 0x7f301c0a5680 2021-01-06T22:10:09.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301bfff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f301c1434e0 con 0x7f301c0a5a60 2021-01-06T22:10:09.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301a7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f301c0a5680 0x7f301c0b9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:09.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301a7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f301c0a5680 0x7f301c0b9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50950/0 (socket says 172.21.15.203:50950) 2021-01-06T22:10:09.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301a7fc700 1 -- 172.21.15.203:0/3694987489 learned_addr learned my addr 172.21.15.203:0/3694987489 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:10:09.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301b7fe700 1 --2- 172.21.15.203:0/3694987489 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f301c0a5a60 0x7f301c140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:09.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301affd700 1 --2- 172.21.15.203:0/3694987489 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f301c060fe0 0x7f301c143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:09.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301a7fc700 1 -- 172.21.15.203:0/3694987489 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f301c0a5a60 msgr2=0x7f301c140400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:09.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301a7fc700 1 --2- 172.21.15.203:0/3694987489 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f301c0a5a60 0x7f301c140400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:09.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301a7fc700 1 -- 172.21.15.203:0/3694987489 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f301c060fe0 msgr2=0x7f301c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:09.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301a7fc700 1 --2- 172.21.15.203:0/3694987489 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f301c060fe0 0x7f301c143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:09.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301a7fc700 1 -- 172.21.15.203:0/3694987489 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c143780 con 0x7f301c0a5680 2021-01-06T22:10:09.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301a7fc700 1 --2- 172.21.15.203:0/3694987489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f301c0a5680 0x7f301c0b9cd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f300400c740 tx=0x7f3004007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:09.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f3012ffd700 1 -- 172.21.15.203:0/3694987489 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004007020 con 0x7f301c0a5680 2021-01-06T22:10:09.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f3012ffd700 1 -- 172.21.15.203:0/3694987489 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3004007180 con 0x7f301c0a5680 2021-01-06T22:10:09.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301bfff700 1 -- 172.21.15.203:0/3694987489 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f301c0b9590 con 0x7f301c0a5680 2021-01-06T22:10:09.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f3012ffd700 1 -- 172.21.15.203:0/3694987489 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30040064e0 con 0x7f301c0a5680 2021-01-06T22:10:09.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.777+0000 7f301bfff700 1 -- 172.21.15.203:0/3694987489 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f301c145060 con 0x7f301c0a5680 2021-01-06T22:10:09.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.781+0000 7f301bfff700 1 -- 172.21.15.203:0/3694987489 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301c0bc520 con 0x7f301c0a5680 2021-01-06T22:10:09.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.781+0000 7f3012ffd700 1 -- 172.21.15.203:0/3694987489 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f300401a060 con 0x7f301c0a5680 2021-01-06T22:10:09.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.781+0000 7f3012ffd700 1 --2- 172.21.15.203:0/3694987489 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3000056400 0x7f30000588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:09.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.781+0000 7f3012ffd700 1 -- 172.21.15.203:0/3694987489 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f300406ee00 con 0x7f301c0a5680 2021-01-06T22:10:09.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.781+0000 7f301affd700 1 --2- 172.21.15.203:0/3694987489 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3000056400 0x7f30000588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:09.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.781+0000 7f301affd700 1 --2- 172.21.15.203:0/3694987489 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3000056400 0x7f30000588c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f300c009fd0 tx=0x7f300c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:09.796 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.785+0000 7f3012ffd700 1 -- 172.21.15.203:0/3694987489 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f301c0bc520 con 0x7f301c0a5680 2021-01-06T22:10:09.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.989+0000 7f301bfff700 1 -- 172.21.15.203:0/3694987489 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f301c0450a0 con 0x7f301c0a5680 2021-01-06T22:10:09.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.989+0000 7f3012ffd700 1 -- 172.21.15.203:0/3694987489 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3561 (secure 0 0 0) 0x7f3004018020 con 0x7f301c0a5680 2021-01-06T22:10:09.995 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:10:09.995 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4859,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:08:55.422280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4274,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4274":{"gid":4274,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.91:6837/3888667247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3888667247},{"type":"v1","addr":"172.21.15.91:6837","nonce":3888667247}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:10:09.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.993+0000 7f301bfff700 1 -- 172.21.15.203:0/3694987489 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3000056400 msgr2=0x7f30000588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:09.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.993+0000 7f301bfff700 1 --2- 172.21.15.203:0/3694987489 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3000056400 0x7f30000588c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f300c009fd0 tx=0x7f300c016000).stop 2021-01-06T22:10:09.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.993+0000 7f301bfff700 1 -- 172.21.15.203:0/3694987489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f301c0a5680 msgr2=0x7f301c0b9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:09.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.993+0000 7f301bfff700 1 --2- 172.21.15.203:0/3694987489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f301c0a5680 0x7f301c0b9cd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f300400c740 tx=0x7f3004007c10).stop 2021-01-06T22:10:09.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.993+0000 7f301bfff700 1 -- 172.21.15.203:0/3694987489 shutdown_connections 2021-01-06T22:10:09.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.993+0000 7f301bfff700 1 --2- 172.21.15.203:0/3694987489 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3000056400 0x7f30000588c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:09.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.993+0000 7f301bfff700 1 --2- 172.21.15.203:0/3694987489 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f301c0a5a60 0x7f301c140400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:09.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.993+0000 7f301bfff700 1 --2- 172.21.15.203:0/3694987489 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f301c060fe0 0x7f301c143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:09.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.993+0000 7f301bfff700 1 --2- 172.21.15.203:0/3694987489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f301c0a5680 0x7f301c0b9cd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:09.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.993+0000 7f301bfff700 1 -- 172.21.15.203:0/3694987489 >> 172.21.15.203:0/3694987489 conn(0x7f301c061530 msgr2=0x7f301c0bee00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:09.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.993+0000 7f301bfff700 1 -- 172.21.15.203:0/3694987489 shutdown_connections 2021-01-06T22:10:09.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:09.993+0000 7f301bfff700 1 -- 172.21.15.203:0/3694987489 wait complete. 2021-01-06T22:10:10.000 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 38 2021-01-06T22:10:10.014 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:10:12.015 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:10:12.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.141+0000 7f1bcb59e700 1 Processor -- start 2021-01-06T22:10:12.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.141+0000 7f1bcb59e700 1 -- start start 2021-01-06T22:10:12.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bcb59e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1bcc0b9b80 0x7f1bcc065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:12.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bcb59e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1bcc0bed80 0x7f1bcc14c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:12.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bcb59e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bcc0be410 con 0x7f1bcc0b97a0 2021-01-06T22:10:12.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bcb59e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bcc066510 con 0x7f1bcc0bed80 2021-01-06T22:10:12.148 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bcb59e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bcc0667b0 con 0x7f1bcc0b9b80 2021-01-06T22:10:12.148 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bc9d9b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1bcc0b9b80 0x7f1bcc065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:12.148 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bca59c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f1bcc0b97a0 0x7f1bcc0bd420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49558/0 (socket says 172.21.15.203:49558) 2021-01-06T22:10:12.148 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bca59c700 1 -- 172.21.15.203:0/3607859132 learned_addr learned my addr 172.21.15.203:0/3607859132 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:10:12.149 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bcad9d700 1 --2- 172.21.15.203:0/3607859132 >> v2:172.21.15.91:3300/0 conn(0x7f1bcc0bed80 0x7f1bcc14c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:12.150 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bc959a700 1 -- 172.21.15.203:0/3607859132 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 461434207 0 0) 0x7f1bcc0be410 con 0x7f1bcc0b97a0 2021-01-06T22:10:12.151 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bc959a700 1 -- 172.21.15.203:0/3607859132 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bb0003230 con 0x7f1bcc0b97a0 2021-01-06T22:10:12.151 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bc9d9b700 1 -- 172.21.15.203:0/3607859132 >> v2:172.21.15.91:3300/0 conn(0x7f1bcc0bed80 msgr2=0x7f1bcc14c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:10:12.151 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bc9d9b700 1 --2- 172.21.15.203:0/3607859132 >> v2:172.21.15.91:3300/0 conn(0x7f1bcc0bed80 0x7f1bcc14c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.151 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bc9d9b700 1 -- 172.21.15.203:0/3607859132 >> v1:172.21.15.203:6789/0 conn(0x7f1bcc0b97a0 legacy=0x7f1bcc0bd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:12.151 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bc9d9b700 1 -- 172.21.15.203:0/3607859132 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc066a50 con 0x7f1bcc0b9b80 2021-01-06T22:10:12.152 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bc9d9b700 1 --2- 172.21.15.203:0/3607859132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bcc0b9b80 0x7f1bcc065c80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f1bcc066e40 tx=0x7f1bc0006de0).ready entity=mon.1 client_cookie=5b2b510084fa0fc2 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:12.152 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bc959a700 1 -- 172.21.15.203:0/3607859132 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc00096d0 con 0x7f1bcc0b9b80 2021-01-06T22:10:12.152 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bc959a700 1 -- 172.21.15.203:0/3607859132 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc0009830 con 0x7f1bcc0b9b80 2021-01-06T22:10:12.152 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bc959a700 1 -- 172.21.15.203:0/3607859132 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0009bb0 con 0x7f1bcc0b9b80 2021-01-06T22:10:12.152 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bcb59e700 1 -- 172.21.15.203:0/3607859132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bcc0b9b80 msgr2=0x7f1bcc065c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:12.153 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bcb59e700 1 --2- 172.21.15.203:0/3607859132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bcc0b9b80 0x7f1bcc065c80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f1bcc066e40 tx=0x7f1bc0006de0).stop 2021-01-06T22:10:12.153 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bcb59e700 1 -- 172.21.15.203:0/3607859132 shutdown_connections 2021-01-06T22:10:12.153 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bcb59e700 1 --2- 172.21.15.203:0/3607859132 >> v2:172.21.15.91:3300/0 conn(0x7f1bcc0bed80 0x7f1bcc14c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.153 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bcb59e700 1 --2- 172.21.15.203:0/3607859132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bcc0b9b80 0x7f1bcc065c80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.154 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bcb59e700 1 -- 172.21.15.203:0/3607859132 >> 172.21.15.203:0/3607859132 conn(0x7f1bcc0648f0 msgr2=0x7f1bcc064cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:12.154 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bcb59e700 1 -- 172.21.15.203:0/3607859132 shutdown_connections 2021-01-06T22:10:12.154 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.145+0000 7f1bcb59e700 1 -- 172.21.15.203:0/3607859132 wait complete. 2021-01-06T22:10:12.156 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcb59e700 1 Processor -- start 2021-01-06T22:10:12.156 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcb59e700 1 -- start start 2021-01-06T22:10:12.157 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcb59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bcc0b97a0 0x7f1bcc14c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:12.157 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcb59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1bcc0b9b80 0x7f1bcc063220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:12.157 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcb59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bcc0bed80 0x7f1bcc148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:12.157 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcb59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc146520 con 0x7f1bcc0b97a0 2021-01-06T22:10:12.157 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcb59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc147d40 con 0x7f1bcc0bed80 2021-01-06T22:10:12.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcb59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc147fe0 con 0x7f1bcc0b9b80 2021-01-06T22:10:12.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bcc0bed80 0x7f1bcc148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:12.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bcc0bed80 0x7f1bcc148f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50972/0 (socket says 172.21.15.203:50972) 2021-01-06T22:10:12.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcad9d700 1 -- 172.21.15.203:0/3296544348 learned_addr learned my addr 172.21.15.203:0/3296544348 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:10:12.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bc9d9b700 1 --2- 172.21.15.203:0/3296544348 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1bcc0b9b80 0x7f1bcc063220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:12.159 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bca59c700 1 --2- 172.21.15.203:0/3296544348 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bcc0b97a0 0x7f1bcc14c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:12.159 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcad9d700 1 -- 172.21.15.203:0/3296544348 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1bcc0b9b80 msgr2=0x7f1bcc063220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:12.159 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcad9d700 1 --2- 172.21.15.203:0/3296544348 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1bcc0b9b80 0x7f1bcc063220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.159 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcad9d700 1 -- 172.21.15.203:0/3296544348 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bcc0b97a0 msgr2=0x7f1bcc14c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:12.159 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcad9d700 1 --2- 172.21.15.203:0/3296544348 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bcc0b97a0 0x7f1bcc14c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.160 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcad9d700 1 -- 172.21.15.203:0/3296544348 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc148280 con 0x7f1bcc0bed80 2021-01-06T22:10:12.160 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bca59c700 1 --2- 172.21.15.203:0/3296544348 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bcc0b97a0 0x7f1bcc14c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:10:12.160 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcad9d700 1 --2- 172.21.15.203:0/3296544348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bcc0bed80 0x7f1bcc148f00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f1bc400ed50 tx=0x7f1bc40091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:12.160 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bba7fc700 1 -- 172.21.15.203:0/3296544348 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4009580 con 0x7f1bcc0bed80 2021-01-06T22:10:12.161 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcb59e700 1 -- 172.21.15.203:0/3296544348 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc148520 con 0x7f1bcc0bed80 2021-01-06T22:10:12.161 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bba7fc700 1 -- 172.21.15.203:0/3296544348 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc400cc40 con 0x7f1bcc0bed80 2021-01-06T22:10:12.161 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bba7fc700 1 -- 172.21.15.203:0/3296544348 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc40123f0 con 0x7f1bcc0bed80 2021-01-06T22:10:12.161 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.149+0000 7f1bcb59e700 1 -- 172.21.15.203:0/3296544348 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bcc0a77a0 con 0x7f1bcc0bed80 2021-01-06T22:10:12.161 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.153+0000 7f1bcb59e700 1 -- 172.21.15.203:0/3296544348 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bcc0450a0 con 0x7f1bcc0bed80 2021-01-06T22:10:12.162 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.153+0000 7f1bba7fc700 1 -- 172.21.15.203:0/3296544348 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f1bc4022070 con 0x7f1bcc0bed80 2021-01-06T22:10:12.162 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.153+0000 7f1bba7fc700 1 --2- 172.21.15.203:0/3296544348 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1ba80563d0 0x7f1ba8058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:12.162 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.153+0000 7f1bba7fc700 1 -- 172.21.15.203:0/3296544348 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f1bc406fe10 con 0x7f1bcc0bed80 2021-01-06T22:10:12.162 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.153+0000 7f1bca59c700 1 --2- 172.21.15.203:0/3296544348 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1ba80563d0 0x7f1ba8058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:12.163 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.153+0000 7f1bca59c700 1 --2- 172.21.15.203:0/3296544348 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1ba80563d0 0x7f1ba8058890 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f1bbc007fd0 tx=0x7f1bbc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:12.163 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.161+0000 7f1bba7fc700 1 -- 172.21.15.203:0/3296544348 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f1bc4045530 con 0x7f1bcc0bed80 2021-01-06T22:10:12.368 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.365+0000 7f1bcb59e700 1 -- 172.21.15.203:0/3296544348 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1bcc05ca60 con 0x7f1bcc0bed80 2021-01-06T22:10:12.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.365+0000 7f1bba7fc700 1 -- 172.21.15.203:0/3296544348 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+3209 (secure 0 0 0) 0x7f1bc4016030 con 0x7f1bcc0bed80 2021-01-06T22:10:12.370 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:10:12.370 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:10:11.427877+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4859,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4859":{"gid":4859,"name":"a","rank":0,"incarnation":39,"state":"up:replay","state_seq":2,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:10:12.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.369+0000 7f1bcb59e700 1 -- 172.21.15.203:0/3296544348 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1ba80563d0 msgr2=0x7f1ba8058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:12.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.369+0000 7f1bcb59e700 1 --2- 172.21.15.203:0/3296544348 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1ba80563d0 0x7f1ba8058890 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f1bbc007fd0 tx=0x7f1bbc009040).stop 2021-01-06T22:10:12.373 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.369+0000 7f1bcb59e700 1 -- 172.21.15.203:0/3296544348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bcc0bed80 msgr2=0x7f1bcc148f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:12.373 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.369+0000 7f1bcb59e700 1 --2- 172.21.15.203:0/3296544348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bcc0bed80 0x7f1bcc148f00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f1bc400ed50 tx=0x7f1bc40091d0).stop 2021-01-06T22:10:12.374 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.369+0000 7f1bcb59e700 1 -- 172.21.15.203:0/3296544348 shutdown_connections 2021-01-06T22:10:12.374 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.369+0000 7f1bcb59e700 1 --2- 172.21.15.203:0/3296544348 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1ba80563d0 0x7f1ba8058890 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.374 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.369+0000 7f1bcb59e700 1 --2- 172.21.15.203:0/3296544348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bcc0bed80 0x7f1bcc148f00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.369+0000 7f1bcb59e700 1 --2- 172.21.15.203:0/3296544348 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bcc0b97a0 0x7f1bcc14c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.369+0000 7f1bcb59e700 1 --2- 172.21.15.203:0/3296544348 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1bcc0b9b80 0x7f1bcc063220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.369+0000 7f1bcb59e700 1 -- 172.21.15.203:0/3296544348 >> 172.21.15.203:0/3296544348 conn(0x7f1bcc0648f0 msgr2=0x7f1bcc14f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:12.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.369+0000 7f1bcb59e700 1 -- 172.21.15.203:0/3296544348 shutdown_connections 2021-01-06T22:10:12.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.369+0000 7f1bcb59e700 1 -- 172.21.15.203:0/3296544348 wait complete. 2021-01-06T22:10:12.377 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 39 2021-01-06T22:10:12.385 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2021-01-06T22:10:12.385 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2021-01-06T22:10:12.386 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:10:12.514 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.509+0000 7f1ddf237700 1 Processor -- start 2021-01-06T22:10:12.515 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.509+0000 7f1ddf237700 1 -- start start 2021-01-06T22:10:12.516 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1ddf237700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1dd8064ce0 0x7f1dd80b8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:12.516 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1ddf237700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd80603c0 con 0x7f1dd80b8a70 2021-01-06T22:10:12.517 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1ddf237700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd8057390 con 0x7f1dd8064900 2021-01-06T22:10:12.517 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1ddf237700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd8065300 con 0x7f1dd8064ce0 2021-01-06T22:10:12.517 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1ddcfd3700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f1dd8064900 0x7f1dd8061550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49576/0 (socket says 172.21.15.203:49576) 2021-01-06T22:10:12.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1ddcfd3700 1 -- 172.21.15.203:0/3444671311 learned_addr learned my addr 172.21.15.203:0/3444671311 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:10:12.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 -- 172.21.15.203:0/3444671311 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28098501 0 0) 0x7f1dd8057390 con 0x7f1dd8064900 2021-01-06T22:10:12.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd7fff700 1 --2- 172.21.15.203:0/3444671311 >> v2:172.21.15.91:3300/0 conn(0x7f1dd8064ce0 0x7f1dd80b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:12.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 -- 172.21.15.203:0/3444671311 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1db8003230 con 0x7f1dd8064900 2021-01-06T22:10:12.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 -- 172.21.15.203:0/3444671311 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1555299097 0 0) 0x7f1db8003230 con 0x7f1dd8064900 2021-01-06T22:10:12.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 -- 172.21.15.203:0/3444671311 >> v2:172.21.15.91:3300/0 conn(0x7f1dd8064ce0 msgr2=0x7f1dd80b8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:10:12.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 --2- 172.21.15.203:0/3444671311 >> v2:172.21.15.91:3300/0 conn(0x7f1dd8064ce0 0x7f1dd80b8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 -- 172.21.15.203:0/3444671311 >> v1:172.21.15.91:6789/0 conn(0x7f1dd80b8a70 legacy=0x7f1dd80b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:12.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 -- 172.21.15.203:0/3444671311 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd80570f0 con 0x7f1dd8064900 2021-01-06T22:10:12.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 -- 172.21.15.203:0/3444671311 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f1dc80014e0 con 0x7f1dd8064900 2021-01-06T22:10:12.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 -- 172.21.15.203:0/3444671311 >> v1:172.21.15.203:6789/0 conn(0x7f1dd8064900 legacy=0x7f1dd8061550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:12.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 --2- 172.21.15.203:0/3444671311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db8003720 0x7f1db8003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:12.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 --2- 172.21.15.203:0/3444671311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1db80044d0 0x7f1db8004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:12.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 --2- 172.21.15.203:0/3444671311 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1db8004e40 0x7f1db800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:12.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 -- 172.21.15.203:0/3444671311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd8065300 con 0x7f1db80044d0 2021-01-06T22:10:12.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 -- 172.21.15.203:0/3444671311 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db8004250 con 0x7f1db8003720 2021-01-06T22:10:12.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 -- 172.21.15.203:0/3444671311 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db800bc10 con 0x7f1db8004e40 2021-01-06T22:10:12.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1ddcfd3700 1 --2- 172.21.15.203:0/3444671311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db8003720 0x7f1db8003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:12.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd7fff700 1 --2- 172.21.15.203:0/3444671311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1db80044d0 0x7f1db8004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:12.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1ddd7d4700 1 --2- 172.21.15.203:0/3444671311 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1db8004e40 0x7f1db800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:12.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1ddcfd3700 1 -- 172.21.15.203:0/3444671311 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1db8004e40 msgr2=0x7f1db800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:10:12.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1ddcfd3700 1 --2- 172.21.15.203:0/3444671311 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1db8004e40 0x7f1db800b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1ddcfd3700 1 -- 172.21.15.203:0/3444671311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1db80044d0 msgr2=0x7f1db8004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:10:12.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1ddcfd3700 1 --2- 172.21.15.203:0/3444671311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1db80044d0 0x7f1db8004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1ddcfd3700 1 -- 172.21.15.203:0/3444671311 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db8003230 con 0x7f1db8003720 2021-01-06T22:10:12.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1ddd7d4700 1 -- 172.21.15.203:0/3444671311 reap_dead start 2021-01-06T22:10:12.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1ddcfd3700 1 --2- 172.21.15.203:0/3444671311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db8003720 0x7f1db8003b40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f1dc800c440 tx=0x7f1dd80602b0).ready entity=mon.1 client_cookie=8d371c2f855cd0a0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:12.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 -- 172.21.15.203:0/3444671311 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc800ce00 con 0x7f1db8003720 2021-01-06T22:10:12.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 -- 172.21.15.203:0/3444671311 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc80072e0 con 0x7f1db8003720 2021-01-06T22:10:12.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.513+0000 7f1dd77fe700 1 -- 172.21.15.203:0/3444671311 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80055e0 con 0x7f1db8003720 2021-01-06T22:10:12.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1ddf237700 1 -- 172.21.15.203:0/3444671311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db8003720 msgr2=0x7f1db8003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:12.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1ddf237700 1 --2- 172.21.15.203:0/3444671311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db8003720 0x7f1db8003b40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f1dc800c440 tx=0x7f1dd80602b0).stop 2021-01-06T22:10:12.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1ddf237700 1 -- 172.21.15.203:0/3444671311 shutdown_connections 2021-01-06T22:10:12.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1ddf237700 1 --2- 172.21.15.203:0/3444671311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db8003720 0x7f1db8003b40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1ddf237700 1 -- 172.21.15.203:0/3444671311 >> 172.21.15.203:0/3444671311 conn(0x7f1dd80631b0 msgr2=0x7f1dd80c0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:12.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1ddf237700 1 -- 172.21.15.203:0/3444671311 shutdown_connections 2021-01-06T22:10:12.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1ddf237700 1 -- 172.21.15.203:0/3444671311 wait complete. 2021-01-06T22:10:12.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1ddf237700 1 Processor -- start 2021-01-06T22:10:12.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1ddf237700 1 -- start start 2021-01-06T22:10:12.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1ddf237700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dd8150c00 0x7f1dd81539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:12.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1ddf237700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dd8150fe0 0x7f1dd80b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:12.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1ddf237700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dd8161ef0 0x7f1dd8162370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:12.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1ddf237700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd814d280 con 0x7f1dd8150c00 2021-01-06T22:10:12.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1ddf237700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd80b0980 con 0x7f1dd8150fe0 2021-01-06T22:10:12.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1ddf237700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd8162860 con 0x7f1dd8161ef0 2021-01-06T22:10:12.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1dd7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dd8150fe0 0x7f1dd80b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:12.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1dd7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dd8150fe0 0x7f1dd80b79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50996/0 (socket says 172.21.15.203:50996) 2021-01-06T22:10:12.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1dd7fff700 1 -- 172.21.15.203:0/1153899964 learned_addr learned my addr 172.21.15.203:0/1153899964 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:10:12.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.517+0000 7f1ddcfd3700 1 --2- 172.21.15.203:0/1153899964 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dd8150c00 0x7f1dd81539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:12.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.521+0000 7f1ddd7d4700 1 --2- 172.21.15.203:0/1153899964 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dd8161ef0 0x7f1dd8162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:12.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.521+0000 7f1dd7fff700 1 -- 172.21.15.203:0/1153899964 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dd8161ef0 msgr2=0x7f1dd8162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:12.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.521+0000 7f1dd7fff700 1 --2- 172.21.15.203:0/1153899964 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dd8161ef0 0x7f1dd8162370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.521+0000 7f1dd7fff700 1 -- 172.21.15.203:0/1153899964 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dd8150c00 msgr2=0x7f1dd81539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:12.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.521+0000 7f1dd7fff700 1 --2- 172.21.15.203:0/1153899964 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dd8150c00 0x7f1dd81539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.521+0000 7f1dd7fff700 1 -- 172.21.15.203:0/1153899964 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd8165be0 con 0x7f1dd8150fe0 2021-01-06T22:10:12.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.521+0000 7f1dd7fff700 1 --2- 172.21.15.203:0/1153899964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dd8150fe0 0x7f1dd80b79a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f1dc0011cc0 tx=0x7f1dc0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:12.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.521+0000 7f1dd4ff9700 1 -- 172.21.15.203:0/1153899964 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc000f9d0 con 0x7f1dd8150fe0 2021-01-06T22:10:12.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.521+0000 7f1dd4ff9700 1 -- 172.21.15.203:0/1153899964 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc000fb30 con 0x7f1dd8150fe0 2021-01-06T22:10:12.539 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.521+0000 7f1ddf237700 1 -- 172.21.15.203:0/1153899964 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd8165e80 con 0x7f1dd8150fe0 2021-01-06T22:10:12.539 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.521+0000 7f1dd4ff9700 1 -- 172.21.15.203:0/1153899964 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc000feb0 con 0x7f1dd8150fe0 2021-01-06T22:10:12.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.521+0000 7f1ddf237700 1 -- 172.21.15.203:0/1153899964 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd80b0e50 con 0x7f1dd8150fe0 2021-01-06T22:10:12.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.521+0000 7f1ddf237700 1 -- 172.21.15.203:0/1153899964 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd8161be0 con 0x7f1dd8150fe0 2021-01-06T22:10:12.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.521+0000 7f1dd4ff9700 1 -- 172.21.15.203:0/1153899964 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f1dc0024070 con 0x7f1dd8150fe0 2021-01-06T22:10:12.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.521+0000 7f1dd4ff9700 1 --2- 172.21.15.203:0/1153899964 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1dbc056400 0x7f1dbc0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:12.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.525+0000 7f1dd4ff9700 1 -- 172.21.15.203:0/1153899964 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f1dc0070010 con 0x7f1dd8150fe0 2021-01-06T22:10:12.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.525+0000 7f1ddcfd3700 1 --2- 172.21.15.203:0/1153899964 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1dbc056400 0x7f1dbc0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:12.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.525+0000 7f1ddcfd3700 1 --2- 172.21.15.203:0/1153899964 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1dbc056400 0x7f1dbc0588c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f1dd80c1940 tx=0x7f1dc8013000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:12.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.529+0000 7f1dd4ff9700 1 -- 172.21.15.203:0/1153899964 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f1dc0045730 con 0x7f1dd8150fe0 2021-01-06T22:10:12.734 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.729+0000 7f1ddf237700 1 -- 172.21.15.203:0/1153899964 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1dd805d600 con 0x7f1dd8150fe0 2021-01-06T22:10:12.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.733+0000 7f1dd4ff9700 1 -- 172.21.15.203:0/1153899964 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+3211 (secure 0 0 0) 0x7f1dc0048d50 con 0x7f1dd8150fe0 2021-01-06T22:10:12.736 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:10:12.736 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:10:12.482956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4859,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4859":{"gid":4859,"name":"a","rank":0,"incarnation":39,"state":"up:resolve","state_seq":20,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:10:12.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.733+0000 7f1ddf237700 1 -- 172.21.15.203:0/1153899964 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1dbc056400 msgr2=0x7f1dbc0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:12.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.733+0000 7f1ddf237700 1 --2- 172.21.15.203:0/1153899964 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1dbc056400 0x7f1dbc0588c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f1dd80c1940 tx=0x7f1dc8013000).stop 2021-01-06T22:10:12.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.733+0000 7f1ddf237700 1 -- 172.21.15.203:0/1153899964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dd8150fe0 msgr2=0x7f1dd80b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:12.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.733+0000 7f1ddf237700 1 --2- 172.21.15.203:0/1153899964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dd8150fe0 0x7f1dd80b79a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f1dc0011cc0 tx=0x7f1dc0008040).stop 2021-01-06T22:10:12.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.733+0000 7f1ddf237700 1 -- 172.21.15.203:0/1153899964 shutdown_connections 2021-01-06T22:10:12.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.733+0000 7f1ddf237700 1 --2- 172.21.15.203:0/1153899964 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1dbc056400 0x7f1dbc0588c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.733+0000 7f1ddf237700 1 --2- 172.21.15.203:0/1153899964 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1dd8161ef0 0x7f1dd8162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.733+0000 7f1ddf237700 1 --2- 172.21.15.203:0/1153899964 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dd8150c00 0x7f1dd81539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.733+0000 7f1ddf237700 1 --2- 172.21.15.203:0/1153899964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dd8150fe0 0x7f1dd80b79a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:12.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.733+0000 7f1ddf237700 1 -- 172.21.15.203:0/1153899964 >> 172.21.15.203:0/1153899964 conn(0x7f1dd80631b0 msgr2=0x7f1dd8154620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:12.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.733+0000 7f1ddf237700 1 -- 172.21.15.203:0/1153899964 shutdown_connections 2021-01-06T22:10:12.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:12.733+0000 7f1ddf237700 1 -- 172.21.15.203:0/1153899964 wait complete. 2021-01-06T22:10:12.741 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 40 2021-01-06T22:10:12.751 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 40, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 40, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_4859": { "addr": "172.21.15.203:6837/191881002", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6836", "nonce": 191881002, "type": "v2" }, { "addr": "172.21.15.203:6837", "nonce": 191881002, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4859, "incarnation": 39, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:resolve", "state_seq": 20 }, "gid_5068": { "addr": "172.21.15.203:6835/1477297469", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6834", "nonce": 1477297469, "type": "v2" }, { "addr": "172.21.15.203:6835", "nonce": 1477297469, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5068, "incarnation": 33, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2021-01-06T22:10:12.482956+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 4859, "mds_1": 5068 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.91:6836/2428882934", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6834", "nonce": 2428882934, "type": "v2" }, { "addr": "172.21.15.91:6836", "nonce": 2428882934, "type": "v1" } ] }, "epoch": 26, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4724, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2021-01-06T22:10:14.432 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:10:14.434 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:10:14.752 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:10:14.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.881+0000 7fe3a7fff700 1 Processor -- start 2021-01-06T22:10:14.886 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.881+0000 7fe3a7fff700 1 -- start start 2021-01-06T22:10:14.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.881+0000 7fe3a7fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe3a8151160 0x7fe3a8060750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:14.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.881+0000 7fe3a7fff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a80602b0 con 0x7fe3a80633f0 2021-01-06T22:10:14.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.881+0000 7fe3a7fff700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a8151710 con 0x7fe3a8063010 2021-01-06T22:10:14.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.881+0000 7fe3a7fff700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a814b760 con 0x7fe3a8151160 2021-01-06T22:10:14.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.881+0000 7fe3a77fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe3a8151160 0x7fe3a8060750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:14.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.881+0000 7fe3a6ffd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fe3a8063010 0x7fe3a814e3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49598/0 (socket says 172.21.15.203:49598) 2021-01-06T22:10:14.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.881+0000 7fe3a6ffd700 1 -- 172.21.15.203:0/3150146215 learned_addr learned my addr 172.21.15.203:0/3150146215 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:10:14.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.885+0000 7fe3a77fe700 1 -- 172.21.15.203:0/3150146215 >> v1:172.21.15.203:6789/0 conn(0x7fe3a8063010 legacy=0x7fe3a814e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:14.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.885+0000 7fe3a77fe700 1 -- 172.21.15.203:0/3150146215 >> v1:172.21.15.91:6789/0 conn(0x7fe3a80633f0 legacy=0x7fe3a80be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:10:14.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.885+0000 7fe3a77fe700 1 -- 172.21.15.203:0/3150146215 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a81519b0 con 0x7fe3a8151160 2021-01-06T22:10:14.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.885+0000 7fe3a77fe700 1 --2- 172.21.15.203:0/3150146215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3a8151160 0x7fe3a8060750 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fe398009c90 tx=0x7fe3980091d0).ready entity=mon.1 client_cookie=bf0b43cd7cf6930e server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:14.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.885+0000 7fe3a5ffb700 1 -- 172.21.15.203:0/3150146215 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39800bc10 con 0x7fe3a8151160 2021-01-06T22:10:14.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.885+0000 7fe3a5ffb700 1 -- 172.21.15.203:0/3150146215 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39800bd70 con 0x7fe3a8151160 2021-01-06T22:10:14.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.885+0000 7fe3a5ffb700 1 -- 172.21.15.203:0/3150146215 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe398011610 con 0x7fe3a8151160 2021-01-06T22:10:14.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.885+0000 7fe3a7fff700 1 -- 172.21.15.203:0/3150146215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3a8151160 msgr2=0x7fe3a8060750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:14.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.885+0000 7fe3a7fff700 1 --2- 172.21.15.203:0/3150146215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3a8151160 0x7fe3a8060750 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fe398009c90 tx=0x7fe3980091d0).stop 2021-01-06T22:10:14.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.885+0000 7fe3a7fff700 1 -- 172.21.15.203:0/3150146215 shutdown_connections 2021-01-06T22:10:14.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.885+0000 7fe3a7fff700 1 --2- 172.21.15.203:0/3150146215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3a8151160 0x7fe3a8060750 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:14.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.885+0000 7fe3a7fff700 1 -- 172.21.15.203:0/3150146215 >> 172.21.15.203:0/3150146215 conn(0x7fe3a8067e20 msgr2=0x7fe3a8068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:14.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.885+0000 7fe3a7fff700 1 -- 172.21.15.203:0/3150146215 shutdown_connections 2021-01-06T22:10:14.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.885+0000 7fe3a7fff700 1 -- 172.21.15.203:0/3150146215 wait complete. 2021-01-06T22:10:14.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.885+0000 7fe3a7fff700 1 Processor -- start 2021-01-06T22:10:14.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.885+0000 7fe3a7fff700 1 -- start start 2021-01-06T22:10:14.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3a8063010 0x7fe3a80b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:14.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a7fff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3a80633f0 0x7fe3a81611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:14.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a7fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3a8151160 0x7fe3a814fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:14.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a7fff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a8160630 con 0x7fe3a80633f0 2021-01-06T22:10:14.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a7fff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a80b6fc0 con 0x7fe3a8063010 2021-01-06T22:10:14.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a7fff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a80b7260 con 0x7fe3a8151160 2021-01-06T22:10:14.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a6ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3a8063010 0x7fe3a80b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:14.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a6ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3a8063010 0x7fe3a80b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51010/0 (socket says 172.21.15.203:51010) 2021-01-06T22:10:14.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a6ffd700 1 -- 172.21.15.203:0/2604698980 learned_addr learned my addr 172.21.15.203:0/2604698980 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:10:14.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a77fe700 1 --2- 172.21.15.203:0/2604698980 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3a8151160 0x7fe3a814fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:14.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a67fc700 1 --2- 172.21.15.203:0/2604698980 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3a80633f0 0x7fe3a81611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:14.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a6ffd700 1 -- 172.21.15.203:0/2604698980 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3a8151160 msgr2=0x7fe3a814fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:14.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a6ffd700 1 --2- 172.21.15.203:0/2604698980 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3a8151160 0x7fe3a814fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:14.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a6ffd700 1 -- 172.21.15.203:0/2604698980 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3a80633f0 msgr2=0x7fe3a81611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:14.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a6ffd700 1 --2- 172.21.15.203:0/2604698980 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3a80633f0 0x7fe3a81611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:14.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a6ffd700 1 -- 172.21.15.203:0/2604698980 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a8160a70 con 0x7fe3a8063010 2021-01-06T22:10:14.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a6ffd700 1 --2- 172.21.15.203:0/2604698980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3a8063010 0x7fe3a80b1330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fe3a000c710 tx=0x7fe3a0007c70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:14.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe38effd700 1 -- 172.21.15.203:0/2604698980 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0007320 con 0x7fe3a8063010 2021-01-06T22:10:14.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe38effd700 1 -- 172.21.15.203:0/2604698980 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a0007480 con 0x7fe3a8063010 2021-01-06T22:10:14.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a7fff700 1 -- 172.21.15.203:0/2604698980 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a8160d10 con 0x7fe3a8063010 2021-01-06T22:10:14.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe38effd700 1 -- 172.21.15.203:0/2604698980 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a00057f0 con 0x7fe3a8063010 2021-01-06T22:10:14.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a7fff700 1 -- 172.21.15.203:0/2604698980 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a8149a40 con 0x7fe3a8063010 2021-01-06T22:10:14.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a7fff700 1 -- 172.21.15.203:0/2604698980 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a80450a0 con 0x7fe3a8063010 2021-01-06T22:10:14.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe38effd700 1 -- 172.21.15.203:0/2604698980 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fe3a0010030 con 0x7fe3a8063010 2021-01-06T22:10:14.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe38effd700 1 --2- 172.21.15.203:0/2604698980 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe3880563d0 0x7fe388058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:14.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe38effd700 1 -- 172.21.15.203:0/2604698980 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fe3a006f490 con 0x7fe3a8063010 2021-01-06T22:10:14.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a67fc700 1 --2- 172.21.15.203:0/2604698980 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe3880563d0 0x7fe388058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:14.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.889+0000 7fe3a67fc700 1 --2- 172.21.15.203:0/2604698980 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe3880563d0 0x7fe388058890 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fe3a814fc10 tx=0x7fe39c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:14.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:14.893+0000 7fe38effd700 1 -- 172.21.15.203:0/2604698980 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fe3a0044bb0 con 0x7fe3a8063010 2021-01-06T22:10:15.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:15.093+0000 7fe3a7fff700 1 -- 172.21.15.203:0/2604698980 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3a80bc660 con 0x7fe3a8063010 2021-01-06T22:10:15.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:15.093+0000 7fe38effd700 1 -- 172.21.15.203:0/2604698980 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+3210 (secure 0 0 0) 0x7fe3a00481d0 con 0x7fe3a8063010 2021-01-06T22:10:15.098 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:10:15.099 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:10:14.501391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4859,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4859":{"gid":4859,"name":"a","rank":0,"incarnation":39,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:10:15.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:15.097+0000 7fe3a7fff700 1 -- 172.21.15.203:0/2604698980 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe3880563d0 msgr2=0x7fe388058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:15.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:15.097+0000 7fe3a7fff700 1 --2- 172.21.15.203:0/2604698980 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe3880563d0 0x7fe388058890 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fe3a814fc10 tx=0x7fe39c009040).stop 2021-01-06T22:10:15.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:15.097+0000 7fe3a7fff700 1 -- 172.21.15.203:0/2604698980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3a8063010 msgr2=0x7fe3a80b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:15.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:15.097+0000 7fe3a7fff700 1 --2- 172.21.15.203:0/2604698980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3a8063010 0x7fe3a80b1330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fe3a000c710 tx=0x7fe3a0007c70).stop 2021-01-06T22:10:15.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:15.097+0000 7fe3a7fff700 1 -- 172.21.15.203:0/2604698980 shutdown_connections 2021-01-06T22:10:15.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:15.097+0000 7fe3a7fff700 1 --2- 172.21.15.203:0/2604698980 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe3880563d0 0x7fe388058890 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:15.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:15.097+0000 7fe3a7fff700 1 --2- 172.21.15.203:0/2604698980 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3a8151160 0x7fe3a814fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:15.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:15.097+0000 7fe3a7fff700 1 --2- 172.21.15.203:0/2604698980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3a8063010 0x7fe3a80b1330 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:15.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:15.097+0000 7fe3a7fff700 1 --2- 172.21.15.203:0/2604698980 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3a80633f0 0x7fe3a81611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:15.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:15.097+0000 7fe3a7fff700 1 -- 172.21.15.203:0/2604698980 >> 172.21.15.203:0/2604698980 conn(0x7fe3a8067e20 msgr2=0x7fe3a814ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:15.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:15.097+0000 7fe3a7fff700 1 -- 172.21.15.203:0/2604698980 shutdown_connections 2021-01-06T22:10:15.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:15.097+0000 7fe3a7fff700 1 -- 172.21.15.203:0/2604698980 wait complete. 2021-01-06T22:10:15.103 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 42 2021-01-06T22:10:15.114 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:10:17.115 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:10:17.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.237+0000 7fe37f59e700 1 Processor -- start 2021-01-06T22:10:17.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.237+0000 7fe37f59e700 1 -- start start 2021-01-06T22:10:17.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37f59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3800b97a0 0x7fe3800b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:17.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37f59e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe3800bed80 0x7fe38014c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:17.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37f59e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3800be410 con 0x7fe3800bd420 2021-01-06T22:10:17.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37f59e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe380066510 con 0x7fe3800bed80 2021-01-06T22:10:17.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37f59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3800667b0 con 0x7fe3800b97a0 2021-01-06T22:10:17.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37dd9b700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fe3800bd420 0x7fe380065c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49626/0 (socket says 172.21.15.203:49626) 2021-01-06T22:10:17.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37dd9b700 1 -- 172.21.15.203:0/239072632 learned_addr learned my addr 172.21.15.203:0/239072632 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:10:17.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37ed9d700 1 --2- 172.21.15.203:0/239072632 >> v2:172.21.15.91:3300/0 conn(0x7fe3800bed80 0x7fe38014c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:17.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37e59c700 1 --2- 172.21.15.203:0/239072632 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3800b97a0 0x7fe3800b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:17.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 -- 172.21.15.203:0/239072632 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693082618 0 0) 0x7fe3800be410 con 0x7fe3800bd420 2021-01-06T22:10:17.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 -- 172.21.15.203:0/239072632 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe364003230 con 0x7fe3800bd420 2021-01-06T22:10:17.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 -- 172.21.15.203:0/239072632 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1274308752 0 0) 0x7fe364003230 con 0x7fe3800bd420 2021-01-06T22:10:17.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 -- 172.21.15.203:0/239072632 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3800b97a0 msgr2=0x7fe3800b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:10:17.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 --2- 172.21.15.203:0/239072632 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3800b97a0 0x7fe3800b9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:17.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 -- 172.21.15.203:0/239072632 >> v2:172.21.15.91:3300/0 conn(0x7fe3800bed80 msgr2=0x7fe38014c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:10:17.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 --2- 172.21.15.203:0/239072632 >> v2:172.21.15.91:3300/0 conn(0x7fe3800bed80 0x7fe38014c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:17.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 -- 172.21.15.203:0/239072632 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe380066a50 con 0x7fe3800bd420 2021-01-06T22:10:17.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 -- 172.21.15.203:0/239072632 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7fe3740014e0 con 0x7fe3800bd420 2021-01-06T22:10:17.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37ed9d700 1 --2- 172.21.15.203:0/239072632 >> v2:172.21.15.91:3300/0 conn(0x7fe3800bed80 0x7fe38014c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:10:17.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 -- 172.21.15.203:0/239072632 >> v1:172.21.15.203:6789/0 conn(0x7fe3800bd420 legacy=0x7fe380065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:17.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 --2- 172.21.15.203:0/239072632 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe364003720 0x7fe364003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:17.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 --2- 172.21.15.203:0/239072632 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3800b97a0 0x7fe3640044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:17.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 --2- 172.21.15.203:0/239072632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3640049c0 0x7fe364004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:17.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 -- 172.21.15.203:0/239072632 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe380066510 con 0x7fe364003720 2021-01-06T22:10:17.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 -- 172.21.15.203:0/239072632 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3800667b0 con 0x7fe3640049c0 2021-01-06T22:10:17.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 -- 172.21.15.203:0/239072632 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3640041c0 con 0x7fe3800b97a0 2021-01-06T22:10:17.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37ed9d700 1 --2- 172.21.15.203:0/239072632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3640049c0 0x7fe364004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:17.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37e59c700 1 --2- 172.21.15.203:0/239072632 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe364003720 0x7fe364003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:17.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37dd9b700 1 --2- 172.21.15.203:0/239072632 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3800b97a0 0x7fe3640044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:17.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37ed9d700 1 -- 172.21.15.203:0/239072632 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3800b97a0 msgr2=0x7fe3640044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:10:17.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37ed9d700 1 --2- 172.21.15.203:0/239072632 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3800b97a0 0x7fe3640044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:17.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37ed9d700 1 -- 172.21.15.203:0/239072632 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe364003720 msgr2=0x7fe364003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:10:17.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37ed9d700 1 --2- 172.21.15.203:0/239072632 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe364003720 0x7fe364003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:17.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37ed9d700 1 -- 172.21.15.203:0/239072632 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe364003230 con 0x7fe3640049c0 2021-01-06T22:10:17.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37e59c700 1 --2- 172.21.15.203:0/239072632 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe364003720 0x7fe364003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:10:17.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37ed9d700 1 --2- 172.21.15.203:0/239072632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3640049c0 0x7fe364004e40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fe37800e300 tx=0x7fe3780091d0).ready entity=mon.1 client_cookie=803b56d6ee992c94 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:17.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 -- 172.21.15.203:0/239072632 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378009800 con 0x7fe3640049c0 2021-01-06T22:10:17.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 -- 172.21.15.203:0/239072632 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3780123f0 con 0x7fe3640049c0 2021-01-06T22:10:17.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37d59a700 1 -- 172.21.15.203:0/239072632 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378012770 con 0x7fe3640049c0 2021-01-06T22:10:17.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37f59e700 1 -- 172.21.15.203:0/239072632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3640049c0 msgr2=0x7fe364004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:17.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37f59e700 1 --2- 172.21.15.203:0/239072632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3640049c0 0x7fe364004e40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fe37800e300 tx=0x7fe3780091d0).stop 2021-01-06T22:10:17.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37ed9d700 1 -- 172.21.15.203:0/239072632 reap_dead start 2021-01-06T22:10:17.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37f59e700 1 -- 172.21.15.203:0/239072632 shutdown_connections 2021-01-06T22:10:17.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37f59e700 1 -- 172.21.15.203:0/239072632 >> 172.21.15.203:0/239072632 conn(0x7fe3800648f0 msgr2=0x7fe380064cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:17.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37f59e700 1 -- 172.21.15.203:0/239072632 shutdown_connections 2021-01-06T22:10:17.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.241+0000 7fe37f59e700 1 -- 172.21.15.203:0/239072632 wait complete. 2021-01-06T22:10:17.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37f59e700 1 Processor -- start 2021-01-06T22:10:17.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37f59e700 1 -- start start 2021-01-06T22:10:17.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37f59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3800a1a40 0x7fe38015ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:17.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37f59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3800a1e20 0x7fe3800a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:17.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37f59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3800a6730 0x7fe3800a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:17.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37f59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38015bef0 con 0x7fe3800a6730 2021-01-06T22:10:17.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37f59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3800a11d0 con 0x7fe3800a1e20 2021-01-06T22:10:17.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37f59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3800a5830 con 0x7fe3800a1a40 2021-01-06T22:10:17.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37dd9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3800a1e20 0x7fe3800a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:17.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37dd9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3800a1e20 0x7fe3800a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51044/0 (socket says 172.21.15.203:51044) 2021-01-06T22:10:17.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37dd9b700 1 -- 172.21.15.203:0/1854478069 learned_addr learned my addr 172.21.15.203:0/1854478069 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:10:17.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37e59c700 1 --2- 172.21.15.203:0/1854478069 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3800a1a40 0x7fe38015ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:17.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37ed9d700 1 --2- 172.21.15.203:0/1854478069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3800a6730 0x7fe3800a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:17.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37dd9b700 1 -- 172.21.15.203:0/1854478069 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3800a1a40 msgr2=0x7fe38015ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:17.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37dd9b700 1 --2- 172.21.15.203:0/1854478069 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3800a1a40 0x7fe38015ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:17.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37dd9b700 1 -- 172.21.15.203:0/1854478069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3800a6730 msgr2=0x7fe3800a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:17.267 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37dd9b700 1 --2- 172.21.15.203:0/1854478069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3800a6730 0x7fe3800a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:17.267 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37dd9b700 1 -- 172.21.15.203:0/1854478069 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3800a5ad0 con 0x7fe3800a1e20 2021-01-06T22:10:17.267 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37dd9b700 1 --2- 172.21.15.203:0/1854478069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3800a1e20 0x7fe3800a0ce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7fe37400eae0 tx=0x7fe380066be0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:17.267 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe36e7fc700 1 -- 172.21.15.203:0/1854478069 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37400a7c0 con 0x7fe3800a1e20 2021-01-06T22:10:17.268 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe36e7fc700 1 -- 172.21.15.203:0/1854478069 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe374010040 con 0x7fe3800a1e20 2021-01-06T22:10:17.268 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37f59e700 1 -- 172.21.15.203:0/1854478069 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3800a5d70 con 0x7fe3800a1e20 2021-01-06T22:10:17.268 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe36e7fc700 1 -- 172.21.15.203:0/1854478069 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37400ab30 con 0x7fe3800a1e20 2021-01-06T22:10:17.269 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.245+0000 7fe37f59e700 1 -- 172.21.15.203:0/1854478069 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3800a7080 con 0x7fe3800a1e20 2021-01-06T22:10:17.269 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.249+0000 7fe37f59e700 1 -- 172.21.15.203:0/1854478069 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3800a5f00 con 0x7fe3800a1e20 2021-01-06T22:10:17.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.249+0000 7fe36e7fc700 1 -- 172.21.15.203:0/1854478069 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fe37401b020 con 0x7fe3800a1e20 2021-01-06T22:10:17.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.249+0000 7fe36e7fc700 1 --2- 172.21.15.203:0/1854478069 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe35c0563d0 0x7fe35c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:17.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.249+0000 7fe36e7fc700 1 -- 172.21.15.203:0/1854478069 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fe37406f300 con 0x7fe3800a1e20 2021-01-06T22:10:17.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.249+0000 7fe37e59c700 1 --2- 172.21.15.203:0/1854478069 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe35c0563d0 0x7fe35c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:17.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.249+0000 7fe37e59c700 1 --2- 172.21.15.203:0/1854478069 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe35c0563d0 0x7fe35c058890 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fe370006a80 tx=0x7fe370008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:17.272 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.253+0000 7fe36e7fc700 1 -- 172.21.15.203:0/1854478069 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fe374044aa0 con 0x7fe3800a1e20 2021-01-06T22:10:17.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.493+0000 7fe37f59e700 1 -- 172.21.15.203:0/1854478069 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3800450a0 con 0x7fe3800a1e20 2021-01-06T22:10:17.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.493+0000 7fe36e7fc700 1 -- 172.21.15.203:0/1854478069 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3210 (secure 0 0 0) 0x7fe3740480c0 con 0x7fe3800a1e20 2021-01-06T22:10:17.497 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:10:17.498 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:10:15.511645+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4859,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4859":{"gid":4859,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:10:17.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.493+0000 7fe37f59e700 1 -- 172.21.15.203:0/1854478069 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe35c0563d0 msgr2=0x7fe35c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:17.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.493+0000 7fe37f59e700 1 --2- 172.21.15.203:0/1854478069 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe35c0563d0 0x7fe35c058890 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fe370006a80 tx=0x7fe370008040).stop 2021-01-06T22:10:17.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.493+0000 7fe37f59e700 1 -- 172.21.15.203:0/1854478069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3800a1e20 msgr2=0x7fe3800a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:17.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.493+0000 7fe37f59e700 1 --2- 172.21.15.203:0/1854478069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3800a1e20 0x7fe3800a0ce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7fe37400eae0 tx=0x7fe380066be0).stop 2021-01-06T22:10:17.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.493+0000 7fe37f59e700 1 -- 172.21.15.203:0/1854478069 shutdown_connections 2021-01-06T22:10:17.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.493+0000 7fe37f59e700 1 --2- 172.21.15.203:0/1854478069 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe35c0563d0 0x7fe35c058890 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:17.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.493+0000 7fe37f59e700 1 --2- 172.21.15.203:0/1854478069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3800a6730 0x7fe3800a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:17.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.493+0000 7fe37f59e700 1 --2- 172.21.15.203:0/1854478069 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe3800a1a40 0x7fe38015ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:17.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.493+0000 7fe37f59e700 1 --2- 172.21.15.203:0/1854478069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3800a1e20 0x7fe3800a0ce0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:17.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.493+0000 7fe37f59e700 1 -- 172.21.15.203:0/1854478069 >> 172.21.15.203:0/1854478069 conn(0x7fe3800648f0 msgr2=0x7fe3800c15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:17.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.493+0000 7fe37f59e700 1 -- 172.21.15.203:0/1854478069 shutdown_connections 2021-01-06T22:10:17.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:17.493+0000 7fe37f59e700 1 -- 172.21.15.203:0/1854478069 wait complete. 2021-01-06T22:10:17.502 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 43 2021-01-06T22:10:17.515 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=4274 2021-01-06T22:10:17.515 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 4 secs before reviving mds.f 2021-01-06T22:10:17.529 INFO:tasks.ceph.mds.a.smithi203.stderr:2021-01-06T22:10:17.525+0000 7f71e6190700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2021-01-06T22:10:17.530 INFO:tasks.ceph.mds.a.smithi203.stderr:2021-01-06T22:10:17.525+0000 7f71e6190700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2021-01-06T22:10:21.516 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2021-01-06T22:10:21.517 INFO:tasks.ceph.mds.f:Restarting daemon 2021-01-06T22:10:21.517 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2021-01-06T22:10:21.521 INFO:tasks.ceph.mds.f:Started 2021-01-06T22:10:21.521 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:10:21.601 INFO:tasks.ceph.mds.f.smithi091.stdout:starting mds.f at 2021-01-06T22:10:21.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.653+0000 7f09d7fff700 1 Processor -- start 2021-01-06T22:10:21.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.653+0000 7f09d7fff700 1 -- start start 2021-01-06T22:10:21.660 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d7fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f09d80ba9e0 0x7f09d814ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:21.660 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d7fff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d80bccb0 con 0x7f09d814df70 2021-01-06T22:10:21.660 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d7fff700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d80bd180 con 0x7f09d80c0fa0 2021-01-06T22:10:21.661 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d7fff700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d8067ac0 con 0x7f09d80ba9e0 2021-01-06T22:10:21.661 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09cffff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f09d80c0fa0 0x7f09d80641e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49648/0 (socket says 172.21.15.203:49648) 2021-01-06T22:10:21.661 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09cffff700 1 -- 172.21.15.203:0/2074725498 learned_addr learned my addr 172.21.15.203:0/2074725498 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:10:21.661 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d77fe700 1 --2- 172.21.15.203:0/2074725498 >> v2:172.21.15.91:3300/0 conn(0x7f09d80ba9e0 0x7f09d814ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:21.662 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d67fc700 1 -- 172.21.15.203:0/2074725498 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511817722 0 0) 0x7f09d80bd180 con 0x7f09d80c0fa0 2021-01-06T22:10:21.662 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d67fc700 1 -- 172.21.15.203:0/2074725498 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09b8003230 con 0x7f09d80c0fa0 2021-01-06T22:10:21.662 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d77fe700 1 -- 172.21.15.203:0/2074725498 >> v1:172.21.15.203:6789/0 conn(0x7f09d80c0fa0 legacy=0x7f09d80641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:21.662 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d77fe700 1 -- 172.21.15.203:0/2074725498 >> v1:172.21.15.91:6789/0 conn(0x7f09d814df70 legacy=0x7f09d814e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:21.663 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d77fe700 1 -- 172.21.15.203:0/2074725498 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d80bcee0 con 0x7f09d80ba9e0 2021-01-06T22:10:21.663 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d77fe700 1 --2- 172.21.15.203:0/2074725498 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d80ba9e0 0x7f09d814ad90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f09d0009c90 tx=0x7f09d00091d0).ready entity=mon.0 client_cookie=f72c5258a0f6e6c1 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:21.664 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d67fc700 1 -- 172.21.15.203:0/2074725498 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d000bb00 con 0x7f09d80ba9e0 2021-01-06T22:10:21.664 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d67fc700 1 -- 172.21.15.203:0/2074725498 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d000bc60 con 0x7f09d80ba9e0 2021-01-06T22:10:21.665 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d67fc700 1 -- 172.21.15.203:0/2074725498 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d0011400 con 0x7f09d80ba9e0 2021-01-06T22:10:21.665 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d7fff700 1 -- 172.21.15.203:0/2074725498 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d80ba9e0 msgr2=0x7f09d814ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:21.665 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d7fff700 1 --2- 172.21.15.203:0/2074725498 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d80ba9e0 0x7f09d814ad90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f09d0009c90 tx=0x7f09d00091d0).stop 2021-01-06T22:10:21.665 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d7fff700 1 -- 172.21.15.203:0/2074725498 shutdown_connections 2021-01-06T22:10:21.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d7fff700 1 --2- 172.21.15.203:0/2074725498 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d80ba9e0 0x7f09d814ad90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:21.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d7fff700 1 -- 172.21.15.203:0/2074725498 >> 172.21.15.203:0/2074725498 conn(0x7f09d80667b0 msgr2=0x7f09d80c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:21.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d7fff700 1 -- 172.21.15.203:0/2074725498 shutdown_connections 2021-01-06T22:10:21.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.657+0000 7f09d7fff700 1 -- 172.21.15.203:0/2074725498 wait complete. 2021-01-06T22:10:21.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09d7fff700 1 Processor -- start 2021-01-06T22:10:21.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09d7fff700 1 -- start start 2021-01-06T22:10:21.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09d7fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09d80ba9e0 0x7f09d81458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:21.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09d7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d80c0fa0 0x7f09d814eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:21.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09d7fff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d814df70 0x7f09d814d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:21.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09d7fff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d815e140 con 0x7f09d814df70 2021-01-06T22:10:21.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09d7fff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d815dc30 con 0x7f09d80c0fa0 2021-01-06T22:10:21.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09d7fff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d815dea0 con 0x7f09d80ba9e0 2021-01-06T22:10:21.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09cffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d80c0fa0 0x7f09d814eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:21.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09cffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d80c0fa0 0x7f09d814eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51060/0 (socket says 172.21.15.203:51060) 2021-01-06T22:10:21.670 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09cffff700 1 -- 172.21.15.203:0/1774868490 learned_addr learned my addr 172.21.15.203:0/1774868490 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:10:21.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09d6ffd700 1 --2- 172.21.15.203:0/1774868490 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09d80ba9e0 0x7f09d81458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:21.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09d77fe700 1 --2- 172.21.15.203:0/1774868490 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d814df70 0x7f09d814d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:21.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09cffff700 1 -- 172.21.15.203:0/1774868490 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09d80ba9e0 msgr2=0x7f09d81458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:21.671 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09cffff700 1 --2- 172.21.15.203:0/1774868490 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09d80ba9e0 0x7f09d81458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:21.672 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09cffff700 1 -- 172.21.15.203:0/1774868490 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d814df70 msgr2=0x7f09d814d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:21.672 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09cffff700 1 --2- 172.21.15.203:0/1774868490 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d814df70 0x7f09d814d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:21.672 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09cffff700 1 -- 172.21.15.203:0/1774868490 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d81501e0 con 0x7f09d80c0fa0 2021-01-06T22:10:21.672 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09cffff700 1 --2- 172.21.15.203:0/1774868490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d80c0fa0 0x7f09d814eed0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f09c000c710 tx=0x7f09c0007a90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:21.672 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09ceffd700 1 -- 172.21.15.203:0/1774868490 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00072e0 con 0x7f09d80c0fa0 2021-01-06T22:10:21.673 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09ceffd700 1 -- 172.21.15.203:0/1774868490 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c0007440 con 0x7f09d80c0fa0 2021-01-06T22:10:21.673 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09d7fff700 1 -- 172.21.15.203:0/1774868490 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d8150480 con 0x7f09d80c0fa0 2021-01-06T22:10:21.673 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09ceffd700 1 -- 172.21.15.203:0/1774868490 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00057f0 con 0x7f09d80c0fa0 2021-01-06T22:10:21.673 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.661+0000 7f09d7fff700 1 -- 172.21.15.203:0/1774868490 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d814d870 con 0x7f09d80c0fa0 2021-01-06T22:10:21.673 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.665+0000 7f09d7fff700 1 -- 172.21.15.203:0/1774868490 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d80450a0 con 0x7f09d80c0fa0 2021-01-06T22:10:21.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.665+0000 7f09ceffd700 1 -- 172.21.15.203:0/1774868490 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f09c0010070 con 0x7f09d80c0fa0 2021-01-06T22:10:21.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.665+0000 7f09ceffd700 1 --2- 172.21.15.203:0/1774868490 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f09bc0563d0 0x7f09bc058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:21.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.665+0000 7f09ceffd700 1 -- 172.21.15.203:0/1774868490 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f09c006ecc0 con 0x7f09d80c0fa0 2021-01-06T22:10:21.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.665+0000 7f09d6ffd700 1 --2- 172.21.15.203:0/1774868490 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f09bc0563d0 0x7f09bc058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:21.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.665+0000 7f09d6ffd700 1 --2- 172.21.15.203:0/1774868490 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f09bc0563d0 0x7f09bc058890 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f09c8003670 tx=0x7f09c8007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:21.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.673+0000 7f09ceffd700 1 -- 172.21.15.203:0/1774868490 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f09c0044460 con 0x7f09d80c0fa0 2021-01-06T22:10:21.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.897+0000 7f09d7fff700 1 -- 172.21.15.203:0/1774868490 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09d805ca60 con 0x7f09d80c0fa0 2021-01-06T22:10:21.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.897+0000 7f09ceffd700 1 -- 172.21.15.203:0/1774868490 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3210 (secure 0 0 0) 0x7f09c0018030 con 0x7f09d80c0fa0 2021-01-06T22:10:21.903 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:10:21.903 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:10:15.511645+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4859,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4859":{"gid":4859,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:10:21.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.901+0000 7f09d7fff700 1 -- 172.21.15.203:0/1774868490 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f09bc0563d0 msgr2=0x7f09bc058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:21.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.901+0000 7f09d7fff700 1 --2- 172.21.15.203:0/1774868490 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f09bc0563d0 0x7f09bc058890 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f09c8003670 tx=0x7f09c8007b80).stop 2021-01-06T22:10:21.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.901+0000 7f09d7fff700 1 -- 172.21.15.203:0/1774868490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d80c0fa0 msgr2=0x7f09d814eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:21.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.901+0000 7f09d7fff700 1 --2- 172.21.15.203:0/1774868490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d80c0fa0 0x7f09d814eed0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f09c000c710 tx=0x7f09c0007a90).stop 2021-01-06T22:10:21.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.901+0000 7f09d7fff700 1 -- 172.21.15.203:0/1774868490 shutdown_connections 2021-01-06T22:10:21.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.901+0000 7f09d7fff700 1 --2- 172.21.15.203:0/1774868490 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f09bc0563d0 0x7f09bc058890 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:21.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.901+0000 7f09d7fff700 1 --2- 172.21.15.203:0/1774868490 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d814df70 0x7f09d814d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:21.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.901+0000 7f09d7fff700 1 --2- 172.21.15.203:0/1774868490 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09d80ba9e0 0x7f09d81458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:21.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.901+0000 7f09d7fff700 1 --2- 172.21.15.203:0/1774868490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d80c0fa0 0x7f09d814eed0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:21.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.901+0000 7f09d7fff700 1 -- 172.21.15.203:0/1774868490 >> 172.21.15.203:0/1774868490 conn(0x7f09d80667b0 msgr2=0x7f09d80be760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:21.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.901+0000 7f09d7fff700 1 -- 172.21.15.203:0/1774868490 shutdown_connections 2021-01-06T22:10:21.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:21.901+0000 7f09d7fff700 1 -- 172.21.15.203:0/1774868490 wait complete. 2021-01-06T22:10:21.909 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 43 2021-01-06T22:10:21.926 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2021-01-06T22:10:23.927 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:10:24.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.053+0000 7ff706af1700 1 Processor -- start 2021-01-06T22:10:24.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.053+0000 7ff706af1700 1 -- start start 2021-01-06T22:10:24.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.053+0000 7ff706af1700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff7000602b0 0x7ff7000c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:24.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.053+0000 7ff706af1700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff700149150 0x7ff70014bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:24.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.053+0000 7ff706af1700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70014f0a0 con 0x7ff700148d70 2021-01-06T22:10:24.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.053+0000 7ff706af1700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7001495e0 con 0x7ff7000602b0 2021-01-06T22:10:24.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.053+0000 7ff706af1700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70014c3c0 con 0x7ff700149150 2021-01-06T22:10:24.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff7062f0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff700149150 0x7ff70014bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:24.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff7052ee700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff7000602b0 0x7ff7000c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:24.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff7062f0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff700149150 0x7ff70014bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:56152/0 (socket says 172.21.15.203:56152) 2021-01-06T22:10:24.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff7062f0700 1 -- 172.21.15.203:0/2629171446 learned_addr learned my addr 172.21.15.203:0/2629171446 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:10:24.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff7062f0700 1 -- 172.21.15.203:0/2629171446 >> v2:172.21.15.91:3300/0 conn(0x7ff7000602b0 msgr2=0x7ff7000c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:10:24.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff7062f0700 1 --2- 172.21.15.203:0/2629171446 >> v2:172.21.15.91:3300/0 conn(0x7ff7000602b0 0x7ff7000c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:24.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff7062f0700 1 -- 172.21.15.203:0/2629171446 >> v1:172.21.15.91:6789/0 conn(0x7ff700148d70 legacy=0x7ff70014fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:24.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff7062f0700 1 -- 172.21.15.203:0/2629171446 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70014c660 con 0x7ff700149150 2021-01-06T22:10:24.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff7062f0700 1 --2- 172.21.15.203:0/2629171446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff700149150 0x7ff70014bb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7ff6f8009d50 tx=0x7ff6f8009290).ready entity=mon.2 client_cookie=47cff86dae3f2869 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:24.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff704aed700 1 -- 172.21.15.203:0/2629171446 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f800bc50 con 0x7ff700149150 2021-01-06T22:10:24.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff704aed700 1 -- 172.21.15.203:0/2629171446 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f800bdb0 con 0x7ff700149150 2021-01-06T22:10:24.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff704aed700 1 -- 172.21.15.203:0/2629171446 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8011610 con 0x7ff700149150 2021-01-06T22:10:24.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff706af1700 1 -- 172.21.15.203:0/2629171446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff700149150 msgr2=0x7ff70014bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:24.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff706af1700 1 --2- 172.21.15.203:0/2629171446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff700149150 0x7ff70014bb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7ff6f8009d50 tx=0x7ff6f8009290).stop 2021-01-06T22:10:24.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff706af1700 1 -- 172.21.15.203:0/2629171446 shutdown_connections 2021-01-06T22:10:24.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff706af1700 1 --2- 172.21.15.203:0/2629171446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff700149150 0x7ff70014bb30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:24.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff706af1700 1 --2- 172.21.15.203:0/2629171446 >> v2:172.21.15.91:3300/0 conn(0x7ff7000602b0 0x7ff7000c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:24.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff706af1700 1 -- 172.21.15.203:0/2629171446 >> 172.21.15.203:0/2629171446 conn(0x7ff7000667b0 msgr2=0x7ff7000c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:24.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff706af1700 1 -- 172.21.15.203:0/2629171446 shutdown_connections 2021-01-06T22:10:24.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.057+0000 7ff706af1700 1 -- 172.21.15.203:0/2629171446 wait complete. 2021-01-06T22:10:24.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff706af1700 1 Processor -- start 2021-01-06T22:10:24.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff706af1700 1 -- start start 2021-01-06T22:10:24.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff706af1700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff7000602b0 0x7ff7000bd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:24.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff706af1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff700148d70 0x7ff7001441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:24.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff706af1700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff700149150 0x7ff7001446c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:24.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff706af1700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70015f720 con 0x7ff700149150 2021-01-06T22:10:24.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff706af1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000bfba0 con 0x7ff700148d70 2021-01-06T22:10:24.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff706af1700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000bfe40 con 0x7ff7000602b0 2021-01-06T22:10:24.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff7052ee700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff700148d70 0x7ff7001441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:24.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff7052ee700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff700148d70 0x7ff7001441d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51074/0 (socket says 172.21.15.203:51074) 2021-01-06T22:10:24.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff7052ee700 1 -- 172.21.15.203:0/535051698 learned_addr learned my addr 172.21.15.203:0/535051698 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:10:24.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff705aef700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff7000602b0 0x7ff7000bd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:24.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff7062f0700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff700149150 0x7ff7001446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:24.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff7052ee700 1 -- 172.21.15.203:0/535051698 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff7000602b0 msgr2=0x7ff7000bd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:24.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff7052ee700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff7000602b0 0x7ff7000bd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:24.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff7052ee700 1 -- 172.21.15.203:0/535051698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff700149150 msgr2=0x7ff7001446c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:24.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff7052ee700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff700149150 0x7ff7001446c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:24.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff7052ee700 1 -- 172.21.15.203:0/535051698 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7000c00e0 con 0x7ff700148d70 2021-01-06T22:10:24.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff7052ee700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff700148d70 0x7ff7001441d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7ff6f000ebd0 tx=0x7ff6f0006de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:24.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff6edffb700 1 -- 172.21.15.203:0/535051698 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f000c970 con 0x7ff700148d70 2021-01-06T22:10:24.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff6edffb700 1 -- 172.21.15.203:0/535051698 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f000cad0 con 0x7ff700148d70 2021-01-06T22:10:24.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff706af1700 1 -- 172.21.15.203:0/535051698 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7000c0380 con 0x7ff700148d70 2021-01-06T22:10:24.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff6edffb700 1 -- 172.21.15.203:0/535051698 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f000ce50 con 0x7ff700148d70 2021-01-06T22:10:24.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.061+0000 7ff706af1700 1 -- 172.21.15.203:0/535051698 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7000bd700 con 0x7ff700148d70 2021-01-06T22:10:24.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.065+0000 7ff706af1700 1 -- 172.21.15.203:0/535051698 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7000450a0 con 0x7ff700148d70 2021-01-06T22:10:24.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.065+0000 7ff6edffb700 1 -- 172.21.15.203:0/535051698 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff6f0012600 con 0x7ff700148d70 2021-01-06T22:10:24.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.065+0000 7ff6edffb700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6e0056400 0x7ff6e00588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:24.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.065+0000 7ff6edffb700 1 -- 172.21.15.203:0/535051698 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6417+0+0 (secure 0 0 0) 0x7ff6f006fa50 con 0x7ff700148d70 2021-01-06T22:10:24.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.065+0000 7ff705aef700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6e0056400 0x7ff6e00588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:24.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.065+0000 7ff705aef700 1 -- 172.21.15.203:0/535051698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6e0056400 msgr2=0x7ff6e00588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2021-01-06T22:10:24.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.065+0000 7ff705aef700 1 -- 172.21.15.203:0/535051698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6e0056400 msgr2=0x7ff6e00588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2021-01-06T22:10:24.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.065+0000 7ff705aef700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6e0056400 0x7ff6e00588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:10:24.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.065+0000 7ff705aef700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6e0056400 0x7ff6e00588c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-01-06T22:10:24.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.073+0000 7ff6edffb700 1 -- 172.21.15.203:0/535051698 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff6f00451f0 con 0x7ff700148d70 2021-01-06T22:10:24.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.269+0000 7ff705aef700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6e0056400 0x7ff6e00588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:24.272 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.269+0000 7ff705aef700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6e0056400 0x7ff6e00588c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7ff6fc007fd0 tx=0x7ff6fc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:24.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.297+0000 7ff706af1700 1 -- 172.21.15.203:0/535051698 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff700068870 con 0x7ff700148d70 2021-01-06T22:10:24.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.297+0000 7ff6edffb700 1 -- 172.21.15.203:0/535051698 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3210 (secure 0 0 0) 0x7ff6f0048810 con 0x7ff700148d70 2021-01-06T22:10:24.303 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:10:24.303 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:10:15.511645+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4859,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4859":{"gid":4859,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:10:24.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.301+0000 7ff706af1700 1 -- 172.21.15.203:0/535051698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6e0056400 msgr2=0x7ff6e00588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:24.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.301+0000 7ff706af1700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6e0056400 0x7ff6e00588c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7ff6fc007fd0 tx=0x7ff6fc009040).stop 2021-01-06T22:10:24.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.301+0000 7ff706af1700 1 -- 172.21.15.203:0/535051698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff700148d70 msgr2=0x7ff7001441d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:24.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.301+0000 7ff706af1700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff700148d70 0x7ff7001441d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7ff6f000ebd0 tx=0x7ff6f0006de0).stop 2021-01-06T22:10:24.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.301+0000 7ff706af1700 1 -- 172.21.15.203:0/535051698 shutdown_connections 2021-01-06T22:10:24.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.301+0000 7ff706af1700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6e0056400 0x7ff6e00588c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:24.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.301+0000 7ff706af1700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff700149150 0x7ff7001446c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:24.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.301+0000 7ff706af1700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff7000602b0 0x7ff7000bd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:24.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.301+0000 7ff706af1700 1 --2- 172.21.15.203:0/535051698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff700148d70 0x7ff7001441d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:24.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.301+0000 7ff706af1700 1 -- 172.21.15.203:0/535051698 >> 172.21.15.203:0/535051698 conn(0x7ff7000667b0 msgr2=0x7ff7000bc670 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:24.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.301+0000 7ff706af1700 1 -- 172.21.15.203:0/535051698 shutdown_connections 2021-01-06T22:10:24.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:24.301+0000 7ff706af1700 1 -- 172.21.15.203:0/535051698 wait complete. 2021-01-06T22:10:24.308 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 43 2021-01-06T22:10:24.319 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2021-01-06T22:10:26.319 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:10:26.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.441+0000 7f4dcc3ba700 1 Processor -- start 2021-01-06T22:10:26.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.441+0000 7f4dcc3ba700 1 -- start start 2021-01-06T22:10:26.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.441+0000 7f4dcc3ba700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4dc40bb460 0x7f4dc40673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:26.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.441+0000 7f4dcc3ba700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc40653e0 con 0x7f4dc4064f50 2021-01-06T22:10:26.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.441+0000 7f4dcc3ba700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc414b1a0 con 0x7f4dc4064b70 2021-01-06T22:10:26.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.441+0000 7f4dcc3ba700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc40bed30 con 0x7f4dc40bb460 2021-01-06T22:10:26.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.441+0000 7f4dca156700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f4dc4064b70 0x7f4dc4144aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49676/0 (socket says 172.21.15.203:49676) 2021-01-06T22:10:26.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.441+0000 7f4dca156700 1 -- 172.21.15.203:0/4007069535 learned_addr learned my addr 172.21.15.203:0/4007069535 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:10:26.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.441+0000 7f4dca957700 1 --2- 172.21.15.203:0/4007069535 >> v2:172.21.15.91:3300/0 conn(0x7f4dc40bb460 0x7f4dc40673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:26.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dc9154700 1 -- 172.21.15.203:0/4007069535 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501018372 0 0) 0x7f4dc414b1a0 con 0x7f4dc4064b70 2021-01-06T22:10:26.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dc9154700 1 -- 172.21.15.203:0/4007069535 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dac003230 con 0x7f4dc4064b70 2021-01-06T22:10:26.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dc9154700 1 -- 172.21.15.203:0/4007069535 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3422871926 0 0) 0x7f4dac003230 con 0x7f4dc4064b70 2021-01-06T22:10:26.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dca957700 1 -- 172.21.15.203:0/4007069535 >> v1:172.21.15.203:6789/0 conn(0x7f4dc4064b70 legacy=0x7f4dc4144aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:26.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dca957700 1 -- 172.21.15.203:0/4007069535 >> v1:172.21.15.91:6789/0 conn(0x7f4dc4064f50 legacy=0x7f4dc40bad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:26.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dca957700 1 -- 172.21.15.203:0/4007069535 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc4068230 con 0x7f4dc40bb460 2021-01-06T22:10:26.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dca957700 1 --2- 172.21.15.203:0/4007069535 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4dc40bb460 0x7f4dc40673a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f4dc40b9cf0 tx=0x7f4dbc0091d0).ready entity=mon.0 client_cookie=dedb03e4fe50c447 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:26.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dc9154700 1 -- 172.21.15.203:0/4007069535 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc00014e0 con 0x7f4dc40bb460 2021-01-06T22:10:26.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dc9154700 1 -- 172.21.15.203:0/4007069535 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dbc00b6b0 con 0x7f4dc40bb460 2021-01-06T22:10:26.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dc9154700 1 -- 172.21.15.203:0/4007069535 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc00ba30 con 0x7f4dc40bb460 2021-01-06T22:10:26.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dcc3ba700 1 -- 172.21.15.203:0/4007069535 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4dc40bb460 msgr2=0x7f4dc40673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:26.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dcc3ba700 1 --2- 172.21.15.203:0/4007069535 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4dc40bb460 0x7f4dc40673a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f4dc40b9cf0 tx=0x7f4dbc0091d0).stop 2021-01-06T22:10:26.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dcc3ba700 1 -- 172.21.15.203:0/4007069535 shutdown_connections 2021-01-06T22:10:26.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dcc3ba700 1 --2- 172.21.15.203:0/4007069535 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4dc40bb460 0x7f4dc40673a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:26.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dcc3ba700 1 -- 172.21.15.203:0/4007069535 >> 172.21.15.203:0/4007069535 conn(0x7f4dc4056cd0 msgr2=0x7f4dc40570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:26.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dcc3ba700 1 -- 172.21.15.203:0/4007069535 shutdown_connections 2021-01-06T22:10:26.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dcc3ba700 1 -- 172.21.15.203:0/4007069535 wait complete. 2021-01-06T22:10:26.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dcc3ba700 1 Processor -- start 2021-01-06T22:10:26.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.445+0000 7f4dcc3ba700 1 -- start start 2021-01-06T22:10:26.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dcc3ba700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dc4064b70 0x7f4dc4160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:26.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dcc3ba700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4dc4064f50 0x7f4dc415e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:26.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dcc3ba700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4dc40bb460 0x7f4dc4067540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:26.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dcc3ba700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc415d9d0 con 0x7f4dc4064f50 2021-01-06T22:10:26.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dcc3ba700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc4148e10 con 0x7f4dc4064b70 2021-01-06T22:10:26.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dcc3ba700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc4161490 con 0x7f4dc40bb460 2021-01-06T22:10:26.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dca156700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dc4064b70 0x7f4dc4160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:26.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dca156700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dc4064b70 0x7f4dc4160fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51088/0 (socket says 172.21.15.203:51088) 2021-01-06T22:10:26.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dca156700 1 -- 172.21.15.203:0/1015979812 learned_addr learned my addr 172.21.15.203:0/1015979812 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:10:26.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dca957700 1 --2- 172.21.15.203:0/1015979812 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4dc40bb460 0x7f4dc4067540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:26.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dc9955700 1 --2- 172.21.15.203:0/1015979812 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4dc4064f50 0x7f4dc415e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:26.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dca156700 1 -- 172.21.15.203:0/1015979812 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4dc40bb460 msgr2=0x7f4dc4067540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:26.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dca156700 1 --2- 172.21.15.203:0/1015979812 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4dc40bb460 0x7f4dc4067540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:26.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dca156700 1 -- 172.21.15.203:0/1015979812 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4dc4064f50 msgr2=0x7f4dc415e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:26.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dca156700 1 --2- 172.21.15.203:0/1015979812 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4dc4064f50 0x7f4dc415e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:26.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dca156700 1 -- 172.21.15.203:0/1015979812 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc41616e0 con 0x7f4dc4064b70 2021-01-06T22:10:26.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dca156700 1 --2- 172.21.15.203:0/1015979812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dc4064b70 0x7f4dc4160fa0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f4dc000cae0 tx=0x7f4dc0001a40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:26.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4db27fc700 1 -- 172.21.15.203:0/1015979812 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0007170 con 0x7f4dc4064b70 2021-01-06T22:10:26.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4db27fc700 1 -- 172.21.15.203:0/1015979812 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc00072d0 con 0x7f4dc4064b70 2021-01-06T22:10:26.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4db27fc700 1 -- 172.21.15.203:0/1015979812 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc000a960 con 0x7f4dc4064b70 2021-01-06T22:10:26.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dcc3ba700 1 -- 172.21.15.203:0/1015979812 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc4161980 con 0x7f4dc4064b70 2021-01-06T22:10:26.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4dcc3ba700 1 -- 172.21.15.203:0/1015979812 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc415dbc0 con 0x7f4dc4064b70 2021-01-06T22:10:26.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.449+0000 7f4db27fc700 1 -- 172.21.15.203:0/1015979812 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f4dc0010070 con 0x7f4dc4064b70 2021-01-06T22:10:26.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.453+0000 7f4db27fc700 1 --2- 172.21.15.203:0/1015979812 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4da40563d0 0x7f4da4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:10:26.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.453+0000 7f4db27fc700 1 -- 172.21.15.203:0/1015979812 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f4dc006f0f0 con 0x7f4dc4064b70 2021-01-06T22:10:26.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.453+0000 7f4dcc3ba700 1 -- 172.21.15.203:0/1015979812 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc4161e20 con 0x7f4dc4064b70 2021-01-06T22:10:26.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.453+0000 7f4dc9955700 1 --2- 172.21.15.203:0/1015979812 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4da40563d0 0x7f4da4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:10:26.462 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.453+0000 7f4dc9955700 1 --2- 172.21.15.203:0/1015979812 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4da40563d0 0x7f4da4058890 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f4db40011a0 tx=0x7f4db4007a30).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:10:26.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.461+0000 7f4db27fc700 1 -- 172.21.15.203:0/1015979812 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f4dc0044810 con 0x7f4dc4064b70 2021-01-06T22:10:26.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.685+0000 7f4dcc3ba700 1 -- 172.21.15.203:0/1015979812 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dc405d600 con 0x7f4dc4064b70 2021-01-06T22:10:26.690 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.685+0000 7f4db27fc700 1 -- 172.21.15.203:0/1015979812 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3563 (secure 0 0 0) 0x7f4dc004c020 con 0x7f4dc4064b70 2021-01-06T22:10:26.690 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:10:26.690 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4925,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:10:15.511645+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4859,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4859":{"gid":4859,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:10:26.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.689+0000 7f4dcc3ba700 1 -- 172.21.15.203:0/1015979812 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4da40563d0 msgr2=0x7f4da4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:26.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.689+0000 7f4dcc3ba700 1 --2- 172.21.15.203:0/1015979812 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4da40563d0 0x7f4da4058890 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f4db40011a0 tx=0x7f4db4007a30).stop 2021-01-06T22:10:26.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.689+0000 7f4dcc3ba700 1 -- 172.21.15.203:0/1015979812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dc4064b70 msgr2=0x7f4dc4160fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:10:26.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.689+0000 7f4dcc3ba700 1 --2- 172.21.15.203:0/1015979812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dc4064b70 0x7f4dc4160fa0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f4dc000cae0 tx=0x7f4dc0001a40).stop 2021-01-06T22:10:26.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.689+0000 7f4dcc3ba700 1 -- 172.21.15.203:0/1015979812 shutdown_connections 2021-01-06T22:10:26.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.689+0000 7f4dcc3ba700 1 --2- 172.21.15.203:0/1015979812 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4da40563d0 0x7f4da4058890 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:26.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.689+0000 7f4dcc3ba700 1 --2- 172.21.15.203:0/1015979812 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4dc40bb460 0x7f4dc4067540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:26.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.689+0000 7f4dcc3ba700 1 --2- 172.21.15.203:0/1015979812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dc4064b70 0x7f4dc4160fa0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:26.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.689+0000 7f4dcc3ba700 1 --2- 172.21.15.203:0/1015979812 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4dc4064f50 0x7f4dc415e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:10:26.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.689+0000 7f4dcc3ba700 1 -- 172.21.15.203:0/1015979812 >> 172.21.15.203:0/1015979812 conn(0x7f4dc4056cd0 msgr2=0x7f4dc41425a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:10:26.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.689+0000 7f4dcc3ba700 1 -- 172.21.15.203:0/1015979812 shutdown_connections 2021-01-06T22:10:26.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:10:26.689+0000 7f4dcc3ba700 1 -- 172.21.15.203:0/1015979812 wait complete. 2021-01-06T22:10:26.695 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 44 2021-01-06T22:10:26.705 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2021-01-06T22:10:26.705 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 79 secs before thrashing 2021-01-06T22:10:45.376 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:10:45.380 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:11:16.627 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:11:16.632 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:11:45.706 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:11:45.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.842+0000 7ff17c5c7700 1 Processor -- start 2021-01-06T22:11:45.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.842+0000 7ff17c5c7700 1 -- start start 2021-01-06T22:11:45.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.842+0000 7ff17c5c7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff174148320 0x7ff1740626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:45.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17c5c7700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17414f090 con 0x7ff1740602b0 2021-01-06T22:11:45.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17c5c7700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff174068780 con 0x7ff174062bd0 2021-01-06T22:11:45.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17c5c7700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1740badc0 con 0x7ff174148320 2021-01-06T22:11:45.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179b62700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7ff174062bd0 0x7ff1740bb650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49724/0 (socket says 172.21.15.203:49724) 2021-01-06T22:11:45.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17a363700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff174148320 0x7ff1740626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:45.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179b62700 1 -- 172.21.15.203:0/201065815 learned_addr learned my addr 172.21.15.203:0/201065815 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:11:45.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17a363700 1 -- 172.21.15.203:0/201065815 >> v2:172.21.15.203:3300/0 conn(0x7ff174148320 msgr2=0x7ff1740626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send send error: (32) Broken pipe 2021-01-06T22:11:45.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17a363700 1 --2- 172.21.15.203:0/201065815 >> v2:172.21.15.203:3300/0 conn(0x7ff174148320 0x7ff1740626e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).write auth request write failed r=-32 ((32) Broken pipe) 2021-01-06T22:11:45.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17a363700 1 --2- 172.21.15.203:0/201065815 >> v2:172.21.15.203:3300/0 conn(0x7ff174148320 0x7ff1740626e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-01-06T22:11:45.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 -- 172.21.15.203:0/201065815 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2054317578 0 0) 0x7ff174068780 con 0x7ff174062bd0 2021-01-06T22:11:45.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 -- 172.21.15.203:0/201065815 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff15c003230 con 0x7ff174062bd0 2021-01-06T22:11:45.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 -- 172.21.15.203:0/201065815 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4178399547 0 0) 0x7ff15c003230 con 0x7ff174062bd0 2021-01-06T22:11:45.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 -- 172.21.15.203:0/201065815 >> v2:172.21.15.203:3300/0 conn(0x7ff174148320 msgr2=0x7ff1740626e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-01-06T22:11:45.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 --2- 172.21.15.203:0/201065815 >> v2:172.21.15.203:3300/0 conn(0x7ff174148320 0x7ff1740626e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:45.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 -- 172.21.15.203:0/201065815 >> v1:172.21.15.91:6789/0 conn(0x7ff1740602b0 legacy=0x7ff174158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:45.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 -- 172.21.15.203:0/201065815 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17414e170 con 0x7ff174062bd0 2021-01-06T22:11:45.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 -- 172.21.15.203:0/201065815 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7ff1640014e0 con 0x7ff174062bd0 2021-01-06T22:11:45.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 -- 172.21.15.203:0/201065815 >> v1:172.21.15.203:6789/0 conn(0x7ff174062bd0 legacy=0x7ff1740bb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:45.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 --2- 172.21.15.203:0/201065815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff15c003720 0x7ff15c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:45.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 --2- 172.21.15.203:0/201065815 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff15c0044d0 0x7ff15c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:45.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 --2- 172.21.15.203:0/201065815 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff15c004e40 0x7ff15c00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:45.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 -- 172.21.15.203:0/201065815 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1740badc0 con 0x7ff15c004e40 2021-01-06T22:11:45.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 -- 172.21.15.203:0/201065815 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c004250 con 0x7ff15c003720 2021-01-06T22:11:45.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 -- 172.21.15.203:0/201065815 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c014420 con 0x7ff15c0044d0 2021-01-06T22:11:45.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17a363700 1 --2- 172.21.15.203:0/201065815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff15c003720 0x7ff15c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:45.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17ab64700 1 --2- 172.21.15.203:0/201065815 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff15c004e40 0x7ff15c00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:45.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179b62700 1 --2- 172.21.15.203:0/201065815 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff15c0044d0 0x7ff15c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:45.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17a363700 1 -- 172.21.15.203:0/201065815 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff15c0044d0 msgr2=0x7ff15c004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:11:45.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17a363700 1 --2- 172.21.15.203:0/201065815 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff15c0044d0 0x7ff15c004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:45.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17a363700 1 -- 172.21.15.203:0/201065815 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff15c004e40 msgr2=0x7ff15c00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:11:45.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17a363700 1 --2- 172.21.15.203:0/201065815 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff15c004e40 0x7ff15c00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:45.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17ab64700 1 -- 172.21.15.203:0/201065815 reap_dead start 2021-01-06T22:11:45.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17a363700 1 -- 172.21.15.203:0/201065815 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c003230 con 0x7ff15c003720 2021-01-06T22:11:45.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17a363700 1 --2- 172.21.15.203:0/201065815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff15c003720 0x7ff15c003b40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7ff17000bb40 tx=0x7ff170008040).ready entity=mon.1 client_cookie=18982e24fb9aa093 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:45.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 -- 172.21.15.203:0/201065815 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170016070 con 0x7ff15c003720 2021-01-06T22:11:45.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 -- 172.21.15.203:0/201065815 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17000da20 con 0x7ff15c003720 2021-01-06T22:11:45.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff179361700 1 -- 172.21.15.203:0/201065815 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17000dda0 con 0x7ff15c003720 2021-01-06T22:11:45.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17c5c7700 1 -- 172.21.15.203:0/201065815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff15c003720 msgr2=0x7ff15c003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:45.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17c5c7700 1 --2- 172.21.15.203:0/201065815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff15c003720 0x7ff15c003b40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7ff17000bb40 tx=0x7ff170008040).stop 2021-01-06T22:11:45.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17c5c7700 1 -- 172.21.15.203:0/201065815 shutdown_connections 2021-01-06T22:11:45.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17c5c7700 1 --2- 172.21.15.203:0/201065815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff15c003720 0x7ff15c003b40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:45.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17c5c7700 1 -- 172.21.15.203:0/201065815 >> 172.21.15.203:0/201065815 conn(0x7ff1740c0cb0 msgr2=0x7ff1740c1090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:11:45.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17c5c7700 1 -- 172.21.15.203:0/201065815 shutdown_connections 2021-01-06T22:11:45.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.846+0000 7ff17c5c7700 1 -- 172.21.15.203:0/201065815 wait complete. 2021-01-06T22:11:45.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff17c5c7700 1 Processor -- start 2021-01-06T22:11:45.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff17c5c7700 1 -- start start 2021-01-06T22:11:45.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff17c5c7700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff15c003720 0x7ff17414b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:45.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff17c5c7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff174141c90 0x7ff174142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:45.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff17c5c7700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1740b8470 0x7ff1740b88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:45.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff17c5c7700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17415a640 con 0x7ff15c003720 2021-01-06T22:11:45.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff17c5c7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17415a870 con 0x7ff174141c90 2021-01-06T22:11:45.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff17c5c7700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff174148b30 con 0x7ff1740b8470 2021-01-06T22:11:45.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff179b62700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff174141c90 0x7ff174142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:45.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff179b62700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff174141c90 0x7ff174142110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51142/0 (socket says 172.21.15.203:51142) 2021-01-06T22:11:45.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff179b62700 1 -- 172.21.15.203:0/114854419 learned_addr learned my addr 172.21.15.203:0/114854419 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:11:45.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff17a363700 1 --2- 172.21.15.203:0/114854419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff15c003720 0x7ff17414b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:45.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff17ab64700 1 --2- 172.21.15.203:0/114854419 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1740b8470 0x7ff1740b88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:45.878 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff179b62700 1 -- 172.21.15.203:0/114854419 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1740b8470 msgr2=0x7ff1740b88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:45.878 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff179b62700 1 --2- 172.21.15.203:0/114854419 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1740b8470 0x7ff1740b88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:45.879 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff179b62700 1 -- 172.21.15.203:0/114854419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff15c003720 msgr2=0x7ff17414b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:45.879 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff179b62700 1 --2- 172.21.15.203:0/114854419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff15c003720 0x7ff17414b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:45.879 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff179b62700 1 -- 172.21.15.203:0/114854419 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff174148dd0 con 0x7ff174141c90 2021-01-06T22:11:45.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff179b62700 1 --2- 172.21.15.203:0/114854419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff174141c90 0x7ff174142110 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7ff16400eae0 tx=0x7ff174063dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:45.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff1627fc700 1 -- 172.21.15.203:0/114854419 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16400c7b0 con 0x7ff174141c90 2021-01-06T22:11:45.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff1627fc700 1 -- 172.21.15.203:0/114854419 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16400c910 con 0x7ff174141c90 2021-01-06T22:11:45.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff1627fc700 1 -- 172.21.15.203:0/114854419 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16400ccb0 con 0x7ff174141c90 2021-01-06T22:11:45.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff17c5c7700 1 -- 172.21.15.203:0/114854419 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1741417f0 con 0x7ff174141c90 2021-01-06T22:11:45.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.850+0000 7ff17c5c7700 1 -- 172.21.15.203:0/114854419 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1740629d0 con 0x7ff174141c90 2021-01-06T22:11:45.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.854+0000 7ff17c5c7700 1 -- 172.21.15.203:0/114854419 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17415ab10 con 0x7ff174141c90 2021-01-06T22:11:45.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.854+0000 7ff1627fc700 1 -- 172.21.15.203:0/114854419 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff164014640 con 0x7ff174141c90 2021-01-06T22:11:45.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.854+0000 7ff1627fc700 1 --2- 172.21.15.203:0/114854419 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff1540563d0 0x7ff154058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:45.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.854+0000 7ff1627fc700 1 -- 172.21.15.203:0/114854419 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6417+0+0 (secure 0 0 0) 0x7ff16406f150 con 0x7ff174141c90 2021-01-06T22:11:45.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.854+0000 7ff17a363700 1 --2- 172.21.15.203:0/114854419 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff1540563d0 0x7ff154058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:45.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.854+0000 7ff17a363700 1 --2- 172.21.15.203:0/114854419 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff1540563d0 0x7ff154058890 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7ff17000d720 tx=0x7ff170008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:45.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:45.858+0000 7ff1627fc700 1 -- 172.21.15.203:0/114854419 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff17415ab10 con 0x7ff174141c90 2021-01-06T22:11:46.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.110+0000 7ff17c5c7700 1 -- 172.21.15.203:0/114854419 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff17415ab10 con 0x7ff174141c90 2021-01-06T22:11:46.115 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.110+0000 7ff1627fc700 1 -- 172.21.15.203:0/114854419 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3563 (secure 0 0 0) 0x7ff17415ab10 con 0x7ff174141c90 2021-01-06T22:11:46.115 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:11:46.116 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4925,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:10:15.511645+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4859,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4859":{"gid":4859,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:11:46.118 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.114+0000 7ff17c5c7700 1 -- 172.21.15.203:0/114854419 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff1540563d0 msgr2=0x7ff154058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:46.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.114+0000 7ff17c5c7700 1 --2- 172.21.15.203:0/114854419 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff1540563d0 0x7ff154058890 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7ff17000d720 tx=0x7ff170008040).stop 2021-01-06T22:11:46.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.114+0000 7ff17c5c7700 1 -- 172.21.15.203:0/114854419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff174141c90 msgr2=0x7ff174142110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:46.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.114+0000 7ff17c5c7700 1 --2- 172.21.15.203:0/114854419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff174141c90 0x7ff174142110 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7ff16400eae0 tx=0x7ff174063dd0).stop 2021-01-06T22:11:46.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.114+0000 7ff17c5c7700 1 -- 172.21.15.203:0/114854419 shutdown_connections 2021-01-06T22:11:46.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.114+0000 7ff17c5c7700 1 --2- 172.21.15.203:0/114854419 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff1540563d0 0x7ff154058890 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:46.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.114+0000 7ff17c5c7700 1 --2- 172.21.15.203:0/114854419 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1740b8470 0x7ff1740b88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:46.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.114+0000 7ff17c5c7700 1 --2- 172.21.15.203:0/114854419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff15c003720 0x7ff17414b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:46.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.114+0000 7ff17c5c7700 1 --2- 172.21.15.203:0/114854419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff174141c90 0x7ff174142110 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:46.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.114+0000 7ff17c5c7700 1 -- 172.21.15.203:0/114854419 >> 172.21.15.203:0/114854419 conn(0x7ff1740c0cb0 msgr2=0x7ff1740bf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:11:46.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.114+0000 7ff17c5c7700 1 -- 172.21.15.203:0/114854419 shutdown_connections 2021-01-06T22:11:46.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.114+0000 7ff17c5c7700 1 -- 172.21.15.203:0/114854419 wait complete. 2021-01-06T22:11:46.121 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 44 2021-01-06T22:11:46.130 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=0) 2021-01-06T22:11:46.130 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-01-06T22:11:46.130 INFO:teuthology.orchestra.run:waiting for 300 2021-01-06T22:11:46.140 INFO:tasks.ceph.mds.a:Stopped 2021-01-06T22:11:46.141 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:11:46.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.321+0000 7f12a9289700 1 Processor -- start 2021-01-06T22:11:46.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.321+0000 7f12a9289700 1 -- start start 2021-01-06T22:11:46.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.321+0000 7f12a9289700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f129c0099d0 0x7f129c04cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:46.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.321+0000 7f12a9289700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f129c048510 0x7f129c0d8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:46.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.321+0000 7f12a9289700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129c0470f0 con 0x7f129c009db0 2021-01-06T22:11:46.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.321+0000 7f12a9289700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129c046950 con 0x7f129c048510 2021-01-06T22:11:46.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.321+0000 7f12a9289700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129c04df40 con 0x7f129c0099d0 2021-01-06T22:11:46.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.321+0000 7f12a8a88700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f129c048510 0x7f129c0d8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:46.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.321+0000 7f12a8a88700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f129c048510 0x7f129c0d8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51154/0 (socket says 172.21.15.203:51154) 2021-01-06T22:11:46.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.321+0000 7f12a8a88700 1 -- 172.21.15.203:0/862879587 learned_addr learned my addr 172.21.15.203:0/862879587 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:11:46.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.321+0000 7f12a3fff700 1 --2- 172.21.15.203:0/862879587 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f129c0099d0 0x7f129c04cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:46.331 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.321+0000 7f12a8a88700 1 -- 172.21.15.203:0/862879587 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f129c0099d0 msgr2=0x7f129c04cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:11:46.331 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.321+0000 7f12a8a88700 1 --2- 172.21.15.203:0/862879587 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f129c0099d0 0x7f129c04cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:46.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.321+0000 7f12a8a88700 1 -- 172.21.15.203:0/862879587 >> v1:172.21.15.91:6789/0 conn(0x7f129c009db0 legacy=0x7f129c0d8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:46.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.321+0000 7f12a8a88700 1 -- 172.21.15.203:0/862879587 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c04e1e0 con 0x7f129c048510 2021-01-06T22:11:46.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.321+0000 7f12a3fff700 1 --2- 172.21.15.203:0/862879587 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f129c0099d0 0x7f129c04cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:11:46.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.325+0000 7f12a8a88700 1 --2- 172.21.15.203:0/862879587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c048510 0x7f129c0d8940 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f12a4060250 tx=0x7f12a4062440).ready entity=mon.1 client_cookie=be821d6026a7b000 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:46.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.325+0000 7f12a2ffd700 1 -- 172.21.15.203:0/862879587 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4044dc0 con 0x7f129c048510 2021-01-06T22:11:46.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.325+0000 7f12a2ffd700 1 -- 172.21.15.203:0/862879587 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a4084070 con 0x7f129c048510 2021-01-06T22:11:46.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.325+0000 7f12a2ffd700 1 -- 172.21.15.203:0/862879587 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a40626e0 con 0x7f129c048510 2021-01-06T22:11:46.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.325+0000 7f12a9289700 1 -- 172.21.15.203:0/862879587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c048510 msgr2=0x7f129c0d8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:46.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.325+0000 7f12a9289700 1 --2- 172.21.15.203:0/862879587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c048510 0x7f129c0d8940 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f12a4060250 tx=0x7f12a4062440).stop 2021-01-06T22:11:46.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.325+0000 7f12a9289700 1 -- 172.21.15.203:0/862879587 shutdown_connections 2021-01-06T22:11:46.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.325+0000 7f12a9289700 1 --2- 172.21.15.203:0/862879587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c048510 0x7f129c0d8940 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:46.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.325+0000 7f12a9289700 1 --2- 172.21.15.203:0/862879587 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f129c0099d0 0x7f129c04cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:46.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.325+0000 7f12a9289700 1 -- 172.21.15.203:0/862879587 >> 172.21.15.203:0/862879587 conn(0x7f129c00d9b0 msgr2=0x7f129c00dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:11:46.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.325+0000 7f12a9289700 1 -- 172.21.15.203:0/862879587 shutdown_connections 2021-01-06T22:11:46.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.325+0000 7f12a9289700 1 -- 172.21.15.203:0/862879587 wait complete. 2021-01-06T22:11:46.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.325+0000 7f12a9289700 1 Processor -- start 2021-01-06T22:11:46.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.325+0000 7f12a9289700 1 -- start start 2021-01-06T22:11:46.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a9289700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f129c0099d0 0x7f129c0d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:46.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a9289700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f129c009db0 0x7f129c0e0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:46.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a9289700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c048510 0x7f129c0e1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:46.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a9289700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c0d65b0 con 0x7f129c0099d0 2021-01-06T22:11:46.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a9289700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c0d1430 con 0x7f129c048510 2021-01-06T22:11:46.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a9289700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129c0d16d0 con 0x7f129c009db0 2021-01-06T22:11:46.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a8a88700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c048510 0x7f129c0e1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:46.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a8a88700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c048510 0x7f129c0e1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51160/0 (socket says 172.21.15.203:51160) 2021-01-06T22:11:46.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a8a88700 1 -- 172.21.15.203:0/180036963 learned_addr learned my addr 172.21.15.203:0/180036963 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:11:46.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a3fff700 1 --2- 172.21.15.203:0/180036963 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f129c0099d0 0x7f129c0d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:46.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a37fe700 1 --2- 172.21.15.203:0/180036963 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f129c009db0 0x7f129c0e0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:46.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a8a88700 1 -- 172.21.15.203:0/180036963 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f129c009db0 msgr2=0x7f129c0e0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:46.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a8a88700 1 --2- 172.21.15.203:0/180036963 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f129c009db0 0x7f129c0e0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:46.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a8a88700 1 -- 172.21.15.203:0/180036963 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f129c0099d0 msgr2=0x7f129c0d5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:46.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a8a88700 1 --2- 172.21.15.203:0/180036963 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f129c0099d0 0x7f129c0d5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:46.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a8a88700 1 -- 172.21.15.203:0/180036963 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c0d1970 con 0x7f129c048510 2021-01-06T22:11:46.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a37fe700 1 --2- 172.21.15.203:0/180036963 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f129c009db0 0x7f129c0e0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:11:46.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a8a88700 1 --2- 172.21.15.203:0/180036963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c048510 0x7f129c0e1290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f12a405cf70 tx=0x7f12a4087000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:46.342 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f128ffff700 1 -- 172.21.15.203:0/180036963 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a408e070 con 0x7f129c048510 2021-01-06T22:11:46.342 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f128ffff700 1 -- 172.21.15.203:0/180036963 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a4068360 con 0x7f129c048510 2021-01-06T22:11:46.342 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f128ffff700 1 -- 172.21.15.203:0/180036963 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a40686e0 con 0x7f129c048510 2021-01-06T22:11:46.342 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a9289700 1 -- 172.21.15.203:0/180036963 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129c0d1c10 con 0x7f129c048510 2021-01-06T22:11:46.342 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a9289700 1 -- 172.21.15.203:0/180036963 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129c0d62f0 con 0x7f129c048510 2021-01-06T22:11:46.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.329+0000 7f12a9289700 1 -- 172.21.15.203:0/180036963 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f129c04c840 con 0x7f129c048510 2021-01-06T22:11:46.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.333+0000 7f128ffff700 1 -- 172.21.15.203:0/180036963 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f12a4061640 con 0x7f129c048510 2021-01-06T22:11:46.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.333+0000 7f128ffff700 1 --2- 172.21.15.203:0/180036963 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f12840563d0 0x7f1284058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:46.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.333+0000 7f128ffff700 1 -- 172.21.15.203:0/180036963 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f12a40e08d0 con 0x7f129c048510 2021-01-06T22:11:46.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.333+0000 7f12a3fff700 1 --2- 172.21.15.203:0/180036963 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f12840563d0 0x7f1284058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:46.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.333+0000 7f12a3fff700 1 --2- 172.21.15.203:0/180036963 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f12840563d0 0x7f1284058890 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f1298006fd0 tx=0x7f1298008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:46.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.337+0000 7f128ffff700 1 -- 172.21.15.203:0/180036963 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f129c04c840 con 0x7f129c048510 2021-01-06T22:11:46.581 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.577+0000 7f12a9289700 1 -- 172.21.15.203:0/180036963 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f129c04c840 con 0x7f129c048510 2021-01-06T22:11:46.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.577+0000 7f128ffff700 1 -- 172.21.15.203:0/180036963 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3563 (secure 0 0 0) 0x7f129c04c840 con 0x7f129c048510 2021-01-06T22:11:46.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.577+0000 7f12a9289700 1 -- 172.21.15.203:0/180036963 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f12840563d0 msgr2=0x7f1284058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:46.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.577+0000 7f12a9289700 1 --2- 172.21.15.203:0/180036963 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f12840563d0 0x7f1284058890 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f1298006fd0 tx=0x7f1298008040).stop 2021-01-06T22:11:46.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.577+0000 7f12a9289700 1 -- 172.21.15.203:0/180036963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c048510 msgr2=0x7f129c0e1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:46.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.577+0000 7f12a9289700 1 --2- 172.21.15.203:0/180036963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c048510 0x7f129c0e1290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f12a405cf70 tx=0x7f12a4087000).stop 2021-01-06T22:11:46.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.577+0000 7f12a9289700 1 -- 172.21.15.203:0/180036963 shutdown_connections 2021-01-06T22:11:46.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.577+0000 7f12a9289700 1 --2- 172.21.15.203:0/180036963 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f12840563d0 0x7f1284058890 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:46.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.577+0000 7f12a9289700 1 --2- 172.21.15.203:0/180036963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c048510 0x7f129c0e1290 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:46.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.577+0000 7f12a9289700 1 --2- 172.21.15.203:0/180036963 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f129c0099d0 0x7f129c0d5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:46.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.577+0000 7f12a9289700 1 --2- 172.21.15.203:0/180036963 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f129c009db0 0x7f129c0e0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:46.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.577+0000 7f12a9289700 1 -- 172.21.15.203:0/180036963 >> 172.21.15.203:0/180036963 conn(0x7f129c00d9b0 msgr2=0x7f129c04fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:11:46.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.577+0000 7f12a9289700 1 -- 172.21.15.203:0/180036963 shutdown_connections 2021-01-06T22:11:46.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:46.577+0000 7f12a9289700 1 -- 172.21.15.203:0/180036963 wait complete. 2021-01-06T22:11:46.587 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:11:46.587 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4925,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:10:15.511645+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4859,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4859":{"gid":4859,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:11:46.587 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 44 2021-01-06T22:11:46.599 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:11:46.769 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:11:46.776 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:11:48.601 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:11:48.733 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f416113a700 1 Processor -- start 2021-01-06T22:11:48.733 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f416113a700 1 -- start start 2021-01-06T22:11:48.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f416113a700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f415c0bf290 0x7f415c063730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:48.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f416113a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f415c0bf670 0x7f415c0c1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:48.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f416113a700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415c14eb50 con 0x7f415c066c20 2021-01-06T22:11:48.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f416113a700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f415c0c0f80 con 0x7f415c0bf290 2021-01-06T22:11:48.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f416113a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f415c0c1220 con 0x7f415c0bf670 2021-01-06T22:11:48.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f415a59c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f415c0bf670 0x7f415c0c1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:48.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f415a59c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f415c0bf670 0x7f415c0c1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51180/0 (socket says 172.21.15.203:51180) 2021-01-06T22:11:48.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f415a59c700 1 -- 172.21.15.203:0/513592218 learned_addr learned my addr 172.21.15.203:0/513592218 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:11:48.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f415b59e700 1 --1- 172.21.15.203:0/513592218 >> v1:172.21.15.203:6789/0 conn(0x7f415c066c20 0x7f415c14df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49778/0 (socket says 172.21.15.203:49778) 2021-01-06T22:11:48.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f415ad9d700 1 --2- 172.21.15.203:0/513592218 >> v2:172.21.15.91:3300/0 conn(0x7f415c0bf290 0x7f415c063730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:48.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f415a59c700 1 -- 172.21.15.203:0/513592218 >> v2:172.21.15.91:3300/0 conn(0x7f415c0bf290 msgr2=0x7f415c063730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:11:48.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f415a59c700 1 --2- 172.21.15.203:0/513592218 >> v2:172.21.15.91:3300/0 conn(0x7f415c0bf290 0x7f415c063730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:48.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f415a59c700 1 -- 172.21.15.203:0/513592218 >> v1:172.21.15.203:6789/0 conn(0x7f415c066c20 legacy=0x7f415c14df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:48.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f4159d9b700 1 -- 172.21.15.203:0/513592218 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717524910 0 0) 0x7f415c14eb50 con 0x7f415c066c20 2021-01-06T22:11:48.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f415a59c700 1 -- 172.21.15.203:0/513592218 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c0c14c0 con 0x7f415c0bf670 2021-01-06T22:11:48.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f415a59c700 1 --2- 172.21.15.203:0/513592218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c0bf670 0x7f415c0c1c00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f415c0674b0 tx=0x7f4144006ea0).ready entity=mon.1 client_cookie=c02fdb0f8af1969b server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:48.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f4159d9b700 1 -- 172.21.15.203:0/513592218 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144009960 con 0x7f415c0bf670 2021-01-06T22:11:48.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f4159d9b700 1 -- 172.21.15.203:0/513592218 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4144009ac0 con 0x7f415c0bf670 2021-01-06T22:11:48.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f4159d9b700 1 -- 172.21.15.203:0/513592218 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144009e40 con 0x7f415c0bf670 2021-01-06T22:11:48.743 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f416113a700 1 -- 172.21.15.203:0/513592218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c0bf670 msgr2=0x7f415c0c1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:48.744 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f416113a700 1 --2- 172.21.15.203:0/513592218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c0bf670 0x7f415c0c1c00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f415c0674b0 tx=0x7f4144006ea0).stop 2021-01-06T22:11:48.744 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f416113a700 1 -- 172.21.15.203:0/513592218 shutdown_connections 2021-01-06T22:11:48.744 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f416113a700 1 --2- 172.21.15.203:0/513592218 >> v2:172.21.15.91:3300/0 conn(0x7f415c0bf290 0x7f415c063730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:48.744 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f416113a700 1 --2- 172.21.15.203:0/513592218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c0bf670 0x7f415c0c1c00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:48.745 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.729+0000 7f416113a700 1 -- 172.21.15.203:0/513592218 >> 172.21.15.203:0/513592218 conn(0x7f415c0626e0 msgr2=0x7f415c061a30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:11:48.745 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f416113a700 1 -- 172.21.15.203:0/513592218 shutdown_connections 2021-01-06T22:11:48.745 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f416113a700 1 -- 172.21.15.203:0/513592218 wait complete. 2021-01-06T22:11:48.745 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f416113a700 1 Processor -- start 2021-01-06T22:11:48.746 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f416113a700 1 -- start start 2021-01-06T22:11:48.746 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f416113a700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c066c20 0x7f415c15e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:48.746 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f416113a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c0bf290 0x7f415c141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:48.746 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f416113a700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415c0bf670 0x7f415c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:48.746 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f416113a700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c15e830 con 0x7f415c066c20 2021-01-06T22:11:48.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f416113a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c15b8a0 con 0x7f415c0bf290 2021-01-06T22:11:48.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f416113a700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c0b8a60 con 0x7f415c0bf670 2021-01-06T22:11:48.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f415a59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c0bf290 0x7f415c141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:48.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f415a59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c0bf290 0x7f415c141e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51188/0 (socket says 172.21.15.203:51188) 2021-01-06T22:11:48.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f415a59c700 1 -- 172.21.15.203:0/3871265599 learned_addr learned my addr 172.21.15.203:0/3871265599 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:11:48.748 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f415ad9d700 1 --2- 172.21.15.203:0/3871265599 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c066c20 0x7f415c15e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:48.748 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f415b59e700 1 --2- 172.21.15.203:0/3871265599 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415c0bf670 0x7f415c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:48.748 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f415a59c700 1 -- 172.21.15.203:0/3871265599 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415c0bf670 msgr2=0x7f415c15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:48.748 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f415a59c700 1 --2- 172.21.15.203:0/3871265599 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415c0bf670 0x7f415c15bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f415a59c700 1 -- 172.21.15.203:0/3871265599 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c066c20 msgr2=0x7f415c15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f415a59c700 1 --2- 172.21.15.203:0/3871265599 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c066c20 0x7f415c15e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f415a59c700 1 -- 172.21.15.203:0/3871265599 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c0b8d00 con 0x7f415c0bf290 2021-01-06T22:11:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f415ad9d700 1 --2- 172.21.15.203:0/3871265599 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c066c20 0x7f415c15e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:11:48.749 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f415a59c700 1 --2- 172.21.15.203:0/3871265599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c0bf290 0x7f415c141e00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f4144010000 tx=0x7f4144016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:48.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f4152ffd700 1 -- 172.21.15.203:0/3871265599 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144011d50 con 0x7f415c0bf290 2021-01-06T22:11:48.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f4152ffd700 1 -- 172.21.15.203:0/3871265599 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414401d430 con 0x7f415c0bf290 2021-01-06T22:11:48.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f4152ffd700 1 -- 172.21.15.203:0/3871265599 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414401d7d0 con 0x7f415c0bf290 2021-01-06T22:11:48.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f416113a700 1 -- 172.21.15.203:0/3871265599 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415c0b8fa0 con 0x7f415c0bf290 2021-01-06T22:11:48.750 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.733+0000 7f416113a700 1 -- 172.21.15.203:0/3871265599 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415c0b94e0 con 0x7f415c0bf290 2021-01-06T22:11:48.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.737+0000 7f416113a700 1 -- 172.21.15.203:0/3871265599 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f415c15ba10 con 0x7f415c0bf290 2021-01-06T22:11:48.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.737+0000 7f4152ffd700 1 -- 172.21.15.203:0/3871265599 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f414400f070 con 0x7f415c0bf290 2021-01-06T22:11:48.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.737+0000 7f4152ffd700 1 --2- 172.21.15.203:0/3871265599 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41340563d0 0x7f4134058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:48.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.737+0000 7f4152ffd700 1 -- 172.21.15.203:0/3871265599 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f4144071050 con 0x7f415c0bf290 2021-01-06T22:11:48.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.737+0000 7f415ad9d700 1 --2- 172.21.15.203:0/3871265599 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41340563d0 0x7f4134058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:48.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.737+0000 7f415ad9d700 1 --2- 172.21.15.203:0/3871265599 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41340563d0 0x7f4134058890 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f414c00d990 tx=0x7f414c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:48.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.741+0000 7f4152ffd700 1 -- 172.21.15.203:0/3871265599 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f415c15ba10 con 0x7f415c0bf290 2021-01-06T22:11:48.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.945+0000 7f416113a700 1 -- 172.21.15.203:0/3871265599 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f415c05cfe0 con 0x7f415c0bf290 2021-01-06T22:11:48.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.945+0000 7f4152ffd700 1 -- 172.21.15.203:0/3871265599 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3563 (secure 0 0 0) 0x7f41440467b0 con 0x7f415c0bf290 2021-01-06T22:11:48.951 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:11:48.952 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4925,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:10:15.511645+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4859,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4859":{"gid":4859,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:11:48.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.949+0000 7f416113a700 1 -- 172.21.15.203:0/3871265599 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41340563d0 msgr2=0x7f4134058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:48.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.949+0000 7f416113a700 1 --2- 172.21.15.203:0/3871265599 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41340563d0 0x7f4134058890 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f414c00d990 tx=0x7f414c008040).stop 2021-01-06T22:11:48.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.949+0000 7f416113a700 1 -- 172.21.15.203:0/3871265599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c0bf290 msgr2=0x7f415c141e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:48.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.949+0000 7f416113a700 1 --2- 172.21.15.203:0/3871265599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c0bf290 0x7f415c141e00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f4144010000 tx=0x7f4144016000).stop 2021-01-06T22:11:48.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.949+0000 7f416113a700 1 -- 172.21.15.203:0/3871265599 shutdown_connections 2021-01-06T22:11:48.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.949+0000 7f416113a700 1 --2- 172.21.15.203:0/3871265599 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41340563d0 0x7f4134058890 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:48.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.949+0000 7f416113a700 1 --2- 172.21.15.203:0/3871265599 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415c0bf670 0x7f415c15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:48.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.949+0000 7f416113a700 1 --2- 172.21.15.203:0/3871265599 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c066c20 0x7f415c15e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:48.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.949+0000 7f416113a700 1 --2- 172.21.15.203:0/3871265599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c0bf290 0x7f415c141e00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:48.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.949+0000 7f416113a700 1 -- 172.21.15.203:0/3871265599 >> 172.21.15.203:0/3871265599 conn(0x7f415c0626e0 msgr2=0x7f415c14cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:11:48.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.949+0000 7f416113a700 1 -- 172.21.15.203:0/3871265599 shutdown_connections 2021-01-06T22:11:48.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:48.949+0000 7f416113a700 1 -- 172.21.15.203:0/3871265599 wait complete. 2021-01-06T22:11:48.957 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 44 2021-01-06T22:11:48.971 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:11:50.972 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:11:51.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.097+0000 7f70af59e700 1 Processor -- start 2021-01-06T22:11:51.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.097+0000 7f70af59e700 1 -- start start 2021-01-06T22:11:51.105 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70af59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70b00c0fa0 0x7f70b00bccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:51.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70af59e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f70b014ad90 0x7f70b014b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:51.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70af59e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b00bacb0 con 0x7f70b01513f0 2021-01-06T22:11:51.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70af59e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b014b6b0 con 0x7f70b014ad90 2021-01-06T22:11:51.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70af59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b0152000 con 0x7f70b00c0fa0 2021-01-06T22:11:51.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70ae59c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f70b01513f0 0x7f70b01517d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49794/0 (socket says 172.21.15.203:49794) 2021-01-06T22:11:51.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70ae59c700 1 -- 172.21.15.203:0/684571918 learned_addr learned my addr 172.21.15.203:0/684571918 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:11:51.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70aed9d700 1 --2- 172.21.15.203:0/684571918 >> v2:172.21.15.203:3300/0 conn(0x7f70b014ad90 0x7f70b014b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:51.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70add9b700 1 --2- 172.21.15.203:0/684571918 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70b00c0fa0 0x7f70b00bccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:51.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70ad59a700 1 -- 172.21.15.203:0/684571918 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661654766 0 0) 0x7f70a0001450 con 0x7f70b01513f0 2021-01-06T22:11:51.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70ad59a700 1 -- 172.21.15.203:0/684571918 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7094003230 con 0x7f70b01513f0 2021-01-06T22:11:51.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70aed9d700 1 -- 172.21.15.203:0/684571918 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70b00c0fa0 msgr2=0x7f70b00bccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:11:51.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70aed9d700 1 --2- 172.21.15.203:0/684571918 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70b00c0fa0 0x7f70b00bccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:51.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70aed9d700 1 -- 172.21.15.203:0/684571918 >> v1:172.21.15.203:6789/0 conn(0x7f70b01513f0 legacy=0x7f70b01517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:51.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70aed9d700 1 -- 172.21.15.203:0/684571918 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b0067e50 con 0x7f70b014ad90 2021-01-06T22:11:51.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70aed9d700 1 --2- 172.21.15.203:0/684571918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b014ad90 0x7f70b014b1c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f70a800b980 tx=0x7f70a8006de0).ready entity=mon.1 client_cookie=176d2c34a32e4001 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:51.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70ad59a700 1 -- 172.21.15.203:0/684571918 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8009640 con 0x7f70b014ad90 2021-01-06T22:11:51.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70ad59a700 1 -- 172.21.15.203:0/684571918 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a80097a0 con 0x7f70b014ad90 2021-01-06T22:11:51.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70ad59a700 1 -- 172.21.15.203:0/684571918 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8009b20 con 0x7f70b014ad90 2021-01-06T22:11:51.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70af59e700 1 -- 172.21.15.203:0/684571918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b014ad90 msgr2=0x7f70b014b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:51.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.101+0000 7f70af59e700 1 --2- 172.21.15.203:0/684571918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b014ad90 0x7f70b014b1c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f70a800b980 tx=0x7f70a8006de0).stop 2021-01-06T22:11:51.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.105+0000 7f70af59e700 1 -- 172.21.15.203:0/684571918 shutdown_connections 2021-01-06T22:11:51.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.105+0000 7f70af59e700 1 --2- 172.21.15.203:0/684571918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b014ad90 0x7f70b014b1c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:51.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.105+0000 7f70af59e700 1 --2- 172.21.15.203:0/684571918 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70b00c0fa0 0x7f70b00bccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:51.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.105+0000 7f70af59e700 1 -- 172.21.15.203:0/684571918 >> 172.21.15.203:0/684571918 conn(0x7f70b00667b0 msgr2=0x7f70b00c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:11:51.114 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.105+0000 7f70af59e700 1 -- 172.21.15.203:0/684571918 shutdown_connections 2021-01-06T22:11:51.114 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.105+0000 7f70af59e700 1 -- 172.21.15.203:0/684571918 wait complete. 2021-01-06T22:11:51.116 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.105+0000 7f70af59e700 1 Processor -- start 2021-01-06T22:11:51.117 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.105+0000 7f70af59e700 1 -- start start 2021-01-06T22:11:51.117 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70af59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b00c0fa0 0x7f70b0148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:51.117 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70af59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70b014ad90 0x7f70b01477a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:51.118 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70af59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70b01513f0 0x7f70b0141090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:51.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70af59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b01609f0 con 0x7f70b01513f0 2021-01-06T22:11:51.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70af59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b0160c20 con 0x7f70b00c0fa0 2021-01-06T22:11:51.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70af59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b0160ec0 con 0x7f70b014ad90 2021-01-06T22:11:51.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70ae59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b00c0fa0 0x7f70b0148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:51.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70ae59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b00c0fa0 0x7f70b0148dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51206/0 (socket says 172.21.15.203:51206) 2021-01-06T22:11:51.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70ae59c700 1 -- 172.21.15.203:0/188092420 learned_addr learned my addr 172.21.15.203:0/188092420 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:11:51.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70add9b700 1 --2- 172.21.15.203:0/188092420 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70b014ad90 0x7f70b01477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:51.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70aed9d700 1 --2- 172.21.15.203:0/188092420 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70b01513f0 0x7f70b0141090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:51.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70ae59c700 1 -- 172.21.15.203:0/188092420 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70b014ad90 msgr2=0x7f70b01477a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:51.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70ae59c700 1 --2- 172.21.15.203:0/188092420 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70b014ad90 0x7f70b01477a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:51.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70ae59c700 1 -- 172.21.15.203:0/188092420 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70b01513f0 msgr2=0x7f70b0141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:51.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70ae59c700 1 --2- 172.21.15.203:0/188092420 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70b01513f0 0x7f70b0141090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:51.122 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70ae59c700 1 -- 172.21.15.203:0/188092420 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b0161160 con 0x7f70b00c0fa0 2021-01-06T22:11:51.122 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70add9b700 1 --2- 172.21.15.203:0/188092420 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70b014ad90 0x7f70b01477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:11:51.123 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70aed9d700 1 --2- 172.21.15.203:0/188092420 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70b01513f0 0x7f70b0141090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:11:51.123 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70ae59c700 1 --2- 172.21.15.203:0/188092420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b00c0fa0 0x7f70b0148dc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f70a000cac0 tx=0x7f70a0007c70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:51.123 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f709e7fc700 1 -- 172.21.15.203:0/188092420 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0005b70 con 0x7f70b00c0fa0 2021-01-06T22:11:51.123 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f709e7fc700 1 -- 172.21.15.203:0/188092420 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a0005cd0 con 0x7f70b00c0fa0 2021-01-06T22:11:51.123 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70af59e700 1 -- 172.21.15.203:0/188092420 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b0161400 con 0x7f70b00c0fa0 2021-01-06T22:11:51.124 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f709e7fc700 1 -- 172.21.15.203:0/188092420 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0007340 con 0x7f70b00c0fa0 2021-01-06T22:11:51.124 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.109+0000 7f70af59e700 1 -- 172.21.15.203:0/188092420 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b0150d60 con 0x7f70b00c0fa0 2021-01-06T22:11:51.124 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.113+0000 7f709e7fc700 1 -- 172.21.15.203:0/188092420 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f70a0010030 con 0x7f70b00c0fa0 2021-01-06T22:11:51.124 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.113+0000 7f70af59e700 1 -- 172.21.15.203:0/188092420 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b00450a0 con 0x7f70b00c0fa0 2021-01-06T22:11:51.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.113+0000 7f709e7fc700 1 --2- 172.21.15.203:0/188092420 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f708c0563d0 0x7f708c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:51.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.113+0000 7f709e7fc700 1 -- 172.21.15.203:0/188092420 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f70a006f5c0 con 0x7f70b00c0fa0 2021-01-06T22:11:51.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.113+0000 7f70add9b700 1 --2- 172.21.15.203:0/188092420 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f708c0563d0 0x7f708c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:51.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.113+0000 7f70add9b700 1 --2- 172.21.15.203:0/188092420 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f708c0563d0 0x7f708c058890 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f70b0148b80 tx=0x7f70a4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:51.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.117+0000 7f709e7fc700 1 -- 172.21.15.203:0/188092420 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f70a0044c80 con 0x7f70b00c0fa0 2021-01-06T22:11:51.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.317+0000 7f70af59e700 1 -- 172.21.15.203:0/188092420 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70b00bacb0 con 0x7f70b00c0fa0 2021-01-06T22:11:51.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.317+0000 7f709e7fc700 1 -- 172.21.15.203:0/188092420 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3563 (secure 0 0 0) 0x7f70b00bacb0 con 0x7f70b00c0fa0 2021-01-06T22:11:51.324 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:11:51.324 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4925,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:10:15.511645+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4859,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4859":{"gid":4859,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:11:51.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.321+0000 7f70af59e700 1 -- 172.21.15.203:0/188092420 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f708c0563d0 msgr2=0x7f708c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:51.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.321+0000 7f70af59e700 1 --2- 172.21.15.203:0/188092420 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f708c0563d0 0x7f708c058890 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f70b0148b80 tx=0x7f70a4008040).stop 2021-01-06T22:11:51.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.321+0000 7f70af59e700 1 -- 172.21.15.203:0/188092420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b00c0fa0 msgr2=0x7f70b0148dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:51.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.321+0000 7f70af59e700 1 --2- 172.21.15.203:0/188092420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b00c0fa0 0x7f70b0148dc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f70a000cac0 tx=0x7f70a0007c70).stop 2021-01-06T22:11:51.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.321+0000 7f70af59e700 1 -- 172.21.15.203:0/188092420 shutdown_connections 2021-01-06T22:11:51.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.321+0000 7f70af59e700 1 --2- 172.21.15.203:0/188092420 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f708c0563d0 0x7f708c058890 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:51.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.321+0000 7f70af59e700 1 --2- 172.21.15.203:0/188092420 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70b01513f0 0x7f70b0141090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:51.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.321+0000 7f70af59e700 1 --2- 172.21.15.203:0/188092420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b00c0fa0 0x7f70b0148dc0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:51.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.321+0000 7f70af59e700 1 --2- 172.21.15.203:0/188092420 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70b014ad90 0x7f70b01477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:51.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.321+0000 7f70af59e700 1 -- 172.21.15.203:0/188092420 >> 172.21.15.203:0/188092420 conn(0x7f70b00667b0 msgr2=0x7f70b00ba280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:11:51.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.321+0000 7f70af59e700 1 -- 172.21.15.203:0/188092420 shutdown_connections 2021-01-06T22:11:51.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:51.321+0000 7f70af59e700 1 -- 172.21.15.203:0/188092420 wait complete. 2021-01-06T22:11:51.329 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 44 2021-01-06T22:11:51.339 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:11:53.341 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:11:53.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267759e700 1 Processor -- start 2021-01-06T22:11:53.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267759e700 1 -- start start 2021-01-06T22:11:53.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267759e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f26780632f0 0x7f267814e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:53.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267759e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26780c11a0 con 0x7f2678062e70 2021-01-06T22:11:53.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267759e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26780c1440 con 0x7f2678060ba0 2021-01-06T22:11:53.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267759e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26780c16e0 con 0x7f26780632f0 2021-01-06T22:11:53.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267659c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f2678060ba0 0x7f26780bbab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49812/0 (socket says 172.21.15.203:49812) 2021-01-06T22:11:53.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267659c700 1 -- 172.21.15.203:0/2491002141 learned_addr learned my addr 172.21.15.203:0/2491002141 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:11:53.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f2676d9d700 1 --2- 172.21.15.203:0/2491002141 >> v2:172.21.15.91:3300/0 conn(0x7f26780632f0 0x7f267814e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:53.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267559a700 1 -- 172.21.15.203:0/2491002141 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 178763301 0 0) 0x7f26780c1440 con 0x7f2678060ba0 2021-01-06T22:11:53.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267559a700 1 -- 172.21.15.203:0/2491002141 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2658003230 con 0x7f2678060ba0 2021-01-06T22:11:53.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267559a700 1 -- 172.21.15.203:0/2491002141 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3172025382 0 0) 0x7f2658003230 con 0x7f2678060ba0 2021-01-06T22:11:53.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267559a700 1 -- 172.21.15.203:0/2491002141 >> v2:172.21.15.91:3300/0 conn(0x7f26780632f0 msgr2=0x7f267814e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:11:53.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267559a700 1 --2- 172.21.15.203:0/2491002141 >> v2:172.21.15.91:3300/0 conn(0x7f26780632f0 0x7f267814e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:53.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267559a700 1 -- 172.21.15.203:0/2491002141 >> v1:172.21.15.91:6789/0 conn(0x7f2678062e70 legacy=0x7f267814dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:53.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267559a700 1 -- 172.21.15.203:0/2491002141 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26780c1980 con 0x7f2678060ba0 2021-01-06T22:11:53.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267559a700 1 -- 172.21.15.203:0/2491002141 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f26680014e0 con 0x7f2678060ba0 2021-01-06T22:11:53.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267559a700 1 -- 172.21.15.203:0/2491002141 >> v1:172.21.15.203:6789/0 conn(0x7f2678060ba0 legacy=0x7f26780bbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:53.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267559a700 1 --2- 172.21.15.203:0/2491002141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2658003720 0x7f2658003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:53.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267559a700 1 --2- 172.21.15.203:0/2491002141 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26580044d0 0x7f2658004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:53.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267559a700 1 --2- 172.21.15.203:0/2491002141 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2658004e40 0x7f265800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:53.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267559a700 1 -- 172.21.15.203:0/2491002141 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26780c16e0 con 0x7f26580044d0 2021-01-06T22:11:53.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267559a700 1 -- 172.21.15.203:0/2491002141 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2658004250 con 0x7f2658003720 2021-01-06T22:11:53.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.469+0000 7f267559a700 1 -- 172.21.15.203:0/2491002141 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265800bc10 con 0x7f2658004e40 2021-01-06T22:11:53.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267659c700 1 --2- 172.21.15.203:0/2491002141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2658003720 0x7f2658003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:53.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f2675d9b700 1 --2- 172.21.15.203:0/2491002141 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26580044d0 0x7f2658004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:53.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f2676d9d700 1 --2- 172.21.15.203:0/2491002141 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2658004e40 0x7f265800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:53.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267659c700 1 -- 172.21.15.203:0/2491002141 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2658004e40 msgr2=0x7f265800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:11:53.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267659c700 1 --2- 172.21.15.203:0/2491002141 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2658004e40 0x7f265800b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:53.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267659c700 1 -- 172.21.15.203:0/2491002141 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26580044d0 msgr2=0x7f2658004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:11:53.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267659c700 1 --2- 172.21.15.203:0/2491002141 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26580044d0 0x7f2658004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:53.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267659c700 1 -- 172.21.15.203:0/2491002141 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2658003230 con 0x7f2658003720 2021-01-06T22:11:53.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f2675d9b700 1 --2- 172.21.15.203:0/2491002141 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26580044d0 0x7f2658004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:11:53.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f2676d9d700 1 --2- 172.21.15.203:0/2491002141 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2658004e40 0x7f265800b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:11:53.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f2676d9d700 1 -- 172.21.15.203:0/2491002141 reap_dead start 2021-01-06T22:11:53.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267659c700 1 --2- 172.21.15.203:0/2491002141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2658003720 0x7f2658003b40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f266800c410 tx=0x7f26680077c0).ready entity=mon.1 client_cookie=e3ecd00a10ecfcd2 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:53.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267559a700 1 -- 172.21.15.203:0/2491002141 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266800ce80 con 0x7f2658003720 2021-01-06T22:11:53.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267559a700 1 -- 172.21.15.203:0/2491002141 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26680072e0 con 0x7f2658003720 2021-01-06T22:11:53.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267559a700 1 -- 172.21.15.203:0/2491002141 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26680057f0 con 0x7f2658003720 2021-01-06T22:11:53.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267759e700 1 -- 172.21.15.203:0/2491002141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2658003720 msgr2=0x7f2658003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:53.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267759e700 1 --2- 172.21.15.203:0/2491002141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2658003720 0x7f2658003b40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f266800c410 tx=0x7f26680077c0).stop 2021-01-06T22:11:53.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267759e700 1 -- 172.21.15.203:0/2491002141 shutdown_connections 2021-01-06T22:11:53.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267759e700 1 --2- 172.21.15.203:0/2491002141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2658003720 0x7f2658003b40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:53.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267759e700 1 -- 172.21.15.203:0/2491002141 >> 172.21.15.203:0/2491002141 conn(0x7f2678056cd0 msgr2=0x7f26780570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:11:53.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267759e700 1 -- 172.21.15.203:0/2491002141 shutdown_connections 2021-01-06T22:11:53.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267759e700 1 -- 172.21.15.203:0/2491002141 wait complete. 2021-01-06T22:11:53.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267759e700 1 Processor -- start 2021-01-06T22:11:53.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267759e700 1 -- start start 2021-01-06T22:11:53.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267759e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2658003720 0x7f26781559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:53.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267759e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26780b1760 0x7f2678154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:53.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267759e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2678154600 0x7f2678149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:53.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267759e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2678155e90 con 0x7f2658003720 2021-01-06T22:11:53.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267759e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2678154a80 con 0x7f2678154600 2021-01-06T22:11:53.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267759e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2678155460 con 0x7f26780b1760 2021-01-06T22:11:53.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f2676d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2678154600 0x7f2678149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:53.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f2676d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2678154600 0x7f2678149f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51234/0 (socket says 172.21.15.203:51234) 2021-01-06T22:11:53.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f2676d9d700 1 -- 172.21.15.203:0/1632835077 learned_addr learned my addr 172.21.15.203:0/1632835077 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:11:53.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f2675d9b700 1 --2- 172.21.15.203:0/1632835077 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26780b1760 0x7f2678154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:53.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.473+0000 7f267659c700 1 --2- 172.21.15.203:0/1632835077 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2658003720 0x7f26781559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:53.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.477+0000 7f2676d9d700 1 -- 172.21.15.203:0/1632835077 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26780b1760 msgr2=0x7f2678154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:53.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.477+0000 7f2676d9d700 1 --2- 172.21.15.203:0/1632835077 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26780b1760 0x7f2678154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:53.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.477+0000 7f2676d9d700 1 -- 172.21.15.203:0/1632835077 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2658003720 msgr2=0x7f26781559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:53.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.477+0000 7f2676d9d700 1 --2- 172.21.15.203:0/1632835077 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2658003720 0x7f26781559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:53.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.477+0000 7f2676d9d700 1 -- 172.21.15.203:0/1632835077 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2678155700 con 0x7f2678154600 2021-01-06T22:11:53.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.477+0000 7f2676d9d700 1 --2- 172.21.15.203:0/1632835077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2678154600 0x7f2678149f80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f2678066ea0 tx=0x7f2670008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:53.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.477+0000 7f266e7fc700 1 -- 172.21.15.203:0/1632835077 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670014670 con 0x7f2678154600 2021-01-06T22:11:53.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.477+0000 7f266e7fc700 1 -- 172.21.15.203:0/1632835077 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26700147d0 con 0x7f2678154600 2021-01-06T22:11:53.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.477+0000 7f267759e700 1 -- 172.21.15.203:0/1632835077 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26780b5620 con 0x7f2678154600 2021-01-06T22:11:53.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.477+0000 7f266e7fc700 1 -- 172.21.15.203:0/1632835077 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670014b50 con 0x7f2678154600 2021-01-06T22:11:53.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.477+0000 7f267759e700 1 -- 172.21.15.203:0/1632835077 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26780b5b60 con 0x7f2678154600 2021-01-06T22:11:53.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.477+0000 7f267759e700 1 -- 172.21.15.203:0/1632835077 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26780b5d20 con 0x7f2678154600 2021-01-06T22:11:53.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.477+0000 7f266e7fc700 1 -- 172.21.15.203:0/1632835077 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f267001f460 con 0x7f2678154600 2021-01-06T22:11:53.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.481+0000 7f266e7fc700 1 --2- 172.21.15.203:0/1632835077 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f265c0563d0 0x7f265c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:53.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.481+0000 7f266e7fc700 1 -- 172.21.15.203:0/1632835077 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f267006faf0 con 0x7f2678154600 2021-01-06T22:11:53.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.481+0000 7f267659c700 1 --2- 172.21.15.203:0/1632835077 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f265c0563d0 0x7f265c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:53.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.481+0000 7f267659c700 1 --2- 172.21.15.203:0/1632835077 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f265c0563d0 0x7f265c058890 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f266800c410 tx=0x7f2668015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:53.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.481+0000 7f266e7fc700 1 -- 172.21.15.203:0/1632835077 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f26700451b0 con 0x7f2678154600 2021-01-06T22:11:53.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.681+0000 7f267759e700 1 -- 172.21.15.203:0/1632835077 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26780450a0 con 0x7f2678154600 2021-01-06T22:11:53.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.685+0000 7f266e7fc700 1 -- 172.21.15.203:0/1632835077 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3563 (secure 0 0 0) 0x7f26700487d0 con 0x7f2678154600 2021-01-06T22:11:53.689 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:11:53.689 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4925,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:10:15.511645+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4859,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4859":{"gid":4859,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:11:53.691 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.685+0000 7f267759e700 1 -- 172.21.15.203:0/1632835077 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f265c0563d0 msgr2=0x7f265c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:53.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.685+0000 7f267759e700 1 --2- 172.21.15.203:0/1632835077 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f265c0563d0 0x7f265c058890 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f266800c410 tx=0x7f2668015000).stop 2021-01-06T22:11:53.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.685+0000 7f267759e700 1 -- 172.21.15.203:0/1632835077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2678154600 msgr2=0x7f2678149f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:53.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.685+0000 7f267759e700 1 --2- 172.21.15.203:0/1632835077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2678154600 0x7f2678149f80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f2678066ea0 tx=0x7f2670008040).stop 2021-01-06T22:11:53.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.685+0000 7f267759e700 1 -- 172.21.15.203:0/1632835077 shutdown_connections 2021-01-06T22:11:53.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.685+0000 7f267759e700 1 --2- 172.21.15.203:0/1632835077 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f265c0563d0 0x7f265c058890 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:53.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.685+0000 7f267759e700 1 --2- 172.21.15.203:0/1632835077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2678154600 0x7f2678149f80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:53.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.685+0000 7f267759e700 1 --2- 172.21.15.203:0/1632835077 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2658003720 0x7f26781559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:53.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.685+0000 7f267759e700 1 --2- 172.21.15.203:0/1632835077 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26780b1760 0x7f2678154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:53.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.685+0000 7f267759e700 1 -- 172.21.15.203:0/1632835077 >> 172.21.15.203:0/1632835077 conn(0x7f2678056cd0 msgr2=0x7f26780c05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:11:53.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.685+0000 7f267759e700 1 -- 172.21.15.203:0/1632835077 shutdown_connections 2021-01-06T22:11:53.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:53.685+0000 7f267759e700 1 -- 172.21.15.203:0/1632835077 wait complete. 2021-01-06T22:11:53.694 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 44 2021-01-06T22:11:53.704 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:11:55.705 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:11:55.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd1852cb700 1 Processor -- start 2021-01-06T22:11:55.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd1852cb700 1 -- start start 2021-01-06T22:11:55.843 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd1852cb700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd1800bb3f0 0x7fd1800ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:55.843 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd1852cb700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd1800bb7d0 0x7fd18014eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:55.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd1852cb700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1800646b0 con 0x7fd1800a5e50 2021-01-06T22:11:55.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd1852cb700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1800670c0 con 0x7fd1800bb7d0 2021-01-06T22:11:55.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd1852cb700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd180064450 con 0x7fd1800bb3f0 2021-01-06T22:11:55.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd17f7fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fd1800a5e50 0x7fd1800669b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49834/0 (socket says 172.21.15.203:49834) 2021-01-06T22:11:55.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd17f7fe700 1 -- 172.21.15.203:0/258643191 learned_addr learned my addr 172.21.15.203:0/258643191 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:11:55.845 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd17ffff700 1 --2- 172.21.15.203:0/258643191 >> v2:172.21.15.203:3300/0 conn(0x7fd1800bb3f0 0x7fd1800ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:55.845 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd184aca700 1 --2- 172.21.15.203:0/258643191 >> v2:172.21.15.91:3300/0 conn(0x7fd1800bb7d0 0x7fd18014eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:55.845 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd17effd700 1 -- 172.21.15.203:0/258643191 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1565381135 0 0) 0x7fd1800646b0 con 0x7fd1800a5e50 2021-01-06T22:11:55.845 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd17effd700 1 -- 172.21.15.203:0/258643191 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd160003230 con 0x7fd1800a5e50 2021-01-06T22:11:55.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd17ffff700 1 -- 172.21.15.203:0/258643191 >> v2:172.21.15.91:3300/0 conn(0x7fd1800bb7d0 msgr2=0x7fd18014eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:11:55.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd17ffff700 1 --2- 172.21.15.203:0/258643191 >> v2:172.21.15.91:3300/0 conn(0x7fd1800bb7d0 0x7fd18014eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:55.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd17ffff700 1 -- 172.21.15.203:0/258643191 >> v1:172.21.15.203:6789/0 conn(0x7fd1800a5e50 legacy=0x7fd1800669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:55.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd17ffff700 1 -- 172.21.15.203:0/258643191 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd180152cc0 con 0x7fd1800bb3f0 2021-01-06T22:11:55.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd17ffff700 1 --2- 172.21.15.203:0/258643191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1800bb3f0 0x7fd1800ba9d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fd1800c1040 tx=0x7fd1780091d0).ready entity=mon.1 client_cookie=cf997d9d0da90bd1 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:55.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd17effd700 1 -- 172.21.15.203:0/258643191 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17800b9c0 con 0x7fd1800bb3f0 2021-01-06T22:11:55.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd17effd700 1 -- 172.21.15.203:0/258643191 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17800bb20 con 0x7fd1800bb3f0 2021-01-06T22:11:55.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd17effd700 1 -- 172.21.15.203:0/258643191 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17800bea0 con 0x7fd1800bb3f0 2021-01-06T22:11:55.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd1852cb700 1 -- 172.21.15.203:0/258643191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1800bb3f0 msgr2=0x7fd1800ba9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:55.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.837+0000 7fd1852cb700 1 --2- 172.21.15.203:0/258643191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1800bb3f0 0x7fd1800ba9d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fd1800c1040 tx=0x7fd1780091d0).stop 2021-01-06T22:11:55.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd1852cb700 1 -- 172.21.15.203:0/258643191 shutdown_connections 2021-01-06T22:11:55.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd1852cb700 1 --2- 172.21.15.203:0/258643191 >> v2:172.21.15.91:3300/0 conn(0x7fd1800bb7d0 0x7fd18014eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:55.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd1852cb700 1 --2- 172.21.15.203:0/258643191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1800bb3f0 0x7fd1800ba9d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:55.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd1852cb700 1 -- 172.21.15.203:0/258643191 >> 172.21.15.203:0/258643191 conn(0x7fd180061530 msgr2=0x7fd180063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:11:55.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd1852cb700 1 -- 172.21.15.203:0/258643191 shutdown_connections 2021-01-06T22:11:55.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd1852cb700 1 -- 172.21.15.203:0/258643191 wait complete. 2021-01-06T22:11:55.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd1852cb700 1 Processor -- start 2021-01-06T22:11:55.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd1852cb700 1 -- start start 2021-01-06T22:11:55.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd1852cb700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1800a5e50 0x7fd180165d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:55.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd1852cb700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd1800bb7d0 0x7fd1801514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:55.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd1852cb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180155540 0x7fd1800b8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:55.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd1852cb700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd180166210 con 0x7fd1800a5e50 2021-01-06T22:11:55.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd1852cb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1800a1e10 con 0x7fd180155540 2021-01-06T22:11:55.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd1852cb700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1800a1630 con 0x7fd1800bb7d0 2021-01-06T22:11:55.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd184aca700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180155540 0x7fd1800b8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:55.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd184aca700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180155540 0x7fd1800b8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51250/0 (socket says 172.21.15.203:51250) 2021-01-06T22:11:55.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd184aca700 1 -- 172.21.15.203:0/1817280830 learned_addr learned my addr 172.21.15.203:0/1817280830 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:11:55.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd17f7fe700 1 --2- 172.21.15.203:0/1817280830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd1800bb7d0 0x7fd1801514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:55.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd17ffff700 1 --2- 172.21.15.203:0/1817280830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1800a5e50 0x7fd180165d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:55.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd184aca700 1 -- 172.21.15.203:0/1817280830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd1800bb7d0 msgr2=0x7fd1801514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:55.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd184aca700 1 --2- 172.21.15.203:0/1817280830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd1800bb7d0 0x7fd1801514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:55.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd184aca700 1 -- 172.21.15.203:0/1817280830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1800a5e50 msgr2=0x7fd180165d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:55.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd184aca700 1 --2- 172.21.15.203:0/1817280830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1800a5e50 0x7fd180165d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:55.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd184aca700 1 -- 172.21.15.203:0/1817280830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1800a18d0 con 0x7fd180155540 2021-01-06T22:11:55.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd184aca700 1 --2- 172.21.15.203:0/1817280830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180155540 0x7fd1800b8e00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fd17000ebd0 tx=0x7fd170006de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:55.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd177fff700 1 -- 172.21.15.203:0/1817280830 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17000b8d0 con 0x7fd180155540 2021-01-06T22:11:55.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd177fff700 1 -- 172.21.15.203:0/1817280830 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17000ba30 con 0x7fd180155540 2021-01-06T22:11:55.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd177fff700 1 -- 172.21.15.203:0/1817280830 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17000bdd0 con 0x7fd180155540 2021-01-06T22:11:55.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd1852cb700 1 -- 172.21.15.203:0/1817280830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1800a1b70 con 0x7fd180155540 2021-01-06T22:11:55.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.841+0000 7fd1852cb700 1 -- 172.21.15.203:0/1817280830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1800b86c0 con 0x7fd180155540 2021-01-06T22:11:55.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.845+0000 7fd177fff700 1 -- 172.21.15.203:0/1817280830 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fd170012600 con 0x7fd180155540 2021-01-06T22:11:55.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.845+0000 7fd1852cb700 1 -- 172.21.15.203:0/1817280830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1800450a0 con 0x7fd180155540 2021-01-06T22:11:55.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.845+0000 7fd177fff700 1 --2- 172.21.15.203:0/1817280830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd1640563d0 0x7fd164058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:55.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.845+0000 7fd177fff700 1 -- 172.21.15.203:0/1817280830 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fd17006fa90 con 0x7fd180155540 2021-01-06T22:11:55.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.845+0000 7fd17ffff700 1 --2- 172.21.15.203:0/1817280830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd1640563d0 0x7fd164058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:55.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.845+0000 7fd17ffff700 1 --2- 172.21.15.203:0/1817280830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd1640563d0 0x7fd164058890 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fd180057d00 tx=0x7fd178017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:55.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:55.853+0000 7fd177fff700 1 -- 172.21.15.203:0/1817280830 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fd1700451b0 con 0x7fd180155540 2021-01-06T22:11:56.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:56.053+0000 7fd1852cb700 1 -- 172.21.15.203:0/1817280830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd180152580 con 0x7fd180155540 2021-01-06T22:11:56.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:56.053+0000 7fd177fff700 1 -- 172.21.15.203:0/1817280830 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3563 (secure 0 0 0) 0x7fd1700487d0 con 0x7fd180155540 2021-01-06T22:11:56.059 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:11:56.059 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4925,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:10:15.511645+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4859,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4859":{"gid":4859,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:11:56.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:56.057+0000 7fd1852cb700 1 -- 172.21.15.203:0/1817280830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd1640563d0 msgr2=0x7fd164058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:56.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:56.057+0000 7fd1852cb700 1 --2- 172.21.15.203:0/1817280830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd1640563d0 0x7fd164058890 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fd180057d00 tx=0x7fd178017000).stop 2021-01-06T22:11:56.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:56.057+0000 7fd1852cb700 1 -- 172.21.15.203:0/1817280830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180155540 msgr2=0x7fd1800b8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:56.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:56.057+0000 7fd1852cb700 1 --2- 172.21.15.203:0/1817280830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180155540 0x7fd1800b8e00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fd17000ebd0 tx=0x7fd170006de0).stop 2021-01-06T22:11:56.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:56.057+0000 7fd1852cb700 1 -- 172.21.15.203:0/1817280830 shutdown_connections 2021-01-06T22:11:56.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:56.057+0000 7fd1852cb700 1 --2- 172.21.15.203:0/1817280830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd1640563d0 0x7fd164058890 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:56.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:56.057+0000 7fd1852cb700 1 --2- 172.21.15.203:0/1817280830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180155540 0x7fd1800b8e00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:56.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:56.057+0000 7fd1852cb700 1 --2- 172.21.15.203:0/1817280830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1800a5e50 0x7fd180165d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:56.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:56.057+0000 7fd1852cb700 1 --2- 172.21.15.203:0/1817280830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd1800bb7d0 0x7fd1801514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:56.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:56.057+0000 7fd1852cb700 1 -- 172.21.15.203:0/1817280830 >> 172.21.15.203:0/1817280830 conn(0x7fd180061530 msgr2=0x7fd18005bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:11:56.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:56.057+0000 7fd1852cb700 1 -- 172.21.15.203:0/1817280830 shutdown_connections 2021-01-06T22:11:56.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:56.057+0000 7fd1852cb700 1 -- 172.21.15.203:0/1817280830 wait complete. 2021-01-06T22:11:56.064 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 44 2021-01-06T22:11:56.073 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:11:58.075 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:11:58.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.225+0000 7fa2c1217700 1 Processor -- start 2021-01-06T22:11:58.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.225+0000 7fa2c1217700 1 -- start start 2021-01-06T22:11:58.236 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2c1217700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa2bc1513f0 0x7fa2bc0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:58.236 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2c1217700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa2bc0bccb0 0x7fa2bc1517d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:58.237 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2c1217700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2bc0bacb0 con 0x7fa2bc14ad90 2021-01-06T22:11:58.237 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2c1217700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2bc0bd160 con 0x7fa2bc0bccb0 2021-01-06T22:11:58.238 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2c1217700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2bc152000 con 0x7fa2bc1513f0 2021-01-06T22:11:58.238 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2bbfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa2bc1513f0 0x7fa2bc0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:58.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2bbfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa2bc1513f0 0x7fa2bc0c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51254/0 (socket says 172.21.15.203:51254) 2021-01-06T22:11:58.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2bbfff700 1 -- 172.21.15.203:0/2661341014 learned_addr learned my addr 172.21.15.203:0/2661341014 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:11:58.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2bb7fe700 1 --2- 172.21.15.203:0/2661341014 >> v2:172.21.15.91:3300/0 conn(0x7fa2bc0bccb0 0x7fa2bc1517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:58.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2bbfff700 1 -- 172.21.15.203:0/2661341014 >> v2:172.21.15.91:3300/0 conn(0x7fa2bc0bccb0 msgr2=0x7fa2bc1517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:11:58.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2bbfff700 1 --2- 172.21.15.203:0/2661341014 >> v2:172.21.15.91:3300/0 conn(0x7fa2bc0bccb0 0x7fa2bc1517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:58.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2bbfff700 1 -- 172.21.15.203:0/2661341014 >> v1:172.21.15.91:6789/0 conn(0x7fa2bc14ad90 legacy=0x7fa2bc0641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:11:58.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2bbfff700 1 -- 172.21.15.203:0/2661341014 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc067e50 con 0x7fa2bc1513f0 2021-01-06T22:11:58.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2bbfff700 1 --2- 172.21.15.203:0/2661341014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2bc1513f0 0x7fa2bc0c0fa0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fa2bc061ee0 tx=0x7fa2ac006ea0).ready entity=mon.1 client_cookie=444c61a4540c720f server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:58.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2baffd700 1 -- 172.21.15.203:0/2661341014 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac009890 con 0x7fa2bc1513f0 2021-01-06T22:11:58.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2baffd700 1 -- 172.21.15.203:0/2661341014 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ac0099f0 con 0x7fa2bc1513f0 2021-01-06T22:11:58.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.229+0000 7fa2baffd700 1 -- 172.21.15.203:0/2661341014 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac009d70 con 0x7fa2bc1513f0 2021-01-06T22:11:58.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.233+0000 7fa2c1217700 1 -- 172.21.15.203:0/2661341014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2bc1513f0 msgr2=0x7fa2bc0c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:58.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.233+0000 7fa2c1217700 1 --2- 172.21.15.203:0/2661341014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2bc1513f0 0x7fa2bc0c0fa0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fa2bc061ee0 tx=0x7fa2ac006ea0).stop 2021-01-06T22:11:58.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.233+0000 7fa2c1217700 1 -- 172.21.15.203:0/2661341014 shutdown_connections 2021-01-06T22:11:58.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.233+0000 7fa2c1217700 1 --2- 172.21.15.203:0/2661341014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2bc1513f0 0x7fa2bc0c0fa0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:58.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.233+0000 7fa2c1217700 1 --2- 172.21.15.203:0/2661341014 >> v2:172.21.15.91:3300/0 conn(0x7fa2bc0bccb0 0x7fa2bc1517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:58.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.233+0000 7fa2c1217700 1 -- 172.21.15.203:0/2661341014 >> 172.21.15.203:0/2661341014 conn(0x7fa2bc0667b0 msgr2=0x7fa2bc0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:11:58.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.233+0000 7fa2c1217700 1 -- 172.21.15.203:0/2661341014 shutdown_connections 2021-01-06T22:11:58.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.233+0000 7fa2c1217700 1 -- 172.21.15.203:0/2661341014 wait complete. 2021-01-06T22:11:58.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.233+0000 7fa2c1217700 1 Processor -- start 2021-01-06T22:11:58.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.233+0000 7fa2c1217700 1 -- start start 2021-01-06T22:11:58.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c1217700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2bc0bccb0 0x7fa2bc141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:58.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c1217700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2bc14ad90 0x7fa2bc150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:58.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c1217700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2bc1513f0 0x7fa2bc1467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:58.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c1217700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc146cd0 con 0x7fa2bc0bccb0 2021-01-06T22:11:58.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c1217700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc147f40 con 0x7fa2bc1513f0 2021-01-06T22:11:58.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c1217700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc14fcb0 con 0x7fa2bc14ad90 2021-01-06T22:11:58.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c0a16700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2bc1513f0 0x7fa2bc1467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:58.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2bbfff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2bc0bccb0 0x7fa2bc141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:58.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c0a16700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2bc1513f0 0x7fa2bc1467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51264/0 (socket says 172.21.15.203:51264) 2021-01-06T22:11:58.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c0a16700 1 -- 172.21.15.203:0/187927300 learned_addr learned my addr 172.21.15.203:0/187927300 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:11:58.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2bb7fe700 1 --2- 172.21.15.203:0/187927300 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2bc14ad90 0x7fa2bc150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:58.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c0a16700 1 -- 172.21.15.203:0/187927300 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2bc14ad90 msgr2=0x7fa2bc150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:58.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c0a16700 1 --2- 172.21.15.203:0/187927300 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2bc14ad90 0x7fa2bc150bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:58.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c0a16700 1 -- 172.21.15.203:0/187927300 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2bc0bccb0 msgr2=0x7fa2bc141e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:58.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c0a16700 1 --2- 172.21.15.203:0/187927300 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2bc0bccb0 0x7fa2bc141e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:58.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c0a16700 1 -- 172.21.15.203:0/187927300 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc14ff50 con 0x7fa2bc1513f0 2021-01-06T22:11:58.262 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c0a16700 1 --2- 172.21.15.203:0/187927300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2bc1513f0 0x7fa2bc1467e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fa2b400cae0 tx=0x7fa2b40023b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:58.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2a3fff700 1 -- 172.21.15.203:0/187927300 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b400a730 con 0x7fa2bc1513f0 2021-01-06T22:11:58.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2a3fff700 1 -- 172.21.15.203:0/187927300 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b400a8b0 con 0x7fa2bc1513f0 2021-01-06T22:11:58.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c1217700 1 -- 172.21.15.203:0/187927300 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2bc1501f0 con 0x7fa2bc1513f0 2021-01-06T22:11:58.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2a3fff700 1 -- 172.21.15.203:0/187927300 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b400ac70 con 0x7fa2bc1513f0 2021-01-06T22:11:58.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.237+0000 7fa2c1217700 1 -- 172.21.15.203:0/187927300 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2bc150490 con 0x7fa2bc1513f0 2021-01-06T22:11:58.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.241+0000 7fa2c1217700 1 -- 172.21.15.203:0/187927300 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2bc0450a0 con 0x7fa2bc1513f0 2021-01-06T22:11:58.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.241+0000 7fa2a3fff700 1 -- 172.21.15.203:0/187927300 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa2b4019030 con 0x7fa2bc1513f0 2021-01-06T22:11:58.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.241+0000 7fa2a3fff700 1 --2- 172.21.15.203:0/187927300 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa29c0563d0 0x7fa29c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:11:58.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.241+0000 7fa2a3fff700 1 -- 172.21.15.203:0/187927300 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fa2b406eee0 con 0x7fa2bc1513f0 2021-01-06T22:11:58.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.241+0000 7fa2bbfff700 1 --2- 172.21.15.203:0/187927300 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa29c0563d0 0x7fa29c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:11:58.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.241+0000 7fa2bbfff700 1 --2- 172.21.15.203:0/187927300 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa29c0563d0 0x7fa29c058890 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fa2ac007fd0 tx=0x7fa2ac016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:11:58.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.245+0000 7fa2a3fff700 1 -- 172.21.15.203:0/187927300 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa2b40446b0 con 0x7fa2bc1513f0 2021-01-06T22:11:58.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.449+0000 7fa2c1217700 1 -- 172.21.15.203:0/187927300 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2bc05ca60 con 0x7fa2bc1513f0 2021-01-06T22:11:58.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.449+0000 7fa2a3fff700 1 -- 172.21.15.203:0/187927300 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3563 (secure 0 0 0) 0x7fa2b4021e00 con 0x7fa2bc1513f0 2021-01-06T22:11:58.455 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:11:58.456 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4925,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:10:15.511645+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4859,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4859":{"gid":4859,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:11:58.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.453+0000 7fa2c1217700 1 -- 172.21.15.203:0/187927300 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa29c0563d0 msgr2=0x7fa29c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:58.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.453+0000 7fa2c1217700 1 --2- 172.21.15.203:0/187927300 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa29c0563d0 0x7fa29c058890 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fa2ac007fd0 tx=0x7fa2ac016000).stop 2021-01-06T22:11:58.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.453+0000 7fa2c1217700 1 -- 172.21.15.203:0/187927300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2bc1513f0 msgr2=0x7fa2bc1467e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:11:58.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.453+0000 7fa2c1217700 1 --2- 172.21.15.203:0/187927300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2bc1513f0 0x7fa2bc1467e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fa2b400cae0 tx=0x7fa2b40023b0).stop 2021-01-06T22:11:58.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.453+0000 7fa2c1217700 1 -- 172.21.15.203:0/187927300 shutdown_connections 2021-01-06T22:11:58.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.453+0000 7fa2c1217700 1 --2- 172.21.15.203:0/187927300 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa29c0563d0 0x7fa29c058890 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:58.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.453+0000 7fa2c1217700 1 --2- 172.21.15.203:0/187927300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2bc1513f0 0x7fa2bc1467e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:58.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.453+0000 7fa2c1217700 1 --2- 172.21.15.203:0/187927300 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2bc0bccb0 0x7fa2bc141e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:58.462 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.453+0000 7fa2c1217700 1 --2- 172.21.15.203:0/187927300 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2bc14ad90 0x7fa2bc150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:11:58.462 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.453+0000 7fa2c1217700 1 -- 172.21.15.203:0/187927300 >> 172.21.15.203:0/187927300 conn(0x7fa2bc0667b0 msgr2=0x7fa2bc0becb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:11:58.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.453+0000 7fa2c1217700 1 -- 172.21.15.203:0/187927300 shutdown_connections 2021-01-06T22:11:58.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:11:58.453+0000 7fa2c1217700 1 -- 172.21.15.203:0/187927300 wait complete. 2021-01-06T22:11:58.463 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 44 2021-01-06T22:11:58.474 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:12:00.476 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:12:00.610 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.605+0000 7fce6f59e700 1 Processor -- start 2021-01-06T22:12:00.611 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.605+0000 7fce6f59e700 1 -- start start 2021-01-06T22:12:00.614 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.605+0000 7fce6f59e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fce700a5680 0x7fce7014fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:00.614 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.605+0000 7fce6f59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce70067660 con 0x7fce70060fe0 2021-01-06T22:12:00.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.605+0000 7fce6f59e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce70064ab0 con 0x7fce700a5a60 2021-01-06T22:12:00.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.605+0000 7fce6f59e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce701500f0 con 0x7fce700a5680 2021-01-06T22:12:00.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.605+0000 7fce6dd9b700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fce700a5a60 0x7fce700608d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49874/0 (socket says 172.21.15.203:49874) 2021-01-06T22:12:00.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.605+0000 7fce6dd9b700 1 -- 172.21.15.203:0/2894190089 learned_addr learned my addr 172.21.15.203:0/2894190089 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:12:00.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.605+0000 7fce6e59c700 1 --2- 172.21.15.203:0/2894190089 >> v2:172.21.15.91:3300/0 conn(0x7fce700a5680 0x7fce7014fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:00.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.605+0000 7fce6d59a700 1 -- 172.21.15.203:0/2894190089 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460072878 0 0) 0x7fce70064ab0 con 0x7fce700a5a60 2021-01-06T22:12:00.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.605+0000 7fce6d59a700 1 -- 172.21.15.203:0/2894190089 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce54003230 con 0x7fce700a5a60 2021-01-06T22:12:00.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6d59a700 1 -- 172.21.15.203:0/2894190089 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4055671898 0 0) 0x7fce54003230 con 0x7fce700a5a60 2021-01-06T22:12:00.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6d59a700 1 -- 172.21.15.203:0/2894190089 >> v2:172.21.15.91:3300/0 conn(0x7fce700a5680 msgr2=0x7fce7014fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:12:00.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6d59a700 1 --2- 172.21.15.203:0/2894190089 >> v2:172.21.15.91:3300/0 conn(0x7fce700a5680 0x7fce7014fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:00.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6d59a700 1 -- 172.21.15.203:0/2894190089 >> v1:172.21.15.91:6789/0 conn(0x7fce70060fe0 legacy=0x7fce700bc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:00.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6d59a700 1 -- 172.21.15.203:0/2894190089 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce700678c0 con 0x7fce700a5a60 2021-01-06T22:12:00.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6d59a700 1 -- 172.21.15.203:0/2894190089 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7fce64001500 con 0x7fce700a5a60 2021-01-06T22:12:00.618 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6d59a700 1 -- 172.21.15.203:0/2894190089 >> v1:172.21.15.203:6789/0 conn(0x7fce700a5a60 legacy=0x7fce700608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:00.618 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6d59a700 1 --2- 172.21.15.203:0/2894190089 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce54003720 0x7fce54003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:00.618 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6d59a700 1 --2- 172.21.15.203:0/2894190089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce540044d0 0x7fce54004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:00.618 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6d59a700 1 --2- 172.21.15.203:0/2894190089 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce54004e40 0x7fce5400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:00.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6d59a700 1 -- 172.21.15.203:0/2894190089 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce701500f0 con 0x7fce54003720 2021-01-06T22:12:00.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6d59a700 1 -- 172.21.15.203:0/2894190089 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce54004250 con 0x7fce540044d0 2021-01-06T22:12:00.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6d59a700 1 -- 172.21.15.203:0/2894190089 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5400bc10 con 0x7fce54004e40 2021-01-06T22:12:00.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6dd9b700 1 --2- 172.21.15.203:0/2894190089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce540044d0 0x7fce54004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:00.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6e59c700 1 --2- 172.21.15.203:0/2894190089 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce54003720 0x7fce54003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:00.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6ed9d700 1 --2- 172.21.15.203:0/2894190089 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce54004e40 0x7fce5400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:00.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6dd9b700 1 -- 172.21.15.203:0/2894190089 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce54004e40 msgr2=0x7fce5400b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:12:00.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6dd9b700 1 --2- 172.21.15.203:0/2894190089 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce54004e40 0x7fce5400b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:00.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6dd9b700 1 -- 172.21.15.203:0/2894190089 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce54003720 msgr2=0x7fce54003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:12:00.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6dd9b700 1 --2- 172.21.15.203:0/2894190089 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce54003720 0x7fce54003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:00.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6dd9b700 1 -- 172.21.15.203:0/2894190089 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce54003230 con 0x7fce540044d0 2021-01-06T22:12:00.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6ed9d700 1 --2- 172.21.15.203:0/2894190089 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce54004e40 0x7fce5400b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:12:00.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6ed9d700 1 -- 172.21.15.203:0/2894190089 reap_dead start 2021-01-06T22:12:00.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6dd9b700 1 --2- 172.21.15.203:0/2894190089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce540044d0 0x7fce54004950 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fce700677f0 tx=0x7fce70067470).ready entity=mon.1 client_cookie=32205b248bfebcc4 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:00.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6d59a700 1 -- 172.21.15.203:0/2894190089 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6400ce50 con 0x7fce540044d0 2021-01-06T22:12:00.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6d59a700 1 -- 172.21.15.203:0/2894190089 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce640072e0 con 0x7fce540044d0 2021-01-06T22:12:00.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6d59a700 1 -- 172.21.15.203:0/2894190089 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce640057f0 con 0x7fce540044d0 2021-01-06T22:12:00.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6f59e700 1 -- 172.21.15.203:0/2894190089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce540044d0 msgr2=0x7fce54004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:00.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6f59e700 1 --2- 172.21.15.203:0/2894190089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce540044d0 0x7fce54004950 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fce700677f0 tx=0x7fce70067470).stop 2021-01-06T22:12:00.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6f59e700 1 -- 172.21.15.203:0/2894190089 shutdown_connections 2021-01-06T22:12:00.626 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6f59e700 1 --2- 172.21.15.203:0/2894190089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce540044d0 0x7fce54004950 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:00.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6f59e700 1 -- 172.21.15.203:0/2894190089 >> 172.21.15.203:0/2894190089 conn(0x7fce70061530 msgr2=0x7fce70063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:12:00.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6f59e700 1 -- 172.21.15.203:0/2894190089 shutdown_connections 2021-01-06T22:12:00.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.609+0000 7fce6f59e700 1 -- 172.21.15.203:0/2894190089 wait complete. 2021-01-06T22:12:00.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.613+0000 7fce6f59e700 1 Processor -- start 2021-01-06T22:12:00.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.613+0000 7fce6f59e700 1 -- start start 2021-01-06T22:12:00.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6f59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce540044d0 0x7fce700b9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:00.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6f59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce700bb150 0x7fce70148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:00.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6f59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce700bab80 0x7fce70149380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:00.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6f59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce700b9610 con 0x7fce540044d0 2021-01-06T22:12:00.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6f59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce700a1120 con 0x7fce700bab80 2021-01-06T22:12:00.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6f59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7015c2b0 con 0x7fce700bb150 2021-01-06T22:12:00.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6ed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce700bab80 0x7fce70149380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:00.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6ed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce700bab80 0x7fce70149380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51296/0 (socket says 172.21.15.203:51296) 2021-01-06T22:12:00.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6ed9d700 1 -- 172.21.15.203:0/341202863 learned_addr learned my addr 172.21.15.203:0/341202863 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:12:00.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6e59c700 1 --2- 172.21.15.203:0/341202863 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce540044d0 0x7fce700b9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:00.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6dd9b700 1 --2- 172.21.15.203:0/341202863 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce700bb150 0x7fce70148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:00.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6ed9d700 1 -- 172.21.15.203:0/341202863 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce700bb150 msgr2=0x7fce70148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:00.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6ed9d700 1 --2- 172.21.15.203:0/341202863 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce700bb150 0x7fce70148e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:00.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6ed9d700 1 -- 172.21.15.203:0/341202863 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce540044d0 msgr2=0x7fce700b9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:00.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6ed9d700 1 --2- 172.21.15.203:0/341202863 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce540044d0 0x7fce700b9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:00.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6ed9d700 1 -- 172.21.15.203:0/341202863 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce700a2570 con 0x7fce700bab80 2021-01-06T22:12:00.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6e59c700 1 --2- 172.21.15.203:0/341202863 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce540044d0 0x7fce700b9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:12:00.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6ed9d700 1 --2- 172.21.15.203:0/341202863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce700bab80 0x7fce70149380 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fce70063850 tx=0x7fce68009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:00.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce5e7fc700 1 -- 172.21.15.203:0/341202863 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6800f5d0 con 0x7fce700bab80 2021-01-06T22:12:00.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce5e7fc700 1 -- 172.21.15.203:0/341202863 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6800f730 con 0x7fce700bab80 2021-01-06T22:12:00.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6f59e700 1 -- 172.21.15.203:0/341202863 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce700a2810 con 0x7fce700bab80 2021-01-06T22:12:00.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce5e7fc700 1 -- 172.21.15.203:0/341202863 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6800fab0 con 0x7fce700bab80 2021-01-06T22:12:00.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6f59e700 1 -- 172.21.15.203:0/341202863 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce700a2ab0 con 0x7fce700bab80 2021-01-06T22:12:00.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce6f59e700 1 -- 172.21.15.203:0/341202863 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce700450a0 con 0x7fce700bab80 2021-01-06T22:12:00.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.617+0000 7fce5e7fc700 1 -- 172.21.15.203:0/341202863 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fce6801a030 con 0x7fce700bab80 2021-01-06T22:12:00.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.621+0000 7fce5e7fc700 1 --2- 172.21.15.203:0/341202863 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fce4c056400 0x7fce4c0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:00.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.621+0000 7fce5e7fc700 1 -- 172.21.15.203:0/341202863 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fce6806f180 con 0x7fce700bab80 2021-01-06T22:12:00.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.621+0000 7fce6e59c700 1 --2- 172.21.15.203:0/341202863 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fce4c056400 0x7fce4c0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:00.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.621+0000 7fce6e59c700 1 --2- 172.21.15.203:0/341202863 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fce4c056400 0x7fce4c0588c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fce6000eb00 tx=0x7fce60008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:00.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.625+0000 7fce5e7fc700 1 -- 172.21.15.203:0/341202863 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fce68044950 con 0x7fce700bab80 2021-01-06T22:12:00.835 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.829+0000 7fce6f59e700 1 -- 172.21.15.203:0/341202863 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce7005ca60 con 0x7fce700bab80 2021-01-06T22:12:00.836 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.829+0000 7fce5e7fc700 1 -- 172.21.15.203:0/341202863 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3563 (secure 0 0 0) 0x7fce68047f70 con 0x7fce700bab80 2021-01-06T22:12:00.836 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:12:00.837 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4925,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:10:15.511645+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4859,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4859":{"gid":4859,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6837/191881002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":191881002},{"type":"v1","addr":"172.21.15.203:6837","nonce":191881002}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:12:00.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.833+0000 7fce6f59e700 1 -- 172.21.15.203:0/341202863 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fce4c056400 msgr2=0x7fce4c0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:00.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.833+0000 7fce6f59e700 1 --2- 172.21.15.203:0/341202863 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fce4c056400 0x7fce4c0588c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fce6000eb00 tx=0x7fce60008040).stop 2021-01-06T22:12:00.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.833+0000 7fce6f59e700 1 -- 172.21.15.203:0/341202863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce700bab80 msgr2=0x7fce70149380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:00.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.833+0000 7fce6f59e700 1 --2- 172.21.15.203:0/341202863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce700bab80 0x7fce70149380 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fce70063850 tx=0x7fce68009040).stop 2021-01-06T22:12:00.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.833+0000 7fce6f59e700 1 -- 172.21.15.203:0/341202863 shutdown_connections 2021-01-06T22:12:00.848 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.833+0000 7fce6f59e700 1 --2- 172.21.15.203:0/341202863 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fce4c056400 0x7fce4c0588c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:00.848 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.833+0000 7fce6f59e700 1 --2- 172.21.15.203:0/341202863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce700bab80 0x7fce70149380 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:00.848 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.833+0000 7fce6f59e700 1 --2- 172.21.15.203:0/341202863 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce540044d0 0x7fce700b9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:00.848 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.833+0000 7fce6f59e700 1 --2- 172.21.15.203:0/341202863 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce700bb150 0x7fce70148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:00.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.833+0000 7fce6f59e700 1 -- 172.21.15.203:0/341202863 >> 172.21.15.203:0/341202863 conn(0x7fce70061530 msgr2=0x7fce7014d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:12:00.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.833+0000 7fce6f59e700 1 -- 172.21.15.203:0/341202863 shutdown_connections 2021-01-06T22:12:00.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:00.833+0000 7fce6f59e700 1 -- 172.21.15.203:0/341202863 wait complete. 2021-01-06T22:12:00.849 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 44 2021-01-06T22:12:00.851 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:12:02.852 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:12:02.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.981+0000 7fd84d802700 1 Processor -- start 2021-01-06T22:12:02.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.981+0000 7fd84d802700 1 -- start start 2021-01-06T22:12:02.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd84d802700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd8480a7f90 0x7fd848065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:02.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd84d802700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd848064700 con 0x7fd8480a7bb0 2021-01-06T22:12:02.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd84d802700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd848152d60 con 0x7fd8480bc220 2021-01-06T22:12:02.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd84d802700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8480bc630 con 0x7fd8480a7f90 2021-01-06T22:12:02.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd84d001700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fd8480bc220 0x7fd84815ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49900/0 (socket says 172.21.15.203:49900) 2021-01-06T22:12:02.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd84d001700 1 -- 172.21.15.203:0/845286159 learned_addr learned my addr 172.21.15.203:0/845286159 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:12:02.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd847fff700 1 --2- 172.21.15.203:0/845286159 >> v2:172.21.15.91:3300/0 conn(0x7fd8480a7f90 0x7fd848065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:02.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 -- 172.21.15.203:0/845286159 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611439917 0 0) 0x7fd848152d60 con 0x7fd8480bc220 2021-01-06T22:12:02.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 -- 172.21.15.203:0/845286159 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd834003230 con 0x7fd8480bc220 2021-01-06T22:12:02.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 -- 172.21.15.203:0/845286159 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4046453983 0 0) 0x7fd834003230 con 0x7fd8480bc220 2021-01-06T22:12:02.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 -- 172.21.15.203:0/845286159 >> v2:172.21.15.91:3300/0 conn(0x7fd8480a7f90 msgr2=0x7fd848065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:12:02.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 --2- 172.21.15.203:0/845286159 >> v2:172.21.15.91:3300/0 conn(0x7fd8480a7f90 0x7fd848065c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:02.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 -- 172.21.15.203:0/845286159 >> v1:172.21.15.91:6789/0 conn(0x7fd8480a7bb0 legacy=0x7fd8480626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:02.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 -- 172.21.15.203:0/845286159 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd848153000 con 0x7fd8480bc220 2021-01-06T22:12:02.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 -- 172.21.15.203:0/845286159 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7fd8400018a0 con 0x7fd8480bc220 2021-01-06T22:12:02.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 -- 172.21.15.203:0/845286159 >> v1:172.21.15.203:6789/0 conn(0x7fd8480bc220 legacy=0x7fd84815ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:02.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 --2- 172.21.15.203:0/845286159 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd834003720 0x7fd834003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:02.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 --2- 172.21.15.203:0/845286159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8340044d0 0x7fd834004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:02.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 --2- 172.21.15.203:0/845286159 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd834004e40 0x7fd83400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:03.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 -- 172.21.15.203:0/845286159 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8480bc630 con 0x7fd834004e40 2021-01-06T22:12:03.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 -- 172.21.15.203:0/845286159 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd834004250 con 0x7fd8340044d0 2021-01-06T22:12:03.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 -- 172.21.15.203:0/845286159 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83400bc10 con 0x7fd834003720 2021-01-06T22:12:03.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd847fff700 1 --2- 172.21.15.203:0/845286159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8340044d0 0x7fd834004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:03.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd84c800700 1 --2- 172.21.15.203:0/845286159 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd834003720 0x7fd834003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:03.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd84d001700 1 --2- 172.21.15.203:0/845286159 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd834004e40 0x7fd83400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:03.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd847fff700 1 -- 172.21.15.203:0/845286159 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd834003720 msgr2=0x7fd834003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:12:03.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd847fff700 1 --2- 172.21.15.203:0/845286159 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd834003720 0x7fd834003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd847fff700 1 -- 172.21.15.203:0/845286159 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd834004e40 msgr2=0x7fd83400b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:12:03.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd847fff700 1 --2- 172.21.15.203:0/845286159 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd834004e40 0x7fd83400b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd84d001700 1 -- 172.21.15.203:0/845286159 reap_dead start 2021-01-06T22:12:03.007 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd847fff700 1 -- 172.21.15.203:0/845286159 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd834003230 con 0x7fd8340044d0 2021-01-06T22:12:03.007 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd84c800700 1 --2- 172.21.15.203:0/845286159 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd834003720 0x7fd834003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:12:03.007 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd847fff700 1 --2- 172.21.15.203:0/845286159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8340044d0 0x7fd834004950 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fd83c00bb90 tx=0x7fd83c008040).ready entity=mon.1 client_cookie=b23c905c8363d441 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:03.007 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 -- 172.21.15.203:0/845286159 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c00b2b0 con 0x7fd8340044d0 2021-01-06T22:12:03.008 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 -- 172.21.15.203:0/845286159 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c00e960 con 0x7fd8340044d0 2021-01-06T22:12:03.008 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.985+0000 7fd8477fe700 1 -- 172.21.15.203:0/845286159 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c00ed00 con 0x7fd8340044d0 2021-01-06T22:12:03.008 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.989+0000 7fd84d802700 1 -- 172.21.15.203:0/845286159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8340044d0 msgr2=0x7fd834004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:03.008 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.989+0000 7fd84d802700 1 --2- 172.21.15.203:0/845286159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8340044d0 0x7fd834004950 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fd83c00bb90 tx=0x7fd83c008040).stop 2021-01-06T22:12:03.008 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.989+0000 7fd84d802700 1 -- 172.21.15.203:0/845286159 shutdown_connections 2021-01-06T22:12:03.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.989+0000 7fd84d802700 1 --2- 172.21.15.203:0/845286159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8340044d0 0x7fd834004950 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.989+0000 7fd84d802700 1 -- 172.21.15.203:0/845286159 >> 172.21.15.203:0/845286159 conn(0x7fd8480648f0 msgr2=0x7fd848064cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:12:03.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.989+0000 7fd84d802700 1 -- 172.21.15.203:0/845286159 shutdown_connections 2021-01-06T22:12:03.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.989+0000 7fd84d802700 1 -- 172.21.15.203:0/845286159 wait complete. 2021-01-06T22:12:03.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.989+0000 7fd84d802700 1 Processor -- start 2021-01-06T22:12:03.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.989+0000 7fd84d802700 1 -- start start 2021-01-06T22:12:03.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84d802700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8340044d0 0x7fd84814db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:03.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84d802700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd8480c0a30 0x7fd848162000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:03.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84d802700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd8480c0eb0 0x7fd8481624f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:03.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84d802700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84814e130 con 0x7fd8480c0eb0 2021-01-06T22:12:03.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84d802700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd848156c00 con 0x7fd8340044d0 2021-01-06T22:12:03.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84d802700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd848156ea0 con 0x7fd8480c0a30 2021-01-06T22:12:03.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84c800700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8340044d0 0x7fd84814db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:03.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84c800700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8340044d0 0x7fd84814db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51314/0 (socket says 172.21.15.203:51314) 2021-01-06T22:12:03.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84c800700 1 -- 172.21.15.203:0/1589504004 learned_addr learned my addr 172.21.15.203:0/1589504004 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:12:03.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd847fff700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd8480c0a30 0x7fd848162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:03.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84c800700 1 -- 172.21.15.203:0/1589504004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8340044d0 msgr2=0x7fd84814db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2021-01-06T22:12:03.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84c800700 1 -- 172.21.15.203:0/1589504004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8340044d0 msgr2=0x7fd84814db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2021-01-06T22:12:03.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84d001700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd8480c0eb0 0x7fd8481624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:03.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84c800700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8340044d0 0x7fd84814db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:12:03.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84c800700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8340044d0 0x7fd84814db10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-01-06T22:12:03.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84d001700 1 -- 172.21.15.203:0/1589504004 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd8480c0a30 msgr2=0x7fd848162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:03.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84d001700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd8480c0a30 0x7fd848162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84d001700 1 -- 172.21.15.203:0/1589504004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8340044d0 msgr2=0x7fd84814db10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2021-01-06T22:12:03.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84d001700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8340044d0 0x7fd84814db10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84d001700 1 -- 172.21.15.203:0/1589504004 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd848157140 con 0x7fd8480c0eb0 2021-01-06T22:12:03.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd847fff700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd8480c0a30 0x7fd848162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:12:03.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84d001700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd8480c0eb0 0x7fd8481624f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fd8480bc600 tx=0x7fd8481521d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:03.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd844ff9700 1 -- 172.21.15.203:0/1589504004 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84000fbe0 con 0x7fd8480c0eb0 2021-01-06T22:12:03.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd844ff9700 1 -- 172.21.15.203:0/1589504004 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84000fd40 con 0x7fd8480c0eb0 2021-01-06T22:12:03.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd844ff9700 1 -- 172.21.15.203:0/1589504004 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840012560 con 0x7fd8480c0eb0 2021-01-06T22:12:03.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84d802700 1 -- 172.21.15.203:0/1589504004 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd848161b60 con 0x7fd8480c0eb0 2021-01-06T22:12:03.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.993+0000 7fd84d802700 1 -- 172.21.15.203:0/1589504004 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd848062fc0 con 0x7fd8480c0eb0 2021-01-06T22:12:03.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.997+0000 7fd84d802700 1 -- 172.21.15.203:0/1589504004 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd848161cf0 con 0x7fd8480c0eb0 2021-01-06T22:12:03.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.997+0000 7fd844ff9700 1 -- 172.21.15.203:0/1589504004 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fd8400128d0 con 0x7fd8480c0eb0 2021-01-06T22:12:03.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.997+0000 7fd844ff9700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd8280563d0 0x7fd828058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:03.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.997+0000 7fd844ff9700 1 -- 172.21.15.203:0/1589504004 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6503+0+0 (secure 0 0 0) 0x7fd84006f510 con 0x7fd8480c0eb0 2021-01-06T22:12:03.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.997+0000 7fd84c800700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd8280563d0 0x7fd828058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:03.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:02.997+0000 7fd84c800700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd8280563d0 0x7fd828058890 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fd838000bc0 tx=0x7fd838009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:03.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.009+0000 7fd844ff9700 1 -- 172.21.15.203:0/1589504004 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fd840044c60 con 0x7fd8480c0eb0 2021-01-06T22:12:03.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.209+0000 7fd84d802700 1 -- 172.21.15.203:0/1589504004 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8480450a0 con 0x7fd8480c0eb0 2021-01-06T22:12:03.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.209+0000 7fd844ff9700 1 -- 172.21.15.203:0/1589504004 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+3211 (secure 0 0 0) 0x7fd840019070 con 0x7fd8480c0eb0 2021-01-06T22:12:03.215 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:12:03.215 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:12:02.497757+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4925,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4925":{"gid":4925,"name":"f","rank":0,"incarnation":45,"state":"up:resolve","state_seq":27,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:12:03.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.213+0000 7fd84d802700 1 -- 172.21.15.203:0/1589504004 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd8280563d0 msgr2=0x7fd828058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:03.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.213+0000 7fd84d802700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd8280563d0 0x7fd828058890 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fd838000bc0 tx=0x7fd838009040).stop 2021-01-06T22:12:03.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.213+0000 7fd84d802700 1 -- 172.21.15.203:0/1589504004 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd8480c0eb0 msgr2=0x7fd8481624f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:03.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.213+0000 7fd84d802700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd8480c0eb0 0x7fd8481624f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fd8480bc600 tx=0x7fd8481521d0).stop 2021-01-06T22:12:03.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.213+0000 7fd84d802700 1 -- 172.21.15.203:0/1589504004 shutdown_connections 2021-01-06T22:12:03.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.213+0000 7fd84d802700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd8280563d0 0x7fd828058890 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.213+0000 7fd84d802700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd8480c0eb0 0x7fd8481624f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.213+0000 7fd84d802700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8340044d0 0x7fd84814db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.213+0000 7fd84d802700 1 --2- 172.21.15.203:0/1589504004 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd8480c0a30 0x7fd848162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.213+0000 7fd84d802700 1 -- 172.21.15.203:0/1589504004 >> 172.21.15.203:0/1589504004 conn(0x7fd8480648f0 msgr2=0x7fd8481510b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:12:03.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.213+0000 7fd84d802700 1 -- 172.21.15.203:0/1589504004 shutdown_connections 2021-01-06T22:12:03.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.213+0000 7fd84d802700 1 -- 172.21.15.203:0/1589504004 wait complete. 2021-01-06T22:12:03.221 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 46 2021-01-06T22:12:03.230 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2021-01-06T22:12:03.231 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2021-01-06T22:12:03.231 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:12:03.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4facde9700 1 Processor -- start 2021-01-06T22:12:03.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4facde9700 1 -- start start 2021-01-06T22:12:03.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4facde9700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fa80ba9d0 0x7f4fa815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:03.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4facde9700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa80646b0 con 0x7f4fa80a5e50 2021-01-06T22:12:03.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4facde9700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa8152cc0 con 0x7f4fa80bb3f0 2021-01-06T22:12:03.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4facde9700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa80670c0 con 0x7f4fa80ba9d0 2021-01-06T22:12:03.373 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa77fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f4fa80bb3f0 0x7f4fa80669b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49916/0 (socket says 172.21.15.203:49916) 2021-01-06T22:12:03.373 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa77fe700 1 -- 172.21.15.203:0/3323356714 learned_addr learned my addr 172.21.15.203:0/3323356714 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:12:03.373 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 -- 172.21.15.203:0/3323356714 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755905719 0 0) 0x7f4fa8152cc0 con 0x7f4fa80bb3f0 2021-01-06T22:12:03.374 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 -- 172.21.15.203:0/3323356714 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f94003230 con 0x7f4fa80bb3f0 2021-01-06T22:12:03.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa7fff700 1 --2- 172.21.15.203:0/3323356714 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fa80ba9d0 0x7f4fa815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:03.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 -- 172.21.15.203:0/3323356714 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1435526911 0 0) 0x7f4f94003230 con 0x7f4fa80bb3f0 2021-01-06T22:12:03.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 -- 172.21.15.203:0/3323356714 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fa80ba9d0 msgr2=0x7f4fa815cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:12:03.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 --2- 172.21.15.203:0/3323356714 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fa80ba9d0 0x7f4fa815cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 -- 172.21.15.203:0/3323356714 >> v1:172.21.15.91:6789/0 conn(0x7f4fa80a5e50 legacy=0x7f4fa80bb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:03.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 -- 172.21.15.203:0/3323356714 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa8064450 con 0x7f4fa80bb3f0 2021-01-06T22:12:03.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 -- 172.21.15.203:0/3323356714 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f4fa0001860 con 0x7f4fa80bb3f0 2021-01-06T22:12:03.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 -- 172.21.15.203:0/3323356714 >> v1:172.21.15.203:6789/0 conn(0x7f4fa80bb3f0 legacy=0x7f4fa80669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:03.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 --2- 172.21.15.203:0/3323356714 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fa80ba9d0 0x7f4f94003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:03.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 --2- 172.21.15.203:0/3323356714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f94003c30 0x7f4f940044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:03.379 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 --2- 172.21.15.203:0/3323356714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f940049c0 0x7f4f94004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:03.379 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 -- 172.21.15.203:0/3323356714 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa80670c0 con 0x7f4f940049c0 2021-01-06T22:12:03.379 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 -- 172.21.15.203:0/3323356714 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f940041c0 con 0x7f4f94003c30 2021-01-06T22:12:03.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 -- 172.21.15.203:0/3323356714 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9400b720 con 0x7f4fa80ba9d0 2021-01-06T22:12:03.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa6ffd700 1 --2- 172.21.15.203:0/3323356714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f94003c30 0x7f4f940044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:03.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa77fe700 1 --2- 172.21.15.203:0/3323356714 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fa80ba9d0 0x7f4f94003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:03.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa7fff700 1 --2- 172.21.15.203:0/3323356714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f940049c0 0x7f4f94004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:03.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa6ffd700 1 -- 172.21.15.203:0/3323356714 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fa80ba9d0 msgr2=0x7f4f94003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:12:03.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa6ffd700 1 --2- 172.21.15.203:0/3323356714 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fa80ba9d0 0x7f4f94003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa6ffd700 1 -- 172.21.15.203:0/3323356714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f940049c0 msgr2=0x7f4f94004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:12:03.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa6ffd700 1 --2- 172.21.15.203:0/3323356714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f940049c0 0x7f4f94004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa6ffd700 1 -- 172.21.15.203:0/3323356714 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f94003230 con 0x7f4f94003c30 2021-01-06T22:12:03.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa7fff700 1 --2- 172.21.15.203:0/3323356714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f940049c0 0x7f4f94004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:12:03.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa6ffd700 1 --2- 172.21.15.203:0/3323356714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f94003c30 0x7f4f940044d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f4f9c0018f0 tx=0x7f4f9c007b80).ready entity=mon.1 client_cookie=b7befc7858696e00 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:03.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 -- 172.21.15.203:0/3323356714 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c007020 con 0x7f4f94003c30 2021-01-06T22:12:03.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 -- 172.21.15.203:0/3323356714 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c007180 con 0x7f4f94003c30 2021-01-06T22:12:03.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.353+0000 7f4fa67fc700 1 -- 172.21.15.203:0/3323356714 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00c7d0 con 0x7f4f94003c30 2021-01-06T22:12:03.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.357+0000 7f4facde9700 1 -- 172.21.15.203:0/3323356714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f94003c30 msgr2=0x7f4f940044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:03.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.357+0000 7f4facde9700 1 --2- 172.21.15.203:0/3323356714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f94003c30 0x7f4f940044d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f4f9c0018f0 tx=0x7f4f9c007b80).stop 2021-01-06T22:12:03.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.357+0000 7f4fa7fff700 1 -- 172.21.15.203:0/3323356714 reap_dead start 2021-01-06T22:12:03.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.357+0000 7f4facde9700 1 -- 172.21.15.203:0/3323356714 shutdown_connections 2021-01-06T22:12:03.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.357+0000 7f4facde9700 1 -- 172.21.15.203:0/3323356714 >> 172.21.15.203:0/3323356714 conn(0x7f4fa8061530 msgr2=0x7f4fa8063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:12:03.389 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.357+0000 7f4facde9700 1 -- 172.21.15.203:0/3323356714 shutdown_connections 2021-01-06T22:12:03.389 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.357+0000 7f4facde9700 1 -- 172.21.15.203:0/3323356714 wait complete. 2021-01-06T22:12:03.389 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4facde9700 1 Processor -- start 2021-01-06T22:12:03.390 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4facde9700 1 -- start start 2021-01-06T22:12:03.390 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4facde9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f94003c30 0x7f4fa8149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:03.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4facde9700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fa8155c80 0x7f4fa814b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:03.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4facde9700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fa8154270 0x7f4fa81546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:03.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4facde9700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa8158650 con 0x7f4fa8155c80 2021-01-06T22:12:03.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4facde9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa8162eb0 con 0x7f4f94003c30 2021-01-06T22:12:03.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4facde9700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa81629b0 con 0x7f4fa8154270 2021-01-06T22:12:03.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4fa77fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f94003c30 0x7f4fa8149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:03.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4fa77fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f94003c30 0x7f4fa8149230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51334/0 (socket says 172.21.15.203:51334) 2021-01-06T22:12:03.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4fa77fe700 1 -- 172.21.15.203:0/3352891428 learned_addr learned my addr 172.21.15.203:0/3352891428 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:12:03.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4fa6ffd700 1 --2- 172.21.15.203:0/3352891428 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fa8155c80 0x7f4fa814b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:03.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4fa7fff700 1 --2- 172.21.15.203:0/3352891428 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fa8154270 0x7f4fa81546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:03.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4fa77fe700 1 -- 172.21.15.203:0/3352891428 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fa8154270 msgr2=0x7f4fa81546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:03.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4fa77fe700 1 --2- 172.21.15.203:0/3352891428 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fa8154270 0x7f4fa81546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4fa77fe700 1 -- 172.21.15.203:0/3352891428 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fa8155c80 msgr2=0x7f4fa814b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:03.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4fa77fe700 1 --2- 172.21.15.203:0/3352891428 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fa8155c80 0x7f4fa814b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4fa77fe700 1 -- 172.21.15.203:0/3352891428 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa81568c0 con 0x7f4f94003c30 2021-01-06T22:12:03.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4fa77fe700 1 --2- 172.21.15.203:0/3352891428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f94003c30 0x7f4fa8149230 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f4fa0004ea0 tx=0x7f4fa8066840).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:03.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4f8f7fe700 1 -- 172.21.15.203:0/3352891428 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa000fc40 con 0x7f4f94003c30 2021-01-06T22:12:03.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4f8f7fe700 1 -- 172.21.15.203:0/3352891428 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa000fda0 con 0x7f4f94003c30 2021-01-06T22:12:03.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4facde9700 1 -- 172.21.15.203:0/3352891428 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa8156b60 con 0x7f4f94003c30 2021-01-06T22:12:03.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4f8f7fe700 1 -- 172.21.15.203:0/3352891428 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0015620 con 0x7f4f94003c30 2021-01-06T22:12:03.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.361+0000 7f4facde9700 1 -- 172.21.15.203:0/3352891428 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa81581b0 con 0x7f4f94003c30 2021-01-06T22:12:03.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.365+0000 7f4facde9700 1 -- 172.21.15.203:0/3352891428 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa8158370 con 0x7f4f94003c30 2021-01-06T22:12:03.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.365+0000 7f4f8f7fe700 1 -- 172.21.15.203:0/3352891428 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f4fa0001570 con 0x7f4f94003c30 2021-01-06T22:12:03.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.365+0000 7f4f8f7fe700 1 --2- 172.21.15.203:0/3352891428 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4f88056400 0x7f4f880588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:03.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.365+0000 7f4f8f7fe700 1 -- 172.21.15.203:0/3352891428 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6503+0+0 (secure 0 0 0) 0x7f4fa006fcc0 con 0x7f4f94003c30 2021-01-06T22:12:03.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.365+0000 7f4fa6ffd700 1 --2- 172.21.15.203:0/3352891428 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4f88056400 0x7f4f880588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:03.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.365+0000 7f4fa6ffd700 1 --2- 172.21.15.203:0/3352891428 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4f88056400 0x7f4f880588c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f4fa81560c0 tx=0x7f4f9c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:03.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.369+0000 7f4f8f7fe700 1 -- 172.21.15.203:0/3352891428 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f4fa0045390 con 0x7f4f94003c30 2021-01-06T22:12:03.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.577+0000 7f4facde9700 1 -- 172.21.15.203:0/3352891428 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fa80450a0 con 0x7f4f94003c30 2021-01-06T22:12:03.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.581+0000 7f4f8f7fe700 1 -- 172.21.15.203:0/3352891428 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+3213 (secure 0 0 0) 0x7f4fa00489b0 con 0x7f4f94003c30 2021-01-06T22:12:03.586 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:12:03.586 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:12:03.510412+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4925,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4925":{"gid":4925,"name":"f","rank":0,"incarnation":45,"state":"up:reconnect","state_seq":28,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:12:03.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.581+0000 7f4facde9700 1 -- 172.21.15.203:0/3352891428 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4f88056400 msgr2=0x7f4f880588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:03.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.581+0000 7f4facde9700 1 --2- 172.21.15.203:0/3352891428 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4f88056400 0x7f4f880588c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f4fa81560c0 tx=0x7f4f9c015000).stop 2021-01-06T22:12:03.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.581+0000 7f4facde9700 1 -- 172.21.15.203:0/3352891428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f94003c30 msgr2=0x7f4fa8149230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:03.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.581+0000 7f4facde9700 1 --2- 172.21.15.203:0/3352891428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f94003c30 0x7f4fa8149230 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f4fa0004ea0 tx=0x7f4fa8066840).stop 2021-01-06T22:12:03.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.581+0000 7f4facde9700 1 -- 172.21.15.203:0/3352891428 shutdown_connections 2021-01-06T22:12:03.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.581+0000 7f4facde9700 1 --2- 172.21.15.203:0/3352891428 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4f88056400 0x7f4f880588c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.581+0000 7f4facde9700 1 --2- 172.21.15.203:0/3352891428 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4fa8154270 0x7f4fa81546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.581+0000 7f4facde9700 1 --2- 172.21.15.203:0/3352891428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f94003c30 0x7f4fa8149230 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.581+0000 7f4facde9700 1 --2- 172.21.15.203:0/3352891428 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fa8155c80 0x7f4fa814b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:03.590 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.581+0000 7f4facde9700 1 -- 172.21.15.203:0/3352891428 >> 172.21.15.203:0/3352891428 conn(0x7f4fa8061530 msgr2=0x7f4fa8153960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:12:03.590 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.581+0000 7f4facde9700 1 -- 172.21.15.203:0/3352891428 shutdown_connections 2021-01-06T22:12:03.590 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:03.581+0000 7f4facde9700 1 -- 172.21.15.203:0/3352891428 wait complete. 2021-01-06T22:12:03.591 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 47 2021-01-06T22:12:03.602 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 47, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 47, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_4925": { "addr": "172.21.15.91:6837/1310069455", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6835", "nonce": 1310069455, "type": "v2" }, { "addr": "172.21.15.91:6837", "nonce": 1310069455, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4925, "incarnation": 45, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:reconnect", "state_seq": 28 }, "gid_5068": { "addr": "172.21.15.203:6835/1477297469", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6834", "nonce": 1477297469, "type": "v2" }, { "addr": "172.21.15.203:6835", "nonce": 1477297469, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5068, "incarnation": 33, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 28, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2021-01-06T22:12:03.510412+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 4925, "mds_1": 5068 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.91:6836/2428882934", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6834", "nonce": 2428882934, "type": "v2" }, { "addr": "172.21.15.91:6836", "nonce": 2428882934, "type": "v1" } ] }, "epoch": 26, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4724, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2021-01-06T22:12:05.604 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:12:05.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.733+0000 7fa7fb824700 1 Processor -- start 2021-01-06T22:12:05.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.733+0000 7fa7fb824700 1 -- start start 2021-01-06T22:12:05.742 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7fb824700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa7f40bbe00 0x7fa7f414e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:05.743 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7fb824700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa7f40c1290 0x7fa7f414a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:05.743 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7fb824700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f40b8590 con 0x7fa7f40bc1e0 2021-01-06T22:12:05.743 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7fb824700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f40bba20 con 0x7fa7f40c1290 2021-01-06T22:12:05.743 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7fb824700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f4057110 con 0x7fa7f40bbe00 2021-01-06T22:12:05.744 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7f95c0700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa7f40bbe00 0x7fa7f414e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:05.744 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7f8dbf700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fa7f40bc1e0 0x7fa7f40bb310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49948/0 (socket says 172.21.15.203:49948) 2021-01-06T22:12:05.744 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7f95c0700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa7f40bbe00 0x7fa7f414e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51352/0 (socket says 172.21.15.203:51352) 2021-01-06T22:12:05.744 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7f8dbf700 1 -- 172.21.15.203:0/3249096672 learned_addr learned my addr 172.21.15.203:0/3249096672 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:12:05.744 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7f9dc1700 1 --2- 172.21.15.203:0/3249096672 >> v2:172.21.15.91:3300/0 conn(0x7fa7f40c1290 0x7fa7f414a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:05.745 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7e3fff700 1 -- 172.21.15.203:0/3249096672 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540558027 0 0) 0x7fa7f40b8590 con 0x7fa7f40bc1e0 2021-01-06T22:12:05.745 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7f95c0700 1 -- 172.21.15.203:0/3249096672 >> v2:172.21.15.91:3300/0 conn(0x7fa7f40c1290 msgr2=0x7fa7f414a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:12:05.745 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7f95c0700 1 --2- 172.21.15.203:0/3249096672 >> v2:172.21.15.91:3300/0 conn(0x7fa7f40c1290 0x7fa7f414a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:05.745 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7f95c0700 1 -- 172.21.15.203:0/3249096672 >> v1:172.21.15.203:6789/0 conn(0x7fa7f40bc1e0 legacy=0x7fa7f40bb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:05.746 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7f95c0700 1 -- 172.21.15.203:0/3249096672 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f40b87c0 con 0x7fa7f40bbe00 2021-01-06T22:12:05.746 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7f95c0700 1 --2- 172.21.15.203:0/3249096672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f40bbe00 0x7fa7f414e5c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fa7f40c0880 tx=0x7fa7f0006de0).ready entity=mon.1 client_cookie=1d85b07a71f412d8 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:05.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7e3fff700 1 -- 172.21.15.203:0/3249096672 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0009750 con 0x7fa7f40bbe00 2021-01-06T22:12:05.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7e3fff700 1 -- 172.21.15.203:0/3249096672 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f00098b0 con 0x7fa7f40bbe00 2021-01-06T22:12:05.747 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7e3fff700 1 -- 172.21.15.203:0/3249096672 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0009c30 con 0x7fa7f40bbe00 2021-01-06T22:12:05.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7fb824700 1 -- 172.21.15.203:0/3249096672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f40bbe00 msgr2=0x7fa7f414e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:05.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7fb824700 1 --2- 172.21.15.203:0/3249096672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f40bbe00 0x7fa7f414e5c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fa7f40c0880 tx=0x7fa7f0006de0).stop 2021-01-06T22:12:05.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7fb824700 1 -- 172.21.15.203:0/3249096672 shutdown_connections 2021-01-06T22:12:05.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7fb824700 1 --2- 172.21.15.203:0/3249096672 >> v2:172.21.15.91:3300/0 conn(0x7fa7f40c1290 0x7fa7f414a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:05.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7fb824700 1 --2- 172.21.15.203:0/3249096672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f40bbe00 0x7fa7f414e5c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:05.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7fb824700 1 -- 172.21.15.203:0/3249096672 >> 172.21.15.203:0/3249096672 conn(0x7fa7f4061f90 msgr2=0x7fa7f4066830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:12:05.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7fb824700 1 -- 172.21.15.203:0/3249096672 shutdown_connections 2021-01-06T22:12:05.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.737+0000 7fa7fb824700 1 -- 172.21.15.203:0/3249096672 wait complete. 2021-01-06T22:12:05.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7fb824700 1 Processor -- start 2021-01-06T22:12:05.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7fb824700 1 -- start start 2021-01-06T22:12:05.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7fb824700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7f40bbe00 0x7fa7f41418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:05.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7fb824700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7f40bc1e0 0x7fa7f4145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:05.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7fb824700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f40c1290 0x7fa7f4143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:05.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7fb824700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f4141ec0 con 0x7fa7f40bbe00 2021-01-06T22:12:05.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7fb824700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f4142160 con 0x7fa7f40c1290 2021-01-06T22:12:05.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7fb824700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f4142400 con 0x7fa7f40bc1e0 2021-01-06T22:12:05.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7f9dc1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f40c1290 0x7fa7f4143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:05.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7f9dc1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f40c1290 0x7fa7f4143080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51362/0 (socket says 172.21.15.203:51362) 2021-01-06T22:12:05.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7f9dc1700 1 -- 172.21.15.203:0/286296031 learned_addr learned my addr 172.21.15.203:0/286296031 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:12:05.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7f95c0700 1 --2- 172.21.15.203:0/286296031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7f40bbe00 0x7fa7f41418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:05.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7f8dbf700 1 --2- 172.21.15.203:0/286296031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7f40bc1e0 0x7fa7f4145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:05.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7f9dc1700 1 -- 172.21.15.203:0/286296031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7f40bc1e0 msgr2=0x7fa7f4145f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:05.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7f9dc1700 1 --2- 172.21.15.203:0/286296031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7f40bc1e0 0x7fa7f4145f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:05.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7f9dc1700 1 -- 172.21.15.203:0/286296031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7f40bbe00 msgr2=0x7fa7f41418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:05.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7f9dc1700 1 --2- 172.21.15.203:0/286296031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7f40bbe00 0x7fa7f41418d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:05.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7f9dc1700 1 -- 172.21.15.203:0/286296031 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f41426a0 con 0x7fa7f40c1290 2021-01-06T22:12:05.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7f9dc1700 1 --2- 172.21.15.203:0/286296031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f40c1290 0x7fa7f4143080 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7fa7ec00ed20 tx=0x7fa7ec0091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:05.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7e17fa700 1 -- 172.21.15.203:0/286296031 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec009580 con 0x7fa7f40c1290 2021-01-06T22:12:05.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7e17fa700 1 -- 172.21.15.203:0/286296031 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ec00cb90 con 0x7fa7f40c1290 2021-01-06T22:12:05.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7e17fa700 1 -- 172.21.15.203:0/286296031 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00eee0 con 0x7fa7f40c1290 2021-01-06T22:12:05.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7fb824700 1 -- 172.21.15.203:0/286296031 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f4142940 con 0x7fa7f40c1290 2021-01-06T22:12:05.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.741+0000 7fa7fb824700 1 -- 172.21.15.203:0/286296031 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f414b7d0 con 0x7fa7f40c1290 2021-01-06T22:12:05.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.745+0000 7fa7fb824700 1 -- 172.21.15.203:0/286296031 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f405d600 con 0x7fa7f40c1290 2021-01-06T22:12:05.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.745+0000 7fa7e17fa700 1 -- 172.21.15.203:0/286296031 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa7ec022070 con 0x7fa7f40c1290 2021-01-06T22:12:05.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.745+0000 7fa7e17fa700 1 --2- 172.21.15.203:0/286296031 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa7d40563d0 0x7fa7d4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:05.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.745+0000 7fa7e17fa700 1 -- 172.21.15.203:0/286296031 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6503+0+0 (secure 0 0 0) 0x7fa7ec06fd00 con 0x7fa7f40c1290 2021-01-06T22:12:05.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.745+0000 7fa7f95c0700 1 --2- 172.21.15.203:0/286296031 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa7d40563d0 0x7fa7d4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:05.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.745+0000 7fa7f95c0700 1 --2- 172.21.15.203:0/286296031 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa7d40563d0 0x7fa7d4058890 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fa7f0009450 tx=0x7fa7f0016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:05.760 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.753+0000 7fa7e17fa700 1 -- 172.21.15.203:0/286296031 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa7ec073030 con 0x7fa7f40c1290 2021-01-06T22:12:05.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.981+0000 7fa7fb824700 1 -- 172.21.15.203:0/286296031 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7f40b9d40 con 0x7fa7f40c1290 2021-01-06T22:12:05.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.981+0000 7fa7e17fa700 1 -- 172.21.15.203:0/286296031 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+3210 (secure 0 0 0) 0x7fa7ec016070 con 0x7fa7f40c1290 2021-01-06T22:12:05.987 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:12:05.987 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:12:05.529398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4925,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4925":{"gid":4925,"name":"f","rank":0,"incarnation":45,"state":"up:active","state_seq":30,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:12:05.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.985+0000 7fa7fb824700 1 -- 172.21.15.203:0/286296031 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa7d40563d0 msgr2=0x7fa7d4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:05.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.985+0000 7fa7fb824700 1 --2- 172.21.15.203:0/286296031 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa7d40563d0 0x7fa7d4058890 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fa7f0009450 tx=0x7fa7f0016000).stop 2021-01-06T22:12:05.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.985+0000 7fa7fb824700 1 -- 172.21.15.203:0/286296031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f40c1290 msgr2=0x7fa7f4143080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:05.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.985+0000 7fa7fb824700 1 --2- 172.21.15.203:0/286296031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f40c1290 0x7fa7f4143080 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7fa7ec00ed20 tx=0x7fa7ec0091d0).stop 2021-01-06T22:12:05.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.985+0000 7fa7fb824700 1 -- 172.21.15.203:0/286296031 shutdown_connections 2021-01-06T22:12:05.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.985+0000 7fa7fb824700 1 --2- 172.21.15.203:0/286296031 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa7d40563d0 0x7fa7d4058890 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:05.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.985+0000 7fa7fb824700 1 --2- 172.21.15.203:0/286296031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f40c1290 0x7fa7f4143080 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:05.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.985+0000 7fa7fb824700 1 --2- 172.21.15.203:0/286296031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7f40bbe00 0x7fa7f41418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:05.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.985+0000 7fa7fb824700 1 --2- 172.21.15.203:0/286296031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7f40bc1e0 0x7fa7f4145f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:05.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.985+0000 7fa7fb824700 1 -- 172.21.15.203:0/286296031 >> 172.21.15.203:0/286296031 conn(0x7fa7f4061f90 msgr2=0x7fa7f414abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:12:05.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.985+0000 7fa7fb824700 1 -- 172.21.15.203:0/286296031 shutdown_connections 2021-01-06T22:12:05.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:05.985+0000 7fa7fb824700 1 -- 172.21.15.203:0/286296031 wait complete. 2021-01-06T22:12:05.991 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 49 2021-01-06T22:12:06.002 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=0, replacing gid=4859 2021-01-06T22:12:06.002 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.a 2021-01-06T22:12:06.478 INFO:tasks.ceph.mds.f.smithi091.stderr:2021-01-06T22:12:06.473+0000 7fe69376c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2021-01-06T22:12:06.479 INFO:tasks.ceph.mds.f.smithi091.stderr:2021-01-06T22:12:06.473+0000 7fe69376c700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2021-01-06T22:12:08.003 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2021-01-06T22:12:08.003 INFO:tasks.ceph.mds.a:Restarting daemon 2021-01-06T22:12:08.004 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-01-06T22:12:08.006 INFO:tasks.ceph.mds.a:Started 2021-01-06T22:12:08.007 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:12:08.077 INFO:tasks.ceph.mds.a.smithi203.stdout:starting mds.a at 2021-01-06T22:12:08.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.125+0000 7f1c413cd700 1 Processor -- start 2021-01-06T22:12:08.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.125+0000 7f1c413cd700 1 -- start start 2021-01-06T22:12:08.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c413cd700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1c3c0c0fa0 0x7f1c3c0bccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:08.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c413cd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1c3c14ad90 0x7f1c3c14b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:08.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c413cd700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c3c0bacb0 con 0x7f1c3c1513f0 2021-01-06T22:12:08.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c413cd700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c3c14b680 con 0x7f1c3c0c0fa0 2021-01-06T22:12:08.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c413cd700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c3c152000 con 0x7f1c3c14ad90 2021-01-06T22:12:08.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c40bcc700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1c3c14ad90 0x7f1c3c14b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:08.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c40bcc700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1c3c14ad90 0x7f1c3c14b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51382/0 (socket says 172.21.15.203:51382) 2021-01-06T22:12:08.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c40bcc700 1 -- 172.21.15.203:0/1954159668 learned_addr learned my addr 172.21.15.203:0/1954159668 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:12:08.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c3b7fe700 1 --2- 172.21.15.203:0/1954159668 >> v2:172.21.15.91:3300/0 conn(0x7f1c3c0c0fa0 0x7f1c3c0bccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:08.137 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c40bcc700 1 -- 172.21.15.203:0/1954159668 >> v2:172.21.15.91:3300/0 conn(0x7f1c3c0c0fa0 msgr2=0x7f1c3c0bccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:12:08.137 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c40bcc700 1 --2- 172.21.15.203:0/1954159668 >> v2:172.21.15.91:3300/0 conn(0x7f1c3c0c0fa0 0x7f1c3c0bccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:08.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c40bcc700 1 -- 172.21.15.203:0/1954159668 >> v1:172.21.15.91:6789/0 conn(0x7f1c3c1513f0 legacy=0x7f1c3c1517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:08.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c40bcc700 1 -- 172.21.15.203:0/1954159668 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3c067e50 con 0x7f1c3c14ad90 2021-01-06T22:12:08.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c40bcc700 1 --2- 172.21.15.203:0/1954159668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c3c14ad90 0x7f1c3c14b190 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f1c3c063d40 tx=0x7f1c34009290).ready entity=mon.1 client_cookie=9f71f6df68482a33 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:08.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c3affd700 1 -- 172.21.15.203:0/1954159668 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3400bbc0 con 0x7f1c3c14ad90 2021-01-06T22:12:08.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c3affd700 1 -- 172.21.15.203:0/1954159668 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3400bd20 con 0x7f1c3c14ad90 2021-01-06T22:12:08.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c3affd700 1 -- 172.21.15.203:0/1954159668 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c34011610 con 0x7f1c3c14ad90 2021-01-06T22:12:08.142 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c413cd700 1 -- 172.21.15.203:0/1954159668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c3c14ad90 msgr2=0x7f1c3c14b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:08.142 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c413cd700 1 --2- 172.21.15.203:0/1954159668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c3c14ad90 0x7f1c3c14b190 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f1c3c063d40 tx=0x7f1c34009290).stop 2021-01-06T22:12:08.142 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c413cd700 1 -- 172.21.15.203:0/1954159668 shutdown_connections 2021-01-06T22:12:08.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c413cd700 1 --2- 172.21.15.203:0/1954159668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c3c14ad90 0x7f1c3c14b190 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:08.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c413cd700 1 --2- 172.21.15.203:0/1954159668 >> v2:172.21.15.91:3300/0 conn(0x7f1c3c0c0fa0 0x7f1c3c0bccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:08.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c413cd700 1 -- 172.21.15.203:0/1954159668 >> 172.21.15.203:0/1954159668 conn(0x7f1c3c0667b0 msgr2=0x7f1c3c0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:12:08.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c413cd700 1 -- 172.21.15.203:0/1954159668 shutdown_connections 2021-01-06T22:12:08.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.129+0000 7f1c413cd700 1 -- 172.21.15.203:0/1954159668 wait complete. 2021-01-06T22:12:08.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c413cd700 1 Processor -- start 2021-01-06T22:12:08.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c413cd700 1 -- start start 2021-01-06T22:12:08.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c413cd700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c3c0c0fa0 0x7f1c3c1465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:08.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c413cd700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1c3c14ad90 0x7f1c3c15dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:08.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c413cd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c3c1513f0 0x7f1c3c148940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:08.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c413cd700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c15e1b0 con 0x7f1c3c0c0fa0 2021-01-06T22:12:08.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c413cd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c146c60 con 0x7f1c3c1513f0 2021-01-06T22:12:08.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c413cd700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c146f00 con 0x7f1c3c14ad90 2021-01-06T22:12:08.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c40bcc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c3c1513f0 0x7f1c3c148940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:08.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c40bcc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c3c1513f0 0x7f1c3c148940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51388/0 (socket says 172.21.15.203:51388) 2021-01-06T22:12:08.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c40bcc700 1 -- 172.21.15.203:0/7786649 learned_addr learned my addr 172.21.15.203:0/7786649 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:12:08.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c3b7fe700 1 --2- 172.21.15.203:0/7786649 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1c3c14ad90 0x7f1c3c15dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:08.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c3bfff700 1 --2- 172.21.15.203:0/7786649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c3c0c0fa0 0x7f1c3c1465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:08.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c40bcc700 1 -- 172.21.15.203:0/7786649 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1c3c14ad90 msgr2=0x7f1c3c15dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:08.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c40bcc700 1 --2- 172.21.15.203:0/7786649 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1c3c14ad90 0x7f1c3c15dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:08.148 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c40bcc700 1 -- 172.21.15.203:0/7786649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c3c0c0fa0 msgr2=0x7f1c3c1465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:08.148 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c40bcc700 1 --2- 172.21.15.203:0/7786649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c3c0c0fa0 0x7f1c3c1465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:08.148 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c40bcc700 1 -- 172.21.15.203:0/7786649 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3c1471a0 con 0x7f1c3c1513f0 2021-01-06T22:12:08.148 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c3bfff700 1 --2- 172.21.15.203:0/7786649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c3c0c0fa0 0x7f1c3c1465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:12:08.148 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c40bcc700 1 --2- 172.21.15.203:0/7786649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c3c1513f0 0x7f1c3c148940 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f1c34009e50 tx=0x7f1c34017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:08.149 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c23fff700 1 -- 172.21.15.203:0/7786649 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c34011e70 con 0x7f1c3c1513f0 2021-01-06T22:12:08.149 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c23fff700 1 -- 172.21.15.203:0/7786649 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3400be90 con 0x7f1c3c1513f0 2021-01-06T22:12:08.149 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c413cd700 1 -- 172.21.15.203:0/7786649 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3c0beec0 con 0x7f1c3c1513f0 2021-01-06T22:12:08.149 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c23fff700 1 -- 172.21.15.203:0/7786649 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c34010e30 con 0x7f1c3c1513f0 2021-01-06T22:12:08.150 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c413cd700 1 -- 172.21.15.203:0/7786649 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3c0bf400 con 0x7f1c3c1513f0 2021-01-06T22:12:08.150 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.133+0000 7f1c413cd700 1 -- 172.21.15.203:0/7786649 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3c0450a0 con 0x7f1c3c1513f0 2021-01-06T22:12:08.150 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.137+0000 7f1c23fff700 1 -- 172.21.15.203:0/7786649 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f1c3400f070 con 0x7f1c3c1513f0 2021-01-06T22:12:08.150 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.137+0000 7f1c23fff700 1 --2- 172.21.15.203:0/7786649 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1c1c056400 0x7f1c1c0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:08.151 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.137+0000 7f1c23fff700 1 -- 172.21.15.203:0/7786649 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6503+0+0 (secure 0 0 0) 0x7f1c34071650 con 0x7f1c3c1513f0 2021-01-06T22:12:08.151 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.137+0000 7f1c3bfff700 1 --2- 172.21.15.203:0/7786649 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1c1c056400 0x7f1c1c0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:08.151 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.137+0000 7f1c3bfff700 1 --2- 172.21.15.203:0/7786649 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1c1c056400 0x7f1c1c0588c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f1c2c001490 tx=0x7f1c2c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:08.152 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.141+0000 7f1c23fff700 1 -- 172.21.15.203:0/7786649 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f1c34046d00 con 0x7f1c3c1513f0 2021-01-06T22:12:08.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.365+0000 7f1c413cd700 1 -- 172.21.15.203:0/7786649 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c3c061f70 con 0x7f1c3c1513f0 2021-01-06T22:12:08.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.365+0000 7f1c23fff700 1 -- 172.21.15.203:0/7786649 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+3210 (secure 0 0 0) 0x7f1c3404a320 con 0x7f1c3c1513f0 2021-01-06T22:12:08.371 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:12:08.371 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:12:05.529398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4925,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4925":{"gid":4925,"name":"f","rank":0,"incarnation":45,"state":"up:active","state_seq":30,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:12:08.373 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.365+0000 7f1c413cd700 1 -- 172.21.15.203:0/7786649 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1c1c056400 msgr2=0x7f1c1c0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:08.373 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.365+0000 7f1c413cd700 1 --2- 172.21.15.203:0/7786649 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1c1c056400 0x7f1c1c0588c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f1c2c001490 tx=0x7f1c2c009040).stop 2021-01-06T22:12:08.374 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.365+0000 7f1c413cd700 1 -- 172.21.15.203:0/7786649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c3c1513f0 msgr2=0x7f1c3c148940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:08.374 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.365+0000 7f1c413cd700 1 --2- 172.21.15.203:0/7786649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c3c1513f0 0x7f1c3c148940 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f1c34009e50 tx=0x7f1c34017000).stop 2021-01-06T22:12:08.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.365+0000 7f1c413cd700 1 -- 172.21.15.203:0/7786649 shutdown_connections 2021-01-06T22:12:08.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.365+0000 7f1c413cd700 1 --2- 172.21.15.203:0/7786649 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1c1c056400 0x7f1c1c0588c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:08.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.365+0000 7f1c413cd700 1 --2- 172.21.15.203:0/7786649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c3c1513f0 0x7f1c3c148940 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:08.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.365+0000 7f1c413cd700 1 --2- 172.21.15.203:0/7786649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c3c0c0fa0 0x7f1c3c1465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:08.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.365+0000 7f1c413cd700 1 --2- 172.21.15.203:0/7786649 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1c3c14ad90 0x7f1c3c15dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:08.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.365+0000 7f1c413cd700 1 -- 172.21.15.203:0/7786649 >> 172.21.15.203:0/7786649 conn(0x7f1c3c0667b0 msgr2=0x7f1c3c145bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:12:08.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.365+0000 7f1c413cd700 1 -- 172.21.15.203:0/7786649 shutdown_connections 2021-01-06T22:12:08.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:08.365+0000 7f1c413cd700 1 -- 172.21.15.203:0/7786649 wait complete. 2021-01-06T22:12:08.378 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 49 2021-01-06T22:12:08.387 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2021-01-06T22:12:10.388 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:12:10.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f1509fac700 1 Processor -- start 2021-01-06T22:12:10.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f1509fac700 1 -- start start 2021-01-06T22:12:10.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f1509fac700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1504148320 0x7f15040626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:10.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f1509fac700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1504062bd0 0x7f15040bb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:10.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f1509fac700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150414f090 con 0x7f15040602b0 2021-01-06T22:12:10.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f1509fac700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15040badc0 con 0x7f1504062bd0 2021-01-06T22:12:10.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f1509fac700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1504068780 con 0x7f1504148320 2021-01-06T22:12:10.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f15037fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1504148320 0x7f15040626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:10.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f1503fff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f15040602b0 0x7f1504151740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:49990/0 (socket says 172.21.15.203:49990) 2021-01-06T22:12:10.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f15037fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1504148320 0x7f15040626e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51392/0 (socket says 172.21.15.203:51392) 2021-01-06T22:12:10.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f1503fff700 1 -- 172.21.15.203:0/3173213834 learned_addr learned my addr 172.21.15.203:0/3173213834 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:12:10.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f1502ffd700 1 --2- 172.21.15.203:0/3173213834 >> v2:172.21.15.91:3300/0 conn(0x7f1504062bd0 0x7f15040bb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:10.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f15037fe700 1 -- 172.21.15.203:0/3173213834 >> v2:172.21.15.91:3300/0 conn(0x7f1504062bd0 msgr2=0x7f15040bb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:12:10.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f15037fe700 1 --2- 172.21.15.203:0/3173213834 >> v2:172.21.15.91:3300/0 conn(0x7f1504062bd0 0x7f15040bb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:10.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f15037fe700 1 -- 172.21.15.203:0/3173213834 >> v1:172.21.15.203:6789/0 conn(0x7f15040602b0 legacy=0x7f1504151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:10.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f15027fc700 1 -- 172.21.15.203:0/3173213834 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68955122 0 0) 0x7f150414f090 con 0x7f15040602b0 2021-01-06T22:12:10.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f15037fe700 1 -- 172.21.15.203:0/3173213834 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150414e170 con 0x7f1504148320 2021-01-06T22:12:10.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f15037fe700 1 --2- 172.21.15.203:0/3173213834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1504148320 0x7f15040626e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f15040be760 tx=0x7f14f4006de0).ready entity=mon.1 client_cookie=6b3dfba737b2e985 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:10.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f15027fc700 1 -- 172.21.15.203:0/3173213834 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f40097b0 con 0x7f1504148320 2021-01-06T22:12:10.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f15027fc700 1 -- 172.21.15.203:0/3173213834 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f4009910 con 0x7f1504148320 2021-01-06T22:12:10.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.521+0000 7f15027fc700 1 -- 172.21.15.203:0/3173213834 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f4009c90 con 0x7f1504148320 2021-01-06T22:12:10.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1509fac700 1 -- 172.21.15.203:0/3173213834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1504148320 msgr2=0x7f15040626e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:10.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1509fac700 1 --2- 172.21.15.203:0/3173213834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1504148320 0x7f15040626e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f15040be760 tx=0x7f14f4006de0).stop 2021-01-06T22:12:10.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1509fac700 1 -- 172.21.15.203:0/3173213834 shutdown_connections 2021-01-06T22:12:10.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1509fac700 1 --2- 172.21.15.203:0/3173213834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1504148320 0x7f15040626e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:10.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1509fac700 1 --2- 172.21.15.203:0/3173213834 >> v2:172.21.15.91:3300/0 conn(0x7f1504062bd0 0x7f15040bb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:10.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1509fac700 1 -- 172.21.15.203:0/3173213834 >> 172.21.15.203:0/3173213834 conn(0x7f15040c0cb0 msgr2=0x7f15040c1090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:12:10.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1509fac700 1 -- 172.21.15.203:0/3173213834 shutdown_connections 2021-01-06T22:12:10.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1509fac700 1 -- 172.21.15.203:0/3173213834 wait complete. 2021-01-06T22:12:10.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1509fac700 1 Processor -- start 2021-01-06T22:12:10.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1509fac700 1 -- start start 2021-01-06T22:12:10.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1509fac700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15040602b0 0x7f150415e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:10.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1509fac700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1504062bd0 0x7f15040b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:10.539 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1509fac700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1504148320 0x7f15040b8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:10.539 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1509fac700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150414b240 con 0x7f1504062bd0 2021-01-06T22:12:10.539 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1509fac700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150414b9b0 con 0x7f15040602b0 2021-01-06T22:12:10.539 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1509fac700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15040b38d0 con 0x7f1504148320 2021-01-06T22:12:10.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f15037fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15040602b0 0x7f150415e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:10.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f15037fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15040602b0 0x7f150415e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51398/0 (socket says 172.21.15.203:51398) 2021-01-06T22:12:10.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f15037fe700 1 -- 172.21.15.203:0/2456870396 learned_addr learned my addr 172.21.15.203:0/2456870396 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:12:10.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.525+0000 7f1502ffd700 1 --2- 172.21.15.203:0/2456870396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1504062bd0 0x7f15040b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:10.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f1503fff700 1 --2- 172.21.15.203:0/2456870396 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1504148320 0x7f15040b8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:10.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f15037fe700 1 -- 172.21.15.203:0/2456870396 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1504148320 msgr2=0x7f15040b8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:10.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f15037fe700 1 --2- 172.21.15.203:0/2456870396 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1504148320 0x7f15040b8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:10.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f15037fe700 1 -- 172.21.15.203:0/2456870396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1504062bd0 msgr2=0x7f15040b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:10.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f15037fe700 1 --2- 172.21.15.203:0/2456870396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1504062bd0 0x7f15040b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:10.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f15037fe700 1 -- 172.21.15.203:0/2456870396 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150414b470 con 0x7f15040602b0 2021-01-06T22:12:10.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f1503fff700 1 --2- 172.21.15.203:0/2456870396 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1504148320 0x7f15040b8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:12:10.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f15037fe700 1 --2- 172.21.15.203:0/2456870396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15040602b0 0x7f150415e880 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f14f4006e20 tx=0x7f14f4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:10.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f14eb7fe700 1 -- 172.21.15.203:0/2456870396 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f4012dc0 con 0x7f15040602b0 2021-01-06T22:12:10.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f14eb7fe700 1 -- 172.21.15.203:0/2456870396 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f401c420 con 0x7f15040602b0 2021-01-06T22:12:10.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f1509fac700 1 -- 172.21.15.203:0/2456870396 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150414b710 con 0x7f15040602b0 2021-01-06T22:12:10.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f14eb7fe700 1 -- 172.21.15.203:0/2456870396 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f401c7c0 con 0x7f15040602b0 2021-01-06T22:12:10.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f1509fac700 1 -- 172.21.15.203:0/2456870396 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15040b3430 con 0x7f15040602b0 2021-01-06T22:12:10.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f1509fac700 1 -- 172.21.15.203:0/2456870396 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150405d600 con 0x7f15040602b0 2021-01-06T22:12:10.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f14eb7fe700 1 -- 172.21.15.203:0/2456870396 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f14f400f070 con 0x7f15040602b0 2021-01-06T22:12:10.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f14eb7fe700 1 --2- 172.21.15.203:0/2456870396 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f14e40563d0 0x7f14e4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:10.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.529+0000 7f14eb7fe700 1 -- 172.21.15.203:0/2456870396 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6503+0+0 (secure 0 0 0) 0x7f14f4070db0 con 0x7f15040602b0 2021-01-06T22:12:10.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.533+0000 7f1502ffd700 1 --2- 172.21.15.203:0/2456870396 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f14e40563d0 0x7f14e4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:10.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.533+0000 7f1502ffd700 1 --2- 172.21.15.203:0/2456870396 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f14e40563d0 0x7f14e4058890 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f15040b7da0 tx=0x7f14f8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:10.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.537+0000 7f14eb7fe700 1 -- 172.21.15.203:0/2456870396 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f14f4046480 con 0x7f15040602b0 2021-01-06T22:12:10.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.765+0000 7f1509fac700 1 -- 172.21.15.203:0/2456870396 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15040b4160 con 0x7f15040602b0 2021-01-06T22:12:10.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.769+0000 7f14eb7fe700 1 -- 172.21.15.203:0/2456870396 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+3210 (secure 0 0 0) 0x7f14f4049aa0 con 0x7f15040602b0 2021-01-06T22:12:10.773 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:12:10.773 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:12:05.529398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4925,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4925":{"gid":4925,"name":"f","rank":0,"incarnation":45,"state":"up:active","state_seq":30,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:12:10.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.769+0000 7f1509fac700 1 -- 172.21.15.203:0/2456870396 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f14e40563d0 msgr2=0x7f14e4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:10.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.769+0000 7f1509fac700 1 --2- 172.21.15.203:0/2456870396 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f14e40563d0 0x7f14e4058890 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f15040b7da0 tx=0x7f14f8008040).stop 2021-01-06T22:12:10.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.769+0000 7f1509fac700 1 -- 172.21.15.203:0/2456870396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15040602b0 msgr2=0x7f150415e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:10.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.769+0000 7f1509fac700 1 --2- 172.21.15.203:0/2456870396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15040602b0 0x7f150415e880 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f14f4006e20 tx=0x7f14f4016000).stop 2021-01-06T22:12:10.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.769+0000 7f1509fac700 1 -- 172.21.15.203:0/2456870396 shutdown_connections 2021-01-06T22:12:10.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.769+0000 7f1509fac700 1 --2- 172.21.15.203:0/2456870396 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f14e40563d0 0x7f14e4058890 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:10.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.769+0000 7f1509fac700 1 --2- 172.21.15.203:0/2456870396 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1504148320 0x7f15040b8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:10.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.769+0000 7f1509fac700 1 --2- 172.21.15.203:0/2456870396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15040602b0 0x7f150415e880 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:10.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.769+0000 7f1509fac700 1 --2- 172.21.15.203:0/2456870396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1504062bd0 0x7f15040b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:10.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.769+0000 7f1509fac700 1 -- 172.21.15.203:0/2456870396 >> 172.21.15.203:0/2456870396 conn(0x7f15040c0cb0 msgr2=0x7f15040bba00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:12:10.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.769+0000 7f1509fac700 1 -- 172.21.15.203:0/2456870396 shutdown_connections 2021-01-06T22:12:10.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:10.769+0000 7f1509fac700 1 -- 172.21.15.203:0/2456870396 wait complete. 2021-01-06T22:12:10.778 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 49 2021-01-06T22:12:10.789 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2021-01-06T22:12:12.790 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:12:12.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.921+0000 7ff8c33ac700 1 Processor -- start 2021-01-06T22:12:12.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.921+0000 7ff8c33ac700 1 -- start start 2021-01-06T22:12:12.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.921+0000 7ff8c33ac700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff8bc0633f0 0x7ff8bc14e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:12.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.921+0000 7ff8c33ac700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8bc0be990 con 0x7ff8bc063010 2021-01-06T22:12:12.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.921+0000 7ff8c33ac700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8bc0be6f0 con 0x7ff8bc0be1b0 2021-01-06T22:12:12.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.921+0000 7ff8c33ac700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8bc14ec60 con 0x7ff8bc0633f0 2021-01-06T22:12:12.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.921+0000 7ff8c1ba9700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff8bc0633f0 0x7ff8bc14e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:12.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.921+0000 7ff8c2bab700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7ff8bc0be1b0 0x7ff8bc147790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50006/0 (socket says 172.21.15.203:50006) 2021-01-06T22:12:12.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.921+0000 7ff8c2bab700 1 -- 172.21.15.203:0/1822080551 learned_addr learned my addr 172.21.15.203:0/1822080551 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:12:12.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.921+0000 7ff8c1ba9700 1 -- 172.21.15.203:0/1822080551 >> v1:172.21.15.203:6789/0 conn(0x7ff8bc0be1b0 legacy=0x7ff8bc147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:12.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.921+0000 7ff8c1ba9700 1 -- 172.21.15.203:0/1822080551 >> v1:172.21.15.91:6789/0 conn(0x7ff8bc063010 legacy=0x7ff8bc064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:12.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.921+0000 7ff8c1ba9700 1 -- 172.21.15.203:0/1822080551 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc0c1050 con 0x7ff8bc0633f0 2021-01-06T22:12:12.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.921+0000 7ff8c1ba9700 1 --2- 172.21.15.203:0/1822080551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc0633f0 0x7ff8bc14e770 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7ff8ac00b980 tx=0x7ff8ac006de0).ready entity=mon.1 client_cookie=74b0a3d49ee6d8dd server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:12.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.925+0000 7ff8c13a8700 1 -- 172.21.15.203:0/1822080551 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac009840 con 0x7ff8bc0633f0 2021-01-06T22:12:12.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.925+0000 7ff8c13a8700 1 -- 172.21.15.203:0/1822080551 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac0099a0 con 0x7ff8bc0633f0 2021-01-06T22:12:12.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.925+0000 7ff8c13a8700 1 -- 172.21.15.203:0/1822080551 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac009d20 con 0x7ff8bc0633f0 2021-01-06T22:12:12.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.925+0000 7ff8c33ac700 1 -- 172.21.15.203:0/1822080551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc0633f0 msgr2=0x7ff8bc14e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:12.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.925+0000 7ff8c33ac700 1 --2- 172.21.15.203:0/1822080551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc0633f0 0x7ff8bc14e770 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7ff8ac00b980 tx=0x7ff8ac006de0).stop 2021-01-06T22:12:12.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.925+0000 7ff8c33ac700 1 -- 172.21.15.203:0/1822080551 shutdown_connections 2021-01-06T22:12:12.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.925+0000 7ff8c33ac700 1 --2- 172.21.15.203:0/1822080551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc0633f0 0x7ff8bc14e770 secure :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0x7ff8ac00b980 tx=0x7ff8ac006de0).stop 2021-01-06T22:12:12.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.925+0000 7ff8c33ac700 1 -- 172.21.15.203:0/1822080551 >> 172.21.15.203:0/1822080551 conn(0x7ff8bc067e20 msgr2=0x7ff8bc068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:12:12.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.925+0000 7ff8c33ac700 1 -- 172.21.15.203:0/1822080551 shutdown_connections 2021-01-06T22:12:12.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.925+0000 7ff8c33ac700 1 -- 172.21.15.203:0/1822080551 wait complete. 2021-01-06T22:12:12.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.925+0000 7ff8c33ac700 1 Processor -- start 2021-01-06T22:12:12.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.925+0000 7ff8c33ac700 1 -- start start 2021-01-06T22:12:12.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c33ac700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc063010 0x7ff8bc145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:12.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c33ac700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff8bc0be1b0 0x7ff8bc1461d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:12.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c33ac700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8bc143750 0x7ff8bc143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:12.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c33ac700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc0adc40 con 0x7ff8bc143750 2021-01-06T22:12:12.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c33ac700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc1466c0 con 0x7ff8bc063010 2021-01-06T22:12:12.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c33ac700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc147e30 con 0x7ff8bc0be1b0 2021-01-06T22:12:12.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c23aa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc063010 0x7ff8bc145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:12.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c23aa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc063010 0x7ff8bc145ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51414/0 (socket says 172.21.15.203:51414) 2021-01-06T22:12:12.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c23aa700 1 -- 172.21.15.203:0/531728669 learned_addr learned my addr 172.21.15.203:0/531728669 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:12:12.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c2bab700 1 --2- 172.21.15.203:0/531728669 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8bc143750 0x7ff8bc143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:12.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c1ba9700 1 --2- 172.21.15.203:0/531728669 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff8bc0be1b0 0x7ff8bc1461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:12.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c23aa700 1 -- 172.21.15.203:0/531728669 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff8bc0be1b0 msgr2=0x7ff8bc1461d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:12.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c23aa700 1 --2- 172.21.15.203:0/531728669 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff8bc0be1b0 0x7ff8bc1461d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:12.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c23aa700 1 -- 172.21.15.203:0/531728669 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8bc143750 msgr2=0x7ff8bc143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:12.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c23aa700 1 --2- 172.21.15.203:0/531728669 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8bc143750 0x7ff8bc143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:12.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c23aa700 1 -- 172.21.15.203:0/531728669 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc148090 con 0x7ff8bc063010 2021-01-06T22:12:12.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c23aa700 1 --2- 172.21.15.203:0/531728669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc063010 0x7ff8bc145ce0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7ff8b800c710 tx=0x7ff8b8007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:12.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8aa7fc700 1 -- 172.21.15.203:0/531728669 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8007020 con 0x7ff8bc063010 2021-01-06T22:12:12.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8aa7fc700 1 -- 172.21.15.203:0/531728669 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b8007180 con 0x7ff8bc063010 2021-01-06T22:12:12.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c33ac700 1 -- 172.21.15.203:0/531728669 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8bc0ade30 con 0x7ff8bc063010 2021-01-06T22:12:12.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8aa7fc700 1 -- 172.21.15.203:0/531728669 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b800e560 con 0x7ff8bc063010 2021-01-06T22:12:12.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c33ac700 1 -- 172.21.15.203:0/531728669 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc0ae370 con 0x7ff8bc063010 2021-01-06T22:12:12.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c33ac700 1 -- 172.21.15.203:0/531728669 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8bc0bd4d0 con 0x7ff8bc063010 2021-01-06T22:12:12.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8aa7fc700 1 -- 172.21.15.203:0/531728669 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff8b8010070 con 0x7ff8bc063010 2021-01-06T22:12:12.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8aa7fc700 1 --2- 172.21.15.203:0/531728669 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff89c0563d0 0x7ff89c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:12:12.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8aa7fc700 1 -- 172.21.15.203:0/531728669 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6503+0+0 (secure 0 0 0) 0x7ff8b806ea20 con 0x7ff8bc063010 2021-01-06T22:12:12.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.929+0000 7ff8c1ba9700 1 --2- 172.21.15.203:0/531728669 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff89c0563d0 0x7ff89c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:12:12.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.933+0000 7ff8c1ba9700 1 --2- 172.21.15.203:0/531728669 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff89c0563d0 0x7ff89c058890 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7ff8bc0637d0 tx=0x7ff8ac016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:12:12.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:12.937+0000 7ff8aa7fc700 1 -- 172.21.15.203:0/531728669 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff8b8043670 con 0x7ff8bc063010 2021-01-06T22:12:13.154 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:13.149+0000 7ff8c33ac700 1 -- 172.21.15.203:0/531728669 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8bc0450a0 con 0x7ff8bc063010 2021-01-06T22:12:13.155 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:13.149+0000 7ff8aa7fc700 1 -- 172.21.15.203:0/531728669 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3566 (secure 0 0 0) 0x7ff8bc0bd4d0 con 0x7ff8bc063010 2021-01-06T22:12:13.155 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:12:13.155 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:12:05.529398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4925,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4925":{"gid":4925,"name":"f","rank":0,"incarnation":45,"state":"up:active","state_seq":30,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:12:13.157 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:13.153+0000 7ff8c33ac700 1 -- 172.21.15.203:0/531728669 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff89c0563d0 msgr2=0x7ff89c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:13.157 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:13.153+0000 7ff8c33ac700 1 --2- 172.21.15.203:0/531728669 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff89c0563d0 0x7ff89c058890 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7ff8bc0637d0 tx=0x7ff8ac016000).stop 2021-01-06T22:12:13.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:13.153+0000 7ff8c33ac700 1 -- 172.21.15.203:0/531728669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc063010 msgr2=0x7ff8bc145ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:12:13.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:13.153+0000 7ff8c33ac700 1 --2- 172.21.15.203:0/531728669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc063010 0x7ff8bc145ce0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7ff8b800c710 tx=0x7ff8b8007c40).stop 2021-01-06T22:12:13.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:13.153+0000 7ff8c33ac700 1 -- 172.21.15.203:0/531728669 shutdown_connections 2021-01-06T22:12:13.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:13.153+0000 7ff8c33ac700 1 --2- 172.21.15.203:0/531728669 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff89c0563d0 0x7ff89c058890 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:13.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:13.153+0000 7ff8c33ac700 1 --2- 172.21.15.203:0/531728669 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8bc143750 0x7ff8bc143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:13.159 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:13.153+0000 7ff8c33ac700 1 --2- 172.21.15.203:0/531728669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc063010 0x7ff8bc145ce0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:13.159 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:13.153+0000 7ff8c33ac700 1 --2- 172.21.15.203:0/531728669 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff8bc0be1b0 0x7ff8bc1461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:12:13.159 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:13.153+0000 7ff8c33ac700 1 -- 172.21.15.203:0/531728669 >> 172.21.15.203:0/531728669 conn(0x7ff8bc067e20 msgr2=0x7ff8bc0b98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:12:13.159 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:13.153+0000 7ff8c33ac700 1 -- 172.21.15.203:0/531728669 shutdown_connections 2021-01-06T22:12:13.160 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:12:13.153+0000 7ff8c33ac700 1 -- 172.21.15.203:0/531728669 wait complete. 2021-01-06T22:12:13.160 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 50 2021-01-06T22:12:13.171 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2021-01-06T22:12:13.172 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 106 secs before thrashing 2021-01-06T22:12:16.952 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:12:16.955 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:12:47.014 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:12:47.019 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:13:17.189 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:13:17.192 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:13:47.227 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:13:47.230 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:13:59.173 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:13:59.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f8694d03700 1 Processor -- start 2021-01-06T22:13:59.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f8694d03700 1 -- start start 2021-01-06T22:13:59.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f8694d03700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f86900a5a60 0x7f869014fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:13:59.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f8694d03700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8690067660 con 0x7f86900a5680 2021-01-06T22:13:59.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f8694d03700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8690064ab0 con 0x7f8690060fe0 2021-01-06T22:13:59.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f8694d03700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86901500f0 con 0x7f86900a5a60 2021-01-06T22:13:59.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868effd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f8690060fe0 0x7f86900bc9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50112/0 (socket says 172.21.15.203:50112) 2021-01-06T22:13:59.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868effd700 1 -- 172.21.15.203:0/285072505 learned_addr learned my addr 172.21.15.203:0/285072505 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:13:59.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868ffff700 1 --2- 172.21.15.203:0/285072505 >> v2:172.21.15.91:3300/0 conn(0x7f86900a5a60 0x7f869014fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:13:59.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 -- 172.21.15.203:0/285072505 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868667461 0 0) 0x7f8690064ab0 con 0x7f8690060fe0 2021-01-06T22:13:59.304 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 -- 172.21.15.203:0/285072505 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8670003230 con 0x7f8690060fe0 2021-01-06T22:13:59.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 -- 172.21.15.203:0/285072505 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623476205 0 0) 0x7f8690067660 con 0x7f86900a5680 2021-01-06T22:13:59.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 -- 172.21.15.203:0/285072505 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8690064ab0 con 0x7f86900a5680 2021-01-06T22:13:59.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 -- 172.21.15.203:0/285072505 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4162553812 0 0) 0x7f8670003230 con 0x7f8690060fe0 2021-01-06T22:13:59.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 -- 172.21.15.203:0/285072505 >> v2:172.21.15.91:3300/0 conn(0x7f86900a5a60 msgr2=0x7f869014fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:13:59.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 --2- 172.21.15.203:0/285072505 >> v2:172.21.15.91:3300/0 conn(0x7f86900a5a60 0x7f869014fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 -- 172.21.15.203:0/285072505 >> v1:172.21.15.91:6789/0 conn(0x7f86900a5680 legacy=0x7f86900608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:13:59.308 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 -- 172.21.15.203:0/285072505 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86900678c0 con 0x7f8690060fe0 2021-01-06T22:13:59.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 -- 172.21.15.203:0/285072505 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f8678001500 con 0x7f8690060fe0 2021-01-06T22:13:59.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 -- 172.21.15.203:0/285072505 >> v1:172.21.15.203:6789/0 conn(0x7f8690060fe0 legacy=0x7f86900bc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:13:59.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 --2- 172.21.15.203:0/285072505 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86700038b0 0x7f8670003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:13:59.309 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 --2- 172.21.15.203:0/285072505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86700044d0 0x7f8670004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:13:59.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 --2- 172.21.15.203:0/285072505 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8670004e40 0x7f867000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:13:59.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 -- 172.21.15.203:0/285072505 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86901500f0 con 0x7f8670004e40 2021-01-06T22:13:59.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 -- 172.21.15.203:0/285072505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86700053a0 con 0x7f86700044d0 2021-01-06T22:13:59.310 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868e7fc700 1 -- 172.21.15.203:0/285072505 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867000bc90 con 0x7f86700038b0 2021-01-06T22:13:59.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868effd700 1 --2- 172.21.15.203:0/285072505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86700044d0 0x7f8670004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:13:59.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868ffff700 1 --2- 172.21.15.203:0/285072505 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8670004e40 0x7f867000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:13:59.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868f7fe700 1 --2- 172.21.15.203:0/285072505 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86700038b0 0x7f8670003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:13:59.311 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868effd700 1 -- 172.21.15.203:0/285072505 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86700038b0 msgr2=0x7f8670003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:13:59.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868effd700 1 --2- 172.21.15.203:0/285072505 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86700038b0 0x7f8670003cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868effd700 1 -- 172.21.15.203:0/285072505 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8670004e40 msgr2=0x7f867000b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:13:59.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868effd700 1 --2- 172.21.15.203:0/285072505 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8670004e40 0x7f867000b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868ffff700 1 -- 172.21.15.203:0/285072505 reap_dead start 2021-01-06T22:13:59.312 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868effd700 1 -- 172.21.15.203:0/285072505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670003230 con 0x7f86700044d0 2021-01-06T22:13:59.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.299+0000 7f868f7fe700 1 --2- 172.21.15.203:0/285072505 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86700038b0 0x7f8670003cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:13:59.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.303+0000 7f868effd700 1 --2- 172.21.15.203:0/285072505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86700044d0 0x7f8670004950 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f867800c3f0 tx=0x7f8690067470).ready entity=mon.1 client_cookie=da72098a7fdc1218 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:13:59.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.303+0000 7f868e7fc700 1 -- 172.21.15.203:0/285072505 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780070d0 con 0x7f86700044d0 2021-01-06T22:13:59.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.303+0000 7f868e7fc700 1 -- 172.21.15.203:0/285072505 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8678007230 con 0x7f86700044d0 2021-01-06T22:13:59.313 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.303+0000 7f868e7fc700 1 -- 172.21.15.203:0/285072505 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780055e0 con 0x7f86700044d0 2021-01-06T22:13:59.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.303+0000 7f8694d03700 1 -- 172.21.15.203:0/285072505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86700044d0 msgr2=0x7f8670004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:13:59.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.303+0000 7f8694d03700 1 --2- 172.21.15.203:0/285072505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86700044d0 0x7f8670004950 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f867800c3f0 tx=0x7f8690067470).stop 2021-01-06T22:13:59.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.303+0000 7f8694d03700 1 -- 172.21.15.203:0/285072505 shutdown_connections 2021-01-06T22:13:59.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.303+0000 7f8694d03700 1 --2- 172.21.15.203:0/285072505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86700044d0 0x7f8670004950 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.314 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.303+0000 7f8694d03700 1 -- 172.21.15.203:0/285072505 >> 172.21.15.203:0/285072505 conn(0x7f8690061530 msgr2=0x7f8690063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:13:59.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.303+0000 7f8694d03700 1 -- 172.21.15.203:0/285072505 shutdown_connections 2021-01-06T22:13:59.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.303+0000 7f8694d03700 1 -- 172.21.15.203:0/285072505 wait complete. 2021-01-06T22:13:59.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.303+0000 7f8694d03700 1 Processor -- start 2021-01-06T22:13:59.315 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.303+0000 7f8694d03700 1 -- start start 2021-01-06T22:13:59.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f8694d03700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86700044d0 0x7f86900a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:13:59.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f8694d03700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f869014d470 0x7f86900a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:13:59.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f8694d03700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f869015e030 0x7f86900a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:13:59.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f8694d03700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86900688b0 con 0x7f86700044d0 2021-01-06T22:13:59.318 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f8694d03700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86900b3db0 con 0x7f869014d470 2021-01-06T22:13:59.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f8694d03700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86900b4050 con 0x7f869015e030 2021-01-06T22:13:59.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f868effd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f869014d470 0x7f86900a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:13:59.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f868effd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f869014d470 0x7f86900a3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51530/0 (socket says 172.21.15.203:51530) 2021-01-06T22:13:59.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f868effd700 1 -- 172.21.15.203:0/882606572 learned_addr learned my addr 172.21.15.203:0/882606572 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:13:59.320 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f868f7fe700 1 --2- 172.21.15.203:0/882606572 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86700044d0 0x7f86900a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:13:59.320 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f868ffff700 1 --2- 172.21.15.203:0/882606572 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f869015e030 0x7f86900a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:13:59.320 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f868effd700 1 -- 172.21.15.203:0/882606572 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f869015e030 msgr2=0x7f86900a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:13:59.320 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f868effd700 1 --2- 172.21.15.203:0/882606572 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f869015e030 0x7f86900a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.320 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f868effd700 1 -- 172.21.15.203:0/882606572 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86700044d0 msgr2=0x7f86900a32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:13:59.321 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f868effd700 1 --2- 172.21.15.203:0/882606572 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86700044d0 0x7f86900a32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.321 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f868effd700 1 -- 172.21.15.203:0/882606572 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86900b42f0 con 0x7f869014d470 2021-01-06T22:13:59.321 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f868effd700 1 --2- 172.21.15.203:0/882606572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f869014d470 0x7f86900a3eb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f867000bc60 tx=0x7f8678015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:13:59.321 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f86877fe700 1 -- 172.21.15.203:0/882606572 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780059d0 con 0x7f869014d470 2021-01-06T22:13:59.321 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f86877fe700 1 -- 172.21.15.203:0/882606572 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8678018070 con 0x7f869014d470 2021-01-06T22:13:59.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f8694d03700 1 -- 172.21.15.203:0/882606572 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86900b4590 con 0x7f869014d470 2021-01-06T22:13:59.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f86877fe700 1 -- 172.21.15.203:0/882606572 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678005d40 con 0x7f869014d470 2021-01-06T22:13:59.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f8694d03700 1 -- 172.21.15.203:0/882606572 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86900b4cd0 con 0x7f869014d470 2021-01-06T22:13:59.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f8694d03700 1 -- 172.21.15.203:0/882606572 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86900450a0 con 0x7f869014d470 2021-01-06T22:13:59.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f86877fe700 1 -- 172.21.15.203:0/882606572 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f8678010070 con 0x7f869014d470 2021-01-06T22:13:59.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f86877fe700 1 --2- 172.21.15.203:0/882606572 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f86740563d0 0x7f8674058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:13:59.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.307+0000 7f86877fe700 1 -- 172.21.15.203:0/882606572 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6503+0+0 (secure 0 0 0) 0x7f86780702f0 con 0x7f869014d470 2021-01-06T22:13:59.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.311+0000 7f868f7fe700 1 --2- 172.21.15.203:0/882606572 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f86740563d0 0x7f8674058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:13:59.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.311+0000 7f868f7fe700 1 --2- 172.21.15.203:0/882606572 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f86740563d0 0x7f8674058890 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f869005b830 tx=0x7f8680009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:13:59.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.315+0000 7f86877fe700 1 -- 172.21.15.203:0/882606572 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f86780459c0 con 0x7f869014d470 2021-01-06T22:13:59.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.539+0000 7f8694d03700 1 -- 172.21.15.203:0/882606572 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f869005ca60 con 0x7f869014d470 2021-01-06T22:13:59.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.543+0000 7f86877fe700 1 -- 172.21.15.203:0/882606572 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3566 (secure 0 0 0) 0x7f8678048fe0 con 0x7f869014d470 2021-01-06T22:13:59.544 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:13:59.544 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:12:05.529398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4925,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4925":{"gid":4925,"name":"f","rank":0,"incarnation":45,"state":"up:active","state_seq":30,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:13:59.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.543+0000 7f8694d03700 1 -- 172.21.15.203:0/882606572 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f86740563d0 msgr2=0x7f8674058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:13:59.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.543+0000 7f8694d03700 1 --2- 172.21.15.203:0/882606572 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f86740563d0 0x7f8674058890 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f869005b830 tx=0x7f8680009040).stop 2021-01-06T22:13:59.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.543+0000 7f8694d03700 1 -- 172.21.15.203:0/882606572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f869014d470 msgr2=0x7f86900a3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:13:59.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.543+0000 7f8694d03700 1 --2- 172.21.15.203:0/882606572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f869014d470 0x7f86900a3eb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f867000bc60 tx=0x7f8678015000).stop 2021-01-06T22:13:59.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.543+0000 7f8694d03700 1 -- 172.21.15.203:0/882606572 shutdown_connections 2021-01-06T22:13:59.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.543+0000 7f8694d03700 1 --2- 172.21.15.203:0/882606572 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f86740563d0 0x7f8674058890 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.543+0000 7f8694d03700 1 --2- 172.21.15.203:0/882606572 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f869015e030 0x7f86900a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.543+0000 7f8694d03700 1 --2- 172.21.15.203:0/882606572 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86700044d0 0x7f86900a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.543+0000 7f8694d03700 1 --2- 172.21.15.203:0/882606572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f869014d470 0x7f86900a3eb0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.543+0000 7f8694d03700 1 -- 172.21.15.203:0/882606572 >> 172.21.15.203:0/882606572 conn(0x7f8690061530 msgr2=0x7f86900bc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:13:59.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.543+0000 7f8694d03700 1 -- 172.21.15.203:0/882606572 shutdown_connections 2021-01-06T22:13:59.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.543+0000 7f8694d03700 1 -- 172.21.15.203:0/882606572 wait complete. 2021-01-06T22:13:59.549 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 50 2021-01-06T22:13:59.564 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=0) 2021-01-06T22:13:59.565 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2021-01-06T22:13:59.565 INFO:teuthology.orchestra.run:waiting for 300 2021-01-06T22:13:59.579 INFO:tasks.ceph.mds.f:Stopped 2021-01-06T22:13:59.580 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:13:59.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.711+0000 7f5a8b59e700 1 Processor -- start 2021-01-06T22:13:59.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.711+0000 7f5a8b59e700 1 -- start start 2021-01-06T22:13:59.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.711+0000 7f5a8b59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a8c0602b0 0x7f5a8c14ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:13:59.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.711+0000 7f5a8b59e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5a8c060690 0x7f5a8c060b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:13:59.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.711+0000 7f5a8b59e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a8c0668d0 con 0x7f5a8c0bd6d0 2021-01-06T22:13:59.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.711+0000 7f5a8b59e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a8c067a90 con 0x7f5a8c060690 2021-01-06T22:13:59.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.711+0000 7f5a8b59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c066b70 con 0x7f5a8c0602b0 2021-01-06T22:13:59.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a81d9b700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f5a8c0bd6d0 0x7f5a8c14b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50134/0 (socket says 172.21.15.203:50134) 2021-01-06T22:13:59.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a81d9b700 1 -- 172.21.15.203:0/678322785 learned_addr learned my addr 172.21.15.203:0/678322785 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:13:59.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a8ad9d700 1 --2- 172.21.15.203:0/678322785 >> v2:172.21.15.91:3300/0 conn(0x7f5a8c060690 0x7f5a8c060b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:13:59.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a8a59c700 1 --2- 172.21.15.203:0/678322785 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a8c0602b0 0x7f5a8c14ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:13:59.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a89d9b700 1 -- 172.21.15.203:0/678322785 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1191216531 0 0) 0x7f5a8c0668d0 con 0x7f5a8c0bd6d0 2021-01-06T22:13:59.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a89d9b700 1 -- 172.21.15.203:0/678322785 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a6c003230 con 0x7f5a8c0bd6d0 2021-01-06T22:13:59.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a89d9b700 1 -- 172.21.15.203:0/678322785 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1688697946 0 0) 0x7f5a6c003230 con 0x7f5a8c0bd6d0 2021-01-06T22:13:59.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a8ad9d700 1 -- 172.21.15.203:0/678322785 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a8c0602b0 msgr2=0x7f5a8c14ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:13:59.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a8ad9d700 1 --2- 172.21.15.203:0/678322785 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a8c0602b0 0x7f5a8c14ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a8ad9d700 1 -- 172.21.15.203:0/678322785 >> v1:172.21.15.203:6789/0 conn(0x7f5a8c0bd6d0 legacy=0x7f5a8c14b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:13:59.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a8ad9d700 1 -- 172.21.15.203:0/678322785 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c066e10 con 0x7f5a8c060690 2021-01-06T22:13:59.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a8ad9d700 1 --2- 172.21.15.203:0/678322785 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a8c060690 0x7f5a8c060b20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f5a84009c90 tx=0x7f5a840091d0).ready entity=mon.0 client_cookie=1d2287fc979225ac server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:13:59.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a89d9b700 1 -- 172.21.15.203:0/678322785 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a740014e0 con 0x7f5a8c060690 2021-01-06T22:13:59.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a89d9b700 1 -- 172.21.15.203:0/678322785 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8400b740 con 0x7f5a8c060690 2021-01-06T22:13:59.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a89d9b700 1 -- 172.21.15.203:0/678322785 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8400bac0 con 0x7f5a8c060690 2021-01-06T22:13:59.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a8b59e700 1 -- 172.21.15.203:0/678322785 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a8c060690 msgr2=0x7f5a8c060b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:13:59.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a8b59e700 1 --2- 172.21.15.203:0/678322785 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a8c060690 0x7f5a8c060b20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f5a84009c90 tx=0x7f5a840091d0).stop 2021-01-06T22:13:59.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a8b59e700 1 -- 172.21.15.203:0/678322785 shutdown_connections 2021-01-06T22:13:59.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a8b59e700 1 --2- 172.21.15.203:0/678322785 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a8c060690 0x7f5a8c060b20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a8b59e700 1 --2- 172.21.15.203:0/678322785 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a8c0602b0 0x7f5a8c14ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a8b59e700 1 -- 172.21.15.203:0/678322785 >> 172.21.15.203:0/678322785 conn(0x7f5a8c061530 msgr2=0x7f5a8c063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:13:59.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a8b59e700 1 -- 172.21.15.203:0/678322785 shutdown_connections 2021-01-06T22:13:59.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.715+0000 7f5a8b59e700 1 -- 172.21.15.203:0/678322785 wait complete. 2021-01-06T22:13:59.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8b59e700 1 Processor -- start 2021-01-06T22:13:59.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8b59e700 1 -- start start 2021-01-06T22:13:59.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8b59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8c0602b0 0x7f5a8c15bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:13:59.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8b59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a8c060690 0x7f5a8c0a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:13:59.727 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8b59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a8c0bd6d0 0x7f5a8c0bf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:13:59.727 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8b59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c15c070 con 0x7f5a8c0bd6d0 2021-01-06T22:13:59.727 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8b59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c0ba7d0 con 0x7f5a8c0602b0 2021-01-06T22:13:59.727 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8b59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c0a0ed0 con 0x7f5a8c060690 2021-01-06T22:13:59.728 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8a59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8c0602b0 0x7f5a8c15bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:13:59.728 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8a59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8c0602b0 0x7f5a8c15bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51544/0 (socket says 172.21.15.203:51544) 2021-01-06T22:13:59.728 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8a59c700 1 -- 172.21.15.203:0/1576533548 learned_addr learned my addr 172.21.15.203:0/1576533548 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:13:59.728 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a81d9b700 1 --2- 172.21.15.203:0/1576533548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a8c060690 0x7f5a8c0a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:13:59.728 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8ad9d700 1 --2- 172.21.15.203:0/1576533548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a8c0bd6d0 0x7f5a8c0bf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:13:59.729 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8a59c700 1 -- 172.21.15.203:0/1576533548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a8c060690 msgr2=0x7f5a8c0a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:13:59.729 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8a59c700 1 --2- 172.21.15.203:0/1576533548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a8c060690 0x7f5a8c0a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.729 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8a59c700 1 -- 172.21.15.203:0/1576533548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a8c0bd6d0 msgr2=0x7f5a8c0bf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:13:59.729 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8a59c700 1 --2- 172.21.15.203:0/1576533548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a8c0bd6d0 0x7f5a8c0bf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.729 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8a59c700 1 -- 172.21.15.203:0/1576533548 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c0a1170 con 0x7f5a8c0602b0 2021-01-06T22:13:59.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a81d9b700 1 --2- 172.21.15.203:0/1576533548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a8c060690 0x7f5a8c0a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:13:59.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8a59c700 1 --2- 172.21.15.203:0/1576533548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8c0602b0 0x7f5a8c15bb80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f5a7c00eb20 tx=0x7f5a7c006de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:13:59.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a82ffd700 1 -- 172.21.15.203:0/1576533548 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c00ca40 con 0x7f5a8c0602b0 2021-01-06T22:13:59.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a82ffd700 1 -- 172.21.15.203:0/1576533548 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7c00cba0 con 0x7f5a8c0602b0 2021-01-06T22:13:59.731 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8b59e700 1 -- 172.21.15.203:0/1576533548 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8c0a1410 con 0x7f5a8c0602b0 2021-01-06T22:13:59.731 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a82ffd700 1 -- 172.21.15.203:0/1576533548 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c0123f0 con 0x7f5a8c0602b0 2021-01-06T22:13:59.731 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8b59e700 1 -- 172.21.15.203:0/1576533548 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8c0baca0 con 0x7f5a8c0602b0 2021-01-06T22:13:59.731 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.719+0000 7f5a8b59e700 1 -- 172.21.15.203:0/1576533548 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a8c0bafa0 con 0x7f5a8c0602b0 2021-01-06T22:13:59.731 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.723+0000 7f5a82ffd700 1 -- 172.21.15.203:0/1576533548 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f5a7c012760 con 0x7f5a8c0602b0 2021-01-06T22:13:59.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.723+0000 7f5a82ffd700 1 --2- 172.21.15.203:0/1576533548 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5a700563d0 0x7f5a70058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:13:59.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.723+0000 7f5a82ffd700 1 -- 172.21.15.203:0/1576533548 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6503+0+0 (secure 0 0 0) 0x7f5a7c06fc40 con 0x7f5a8c0602b0 2021-01-06T22:13:59.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.723+0000 7f5a81d9b700 1 --2- 172.21.15.203:0/1576533548 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5a700563d0 0x7f5a70058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:13:59.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.723+0000 7f5a81d9b700 1 --2- 172.21.15.203:0/1576533548 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5a700563d0 0x7f5a70058890 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f5a8c0b6bc0 tx=0x7f5a74009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:13:59.733 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.731+0000 7f5a82ffd700 1 -- 172.21.15.203:0/1576533548 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f5a8c0bafa0 con 0x7f5a8c0602b0 2021-01-06T22:13:59.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.959+0000 7f5a8b59e700 1 -- 172.21.15.203:0/1576533548 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a8c0bafa0 con 0x7f5a8c0602b0 2021-01-06T22:13:59.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.959+0000 7f5a82ffd700 1 -- 172.21.15.203:0/1576533548 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3566 (secure 0 0 0) 0x7f5a8c0bafa0 con 0x7f5a8c0602b0 2021-01-06T22:13:59.961 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:13:59.961 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:12:05.529398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4925,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4925":{"gid":4925,"name":"f","rank":0,"incarnation":45,"state":"up:active","state_seq":30,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:13:59.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.959+0000 7f5a8b59e700 1 -- 172.21.15.203:0/1576533548 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5a700563d0 msgr2=0x7f5a70058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:13:59.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.959+0000 7f5a8b59e700 1 --2- 172.21.15.203:0/1576533548 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5a700563d0 0x7f5a70058890 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f5a8c0b6bc0 tx=0x7f5a74009040).stop 2021-01-06T22:13:59.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.959+0000 7f5a8b59e700 1 -- 172.21.15.203:0/1576533548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8c0602b0 msgr2=0x7f5a8c15bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:13:59.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.959+0000 7f5a8b59e700 1 --2- 172.21.15.203:0/1576533548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8c0602b0 0x7f5a8c15bb80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f5a7c00eb20 tx=0x7f5a7c006de0).stop 2021-01-06T22:13:59.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.959+0000 7f5a8b59e700 1 -- 172.21.15.203:0/1576533548 shutdown_connections 2021-01-06T22:13:59.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.959+0000 7f5a8b59e700 1 --2- 172.21.15.203:0/1576533548 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5a700563d0 0x7f5a70058890 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.959+0000 7f5a8b59e700 1 --2- 172.21.15.203:0/1576533548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a8c0bd6d0 0x7f5a8c0bf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.959+0000 7f5a8b59e700 1 --2- 172.21.15.203:0/1576533548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8c0602b0 0x7f5a8c15bb80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.959+0000 7f5a8b59e700 1 --2- 172.21.15.203:0/1576533548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a8c060690 0x7f5a8c0a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:13:59.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.959+0000 7f5a8b59e700 1 -- 172.21.15.203:0/1576533548 >> 172.21.15.203:0/1576533548 conn(0x7f5a8c061530 msgr2=0x7f5a8c0bbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:13:59.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.959+0000 7f5a8b59e700 1 -- 172.21.15.203:0/1576533548 shutdown_connections 2021-01-06T22:13:59.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:13:59.959+0000 7f5a8b59e700 1 -- 172.21.15.203:0/1576533548 wait complete. 2021-01-06T22:13:59.965 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 50 2021-01-06T22:13:59.975 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:14:01.976 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:14:02.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.103+0000 7f9e92c39700 1 Processor -- start 2021-01-06T22:14:02.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.103+0000 7f9e92c39700 1 -- start start 2021-01-06T22:14:02.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e92c39700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9e8c0bc700 0x7f9e8c0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:02.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e92c39700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8c14c1c0 con 0x7f9e8c061130 2021-01-06T22:14:02.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e92c39700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8c14c460 con 0x7f9e8c14bb30 2021-01-06T22:14:02.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e92c39700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e8c14ff90 con 0x7f9e8c0bc700 2021-01-06T22:14:02.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e91436700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9e8c0bc700 0x7f9e8c0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:02.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e92438700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f9e8c14bb30 0x7f9e8c148d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50162/0 (socket says 172.21.15.203:50162) 2021-01-06T22:14:02.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e92438700 1 -- 172.21.15.203:0/1852210509 learned_addr learned my addr 172.21.15.203:0/1852210509 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:14:02.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e91436700 1 -- 172.21.15.203:0/1852210509 >> v1:172.21.15.203:6789/0 conn(0x7f9e8c14bb30 legacy=0x7f9e8c148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:02.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e91436700 1 -- 172.21.15.203:0/1852210509 >> v1:172.21.15.91:6789/0 conn(0x7f9e8c061130 legacy=0x7f9e8c14f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:02.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e90c35700 1 -- 172.21.15.203:0/1852210509 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2798359514 0 0) 0x7f9e8c14c460 con 0x7f9e8c14bb30 2021-01-06T22:14:02.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e91436700 1 -- 172.21.15.203:0/1852210509 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8c14c700 con 0x7f9e8c0bc700 2021-01-06T22:14:02.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e91436700 1 --2- 172.21.15.203:0/1852210509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e8c0bc700 0x7f9e8c0c0fa0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f9e7c00b9b0 tx=0x7f9e7c006de0).ready entity=mon.1 client_cookie=f66693ba3742f24b server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:02.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e90c35700 1 -- 172.21.15.203:0/1852210509 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c0098c0 con 0x7f9e8c0bc700 2021-01-06T22:14:02.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e90c35700 1 -- 172.21.15.203:0/1852210509 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7c009a20 con 0x7f9e8c0bc700 2021-01-06T22:14:02.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e90c35700 1 -- 172.21.15.203:0/1852210509 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c009da0 con 0x7f9e8c0bc700 2021-01-06T22:14:02.114 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e92c39700 1 -- 172.21.15.203:0/1852210509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e8c0bc700 msgr2=0x7f9e8c0c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:02.114 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e92c39700 1 --2- 172.21.15.203:0/1852210509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e8c0bc700 0x7f9e8c0c0fa0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f9e7c00b9b0 tx=0x7f9e7c006de0).stop 2021-01-06T22:14:02.114 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e92c39700 1 -- 172.21.15.203:0/1852210509 shutdown_connections 2021-01-06T22:14:02.114 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e92c39700 1 --2- 172.21.15.203:0/1852210509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e8c0bc700 0x7f9e8c0c0fa0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:02.115 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e92c39700 1 -- 172.21.15.203:0/1852210509 >> 172.21.15.203:0/1852210509 conn(0x7f9e8c0667b0 msgr2=0x7f9e8c0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:02.115 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e92c39700 1 -- 172.21.15.203:0/1852210509 shutdown_connections 2021-01-06T22:14:02.115 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.107+0000 7f9e92c39700 1 -- 172.21.15.203:0/1852210509 wait complete. 2021-01-06T22:14:02.124 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e92c39700 1 Processor -- start 2021-01-06T22:14:02.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e92c39700 1 -- start start 2021-01-06T22:14:02.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e92c39700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e8c061130 0x7f9e8c143b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:02.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e92c39700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e8c0bc700 0x7f9e8c141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:02.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e92c39700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e8c14bb30 0x7f9e8c142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:02.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e92c39700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c1513b0 con 0x7f9e8c14bb30 2021-01-06T22:14:02.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e92c39700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c15b860 con 0x7f9e8c061130 2021-01-06T22:14:02.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e92c39700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c15bb00 con 0x7f9e8c0bc700 2021-01-06T22:14:02.128 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e91c37700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e8c061130 0x7f9e8c143b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:02.128 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e91c37700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e8c061130 0x7f9e8c143b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51570/0 (socket says 172.21.15.203:51570) 2021-01-06T22:14:02.128 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e91c37700 1 -- 172.21.15.203:0/245485186 learned_addr learned my addr 172.21.15.203:0/245485186 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:14:02.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e92438700 1 --2- 172.21.15.203:0/245485186 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e8c14bb30 0x7f9e8c142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:02.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e91436700 1 --2- 172.21.15.203:0/245485186 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e8c0bc700 0x7f9e8c141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:02.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e91c37700 1 -- 172.21.15.203:0/245485186 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e8c0bc700 msgr2=0x7f9e8c141d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:02.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e91c37700 1 --2- 172.21.15.203:0/245485186 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e8c0bc700 0x7f9e8c141d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:02.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e91c37700 1 -- 172.21.15.203:0/245485186 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e8c14bb30 msgr2=0x7f9e8c142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:02.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e91c37700 1 --2- 172.21.15.203:0/245485186 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e8c14bb30 0x7f9e8c142280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:02.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.111+0000 7f9e91c37700 1 -- 172.21.15.203:0/245485186 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8c15bda0 con 0x7f9e8c061130 2021-01-06T22:14:02.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.115+0000 7f9e91c37700 1 --2- 172.21.15.203:0/245485186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e8c061130 0x7f9e8c143b20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f9e8800c710 tx=0x7f9e88007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:02.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.115+0000 7f9e79ffb700 1 -- 172.21.15.203:0/245485186 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e88007020 con 0x7f9e8c061130 2021-01-06T22:14:02.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.115+0000 7f9e79ffb700 1 -- 172.21.15.203:0/245485186 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e88007180 con 0x7f9e8c061130 2021-01-06T22:14:02.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.115+0000 7f9e92c39700 1 -- 172.21.15.203:0/245485186 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8c15c040 con 0x7f9e8c061130 2021-01-06T22:14:02.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.115+0000 7f9e79ffb700 1 -- 172.21.15.203:0/245485186 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8800e560 con 0x7f9e8c061130 2021-01-06T22:14:02.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.115+0000 7f9e92c39700 1 -- 172.21.15.203:0/245485186 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8c142770 con 0x7f9e8c061130 2021-01-06T22:14:02.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.115+0000 7f9e92c39700 1 -- 172.21.15.203:0/245485186 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8c1496c0 con 0x7f9e8c061130 2021-01-06T22:14:02.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.115+0000 7f9e79ffb700 1 -- 172.21.15.203:0/245485186 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f9e88010070 con 0x7f9e8c061130 2021-01-06T22:14:02.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.115+0000 7f9e79ffb700 1 --2- 172.21.15.203:0/245485186 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9e6c0563d0 0x7f9e6c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:02.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.115+0000 7f9e79ffb700 1 -- 172.21.15.203:0/245485186 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6503+0+0 (secure 0 0 0) 0x7f9e8806eb90 con 0x7f9e8c061130 2021-01-06T22:14:02.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.115+0000 7f9e91436700 1 --2- 172.21.15.203:0/245485186 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9e6c0563d0 0x7f9e6c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:02.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.119+0000 7f9e91436700 1 --2- 172.21.15.203:0/245485186 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9e6c0563d0 0x7f9e6c058890 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f9e7c006e20 tx=0x7f9e7c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:02.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.122+0000 7f9e79ffb700 1 -- 172.21.15.203:0/245485186 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f9e88043670 con 0x7f9e8c061130 2021-01-06T22:14:02.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.322+0000 7f9e92c39700 1 -- 172.21.15.203:0/245485186 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e8c0450a0 con 0x7f9e8c061130 2021-01-06T22:14:02.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.322+0000 7f9e79ffb700 1 -- 172.21.15.203:0/245485186 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3566 (secure 0 0 0) 0x7f9e8c1496c0 con 0x7f9e8c061130 2021-01-06T22:14:02.325 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:14:02.325 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:12:05.529398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4925,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4925":{"gid":4925,"name":"f","rank":0,"incarnation":45,"state":"up:active","state_seq":30,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:14:02.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.322+0000 7f9e92c39700 1 -- 172.21.15.203:0/245485186 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9e6c0563d0 msgr2=0x7f9e6c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:02.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.322+0000 7f9e92c39700 1 --2- 172.21.15.203:0/245485186 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9e6c0563d0 0x7f9e6c058890 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f9e7c006e20 tx=0x7f9e7c016000).stop 2021-01-06T22:14:02.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.322+0000 7f9e92c39700 1 -- 172.21.15.203:0/245485186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e8c061130 msgr2=0x7f9e8c143b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:02.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.322+0000 7f9e92c39700 1 --2- 172.21.15.203:0/245485186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e8c061130 0x7f9e8c143b20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f9e8800c710 tx=0x7f9e88007c40).stop 2021-01-06T22:14:02.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.322+0000 7f9e92c39700 1 -- 172.21.15.203:0/245485186 shutdown_connections 2021-01-06T22:14:02.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.322+0000 7f9e92c39700 1 --2- 172.21.15.203:0/245485186 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9e6c0563d0 0x7f9e6c058890 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:02.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.322+0000 7f9e92c39700 1 --2- 172.21.15.203:0/245485186 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e8c14bb30 0x7f9e8c142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:02.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.322+0000 7f9e92c39700 1 --2- 172.21.15.203:0/245485186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e8c061130 0x7f9e8c143b20 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:02.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.322+0000 7f9e92c39700 1 --2- 172.21.15.203:0/245485186 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e8c0bc700 0x7f9e8c141d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:02.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.322+0000 7f9e92c39700 1 -- 172.21.15.203:0/245485186 >> 172.21.15.203:0/245485186 conn(0x7f9e8c0667b0 msgr2=0x7f9e8c067cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:02.330 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.322+0000 7f9e92c39700 1 -- 172.21.15.203:0/245485186 shutdown_connections 2021-01-06T22:14:02.330 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:02.322+0000 7f9e92c39700 1 -- 172.21.15.203:0/245485186 wait complete. 2021-01-06T22:14:02.330 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 50 2021-01-06T22:14:02.341 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:14:04.342 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:14:04.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff594b3700 1 Processor -- start 2021-01-06T22:14:04.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff594b3700 1 -- start start 2021-01-06T22:14:04.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff594b3700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff54063010 0x7eff5414e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:04.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff594b3700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff540be990 con 0x7eff540be1b0 2021-01-06T22:14:04.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff594b3700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff540be6f0 con 0x7eff540633f0 2021-01-06T22:14:04.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff594b3700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5414ec60 con 0x7eff54063010 2021-01-06T22:14:04.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff537fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7eff540633f0 0x7eff54064b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50184/0 (socket says 172.21.15.203:50184) 2021-01-06T22:14:04.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff537fe700 1 -- 172.21.15.203:0/2409949567 learned_addr learned my addr 172.21.15.203:0/2409949567 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:14:04.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff53fff700 1 --2- 172.21.15.203:0/2409949567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff54063010 0x7eff5414e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:04.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff52ffd700 1 -- 172.21.15.203:0/2409949567 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599999557 0 0) 0x7eff540be6f0 con 0x7eff540633f0 2021-01-06T22:14:04.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff52ffd700 1 -- 172.21.15.203:0/2409949567 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff34003230 con 0x7eff540633f0 2021-01-06T22:14:04.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff52ffd700 1 -- 172.21.15.203:0/2409949567 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2575415510 0 0) 0x7eff34003230 con 0x7eff540633f0 2021-01-06T22:14:04.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff52ffd700 1 -- 172.21.15.203:0/2409949567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff54063010 msgr2=0x7eff5414e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:04.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff52ffd700 1 --2- 172.21.15.203:0/2409949567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff54063010 0x7eff5414e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:04.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff52ffd700 1 -- 172.21.15.203:0/2409949567 >> v1:172.21.15.91:6789/0 conn(0x7eff540be1b0 legacy=0x7eff54147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:04.480 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff52ffd700 1 -- 172.21.15.203:0/2409949567 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff540c1050 con 0x7eff540633f0 2021-01-06T22:14:04.480 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.470+0000 7eff52ffd700 1 -- 172.21.15.203:0/2409949567 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7eff3c001860 con 0x7eff540633f0 2021-01-06T22:14:04.480 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff52ffd700 1 -- 172.21.15.203:0/2409949567 >> v1:172.21.15.203:6789/0 conn(0x7eff540633f0 legacy=0x7eff54064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:04.480 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff52ffd700 1 --2- 172.21.15.203:0/2409949567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff34003720 0x7eff34003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:04.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff52ffd700 1 --2- 172.21.15.203:0/2409949567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff34004010 0x7eff340044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:04.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff52ffd700 1 --2- 172.21.15.203:0/2409949567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff54063010 0x7eff34004b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:04.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff53fff700 1 --2- 172.21.15.203:0/2409949567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff34003720 0x7eff34003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:04.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff52ffd700 1 -- 172.21.15.203:0/2409949567 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5414ec60 con 0x7eff34004010 2021-01-06T22:14:04.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff52ffd700 1 -- 172.21.15.203:0/2409949567 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff34005070 con 0x7eff34003720 2021-01-06T22:14:04.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff52ffd700 1 -- 172.21.15.203:0/2409949567 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff34005230 con 0x7eff54063010 2021-01-06T22:14:04.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff537fe700 1 --2- 172.21.15.203:0/2409949567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff34004010 0x7eff340044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:04.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff58cb2700 1 --2- 172.21.15.203:0/2409949567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff54063010 0x7eff34004b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:04.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff53fff700 1 -- 172.21.15.203:0/2409949567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff54063010 msgr2=0x7eff34004b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:04.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff53fff700 1 --2- 172.21.15.203:0/2409949567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff54063010 0x7eff34004b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:04.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff53fff700 1 -- 172.21.15.203:0/2409949567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff34004010 msgr2=0x7eff340044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:04.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff53fff700 1 --2- 172.21.15.203:0/2409949567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff34004010 0x7eff340044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:04.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff53fff700 1 -- 172.21.15.203:0/2409949567 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff34003230 con 0x7eff34003720 2021-01-06T22:14:04.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff58cb2700 1 --2- 172.21.15.203:0/2409949567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff54063010 0x7eff34004b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:14:04.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff53fff700 1 --2- 172.21.15.203:0/2409949567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff34003720 0x7eff34003b00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7eff4400bbd0 tx=0x7eff44006de0).ready entity=mon.1 client_cookie=d154c18ce18e80f8 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:04.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff52ffd700 1 -- 172.21.15.203:0/2409949567 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4400b2b0 con 0x7eff34003720 2021-01-06T22:14:04.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff52ffd700 1 -- 172.21.15.203:0/2409949567 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4400eae0 con 0x7eff34003720 2021-01-06T22:14:04.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff52ffd700 1 -- 172.21.15.203:0/2409949567 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4400ee80 con 0x7eff34003720 2021-01-06T22:14:04.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff594b3700 1 -- 172.21.15.203:0/2409949567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff34003720 msgr2=0x7eff34003b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:04.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff594b3700 1 --2- 172.21.15.203:0/2409949567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff34003720 0x7eff34003b00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7eff4400bbd0 tx=0x7eff44006de0).stop 2021-01-06T22:14:04.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff58cb2700 1 -- 172.21.15.203:0/2409949567 reap_dead start 2021-01-06T22:14:04.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff594b3700 1 -- 172.21.15.203:0/2409949567 shutdown_connections 2021-01-06T22:14:04.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff594b3700 1 -- 172.21.15.203:0/2409949567 >> 172.21.15.203:0/2409949567 conn(0x7eff54067e20 msgr2=0x7eff54068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:04.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff594b3700 1 -- 172.21.15.203:0/2409949567 shutdown_connections 2021-01-06T22:14:04.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff594b3700 1 -- 172.21.15.203:0/2409949567 wait complete. 2021-01-06T22:14:04.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff594b3700 1 Processor -- start 2021-01-06T22:14:04.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.474+0000 7eff594b3700 1 -- start start 2021-01-06T22:14:04.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff594b3700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff34003720 0x7eff540b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:04.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff594b3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff540acf20 0x7eff540bf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:04.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff594b3700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff5414de60 0x7eff541439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:04.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff594b3700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff540b7d40 con 0x7eff34003720 2021-01-06T22:14:04.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff594b3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff540ae810 con 0x7eff540acf20 2021-01-06T22:14:04.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff594b3700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff540aea70 con 0x7eff5414de60 2021-01-06T22:14:04.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff537fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff540acf20 0x7eff540bf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:04.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff537fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff540acf20 0x7eff540bf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51604/0 (socket says 172.21.15.203:51604) 2021-01-06T22:14:04.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff537fe700 1 -- 172.21.15.203:0/261423615 learned_addr learned my addr 172.21.15.203:0/261423615 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:14:04.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff53fff700 1 --2- 172.21.15.203:0/261423615 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff34003720 0x7eff540b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:04.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff58cb2700 1 --2- 172.21.15.203:0/261423615 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff5414de60 0x7eff541439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:04.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff537fe700 1 -- 172.21.15.203:0/261423615 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff5414de60 msgr2=0x7eff541439a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:04.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff537fe700 1 --2- 172.21.15.203:0/261423615 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff5414de60 0x7eff541439a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:04.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff537fe700 1 -- 172.21.15.203:0/261423615 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff34003720 msgr2=0x7eff540b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:04.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff537fe700 1 --2- 172.21.15.203:0/261423615 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff34003720 0x7eff540b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:04.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff537fe700 1 -- 172.21.15.203:0/261423615 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff540aed10 con 0x7eff540acf20 2021-01-06T22:14:04.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff53fff700 1 --2- 172.21.15.203:0/261423615 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff34003720 0x7eff540b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:14:04.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff537fe700 1 --2- 172.21.15.203:0/261423615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff540acf20 0x7eff540bf980 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7eff3c00e750 tx=0x7eff540bdb10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:04.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff4bfff700 1 -- 172.21.15.203:0/261423615 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c010500 con 0x7eff540acf20 2021-01-06T22:14:04.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff4bfff700 1 -- 172.21.15.203:0/261423615 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3c010660 con 0x7eff540acf20 2021-01-06T22:14:04.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff594b3700 1 -- 172.21.15.203:0/261423615 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff540aefb0 con 0x7eff540acf20 2021-01-06T22:14:04.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff4bfff700 1 -- 172.21.15.203:0/261423615 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0109e0 con 0x7eff540acf20 2021-01-06T22:14:04.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff594b3700 1 -- 172.21.15.203:0/261423615 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5414a180 con 0x7eff540acf20 2021-01-06T22:14:04.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff594b3700 1 -- 172.21.15.203:0/261423615 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff540450a0 con 0x7eff540acf20 2021-01-06T22:14:04.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff4bfff700 1 -- 172.21.15.203:0/261423615 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7eff3c010d50 con 0x7eff540acf20 2021-01-06T22:14:04.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.478+0000 7eff4bfff700 1 --2- 172.21.15.203:0/261423615 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7eff38056400 0x7eff380588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:04.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.482+0000 7eff4bfff700 1 -- 172.21.15.203:0/261423615 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6503+0+0 (secure 0 0 0) 0x7eff3c06fd50 con 0x7eff540acf20 2021-01-06T22:14:04.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.482+0000 7eff53fff700 1 --2- 172.21.15.203:0/261423615 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7eff38056400 0x7eff380588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:04.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.482+0000 7eff53fff700 1 --2- 172.21.15.203:0/261423615 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7eff38056400 0x7eff380588c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7eff44006cf0 tx=0x7eff44017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:04.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.486+0000 7eff4bfff700 1 -- 172.21.15.203:0/261423615 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7eff3c045420 con 0x7eff540acf20 2021-01-06T22:14:04.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.682+0000 7eff594b3700 1 -- 172.21.15.203:0/261423615 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff540bfe70 con 0x7eff540acf20 2021-01-06T22:14:04.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.686+0000 7eff4bfff700 1 -- 172.21.15.203:0/261423615 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3566 (secure 0 0 0) 0x7eff3c022e00 con 0x7eff540acf20 2021-01-06T22:14:04.689 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:14:04.689 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:12:05.529398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4925,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4925":{"gid":4925,"name":"f","rank":0,"incarnation":45,"state":"up:active","state_seq":30,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:14:04.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.690+0000 7eff594b3700 1 -- 172.21.15.203:0/261423615 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7eff38056400 msgr2=0x7eff380588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:04.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.690+0000 7eff594b3700 1 --2- 172.21.15.203:0/261423615 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7eff38056400 0x7eff380588c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7eff44006cf0 tx=0x7eff44017000).stop 2021-01-06T22:14:04.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.690+0000 7eff594b3700 1 -- 172.21.15.203:0/261423615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff540acf20 msgr2=0x7eff540bf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:04.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.690+0000 7eff594b3700 1 --2- 172.21.15.203:0/261423615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff540acf20 0x7eff540bf980 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7eff3c00e750 tx=0x7eff540bdb10).stop 2021-01-06T22:14:04.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.690+0000 7eff594b3700 1 -- 172.21.15.203:0/261423615 shutdown_connections 2021-01-06T22:14:04.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.690+0000 7eff594b3700 1 --2- 172.21.15.203:0/261423615 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7eff38056400 0x7eff380588c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:04.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.690+0000 7eff594b3700 1 --2- 172.21.15.203:0/261423615 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff5414de60 0x7eff541439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:04.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.690+0000 7eff594b3700 1 --2- 172.21.15.203:0/261423615 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff34003720 0x7eff540b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:04.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.690+0000 7eff594b3700 1 --2- 172.21.15.203:0/261423615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff540acf20 0x7eff540bf980 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:04.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.690+0000 7eff594b3700 1 -- 172.21.15.203:0/261423615 >> 172.21.15.203:0/261423615 conn(0x7eff54067e20 msgr2=0x7eff540652e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:04.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.690+0000 7eff594b3700 1 -- 172.21.15.203:0/261423615 shutdown_connections 2021-01-06T22:14:04.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:04.690+0000 7eff594b3700 1 -- 172.21.15.203:0/261423615 wait complete. 2021-01-06T22:14:04.695 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 50 2021-01-06T22:14:04.708 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:14:06.710 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:14:06.843 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.838+0000 7ff791ca7700 1 Processor -- start 2021-01-06T22:14:06.843 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.838+0000 7ff791ca7700 1 -- start start 2021-01-06T22:14:06.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff791ca7700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff78c0ba9d0 0x7ff78c15cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:06.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff791ca7700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78c0646b0 con 0x7ff78c0a5e50 2021-01-06T22:14:06.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff791ca7700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78c152cc0 con 0x7ff78c0bb3f0 2021-01-06T22:14:06.848 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff791ca7700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78c0670c0 con 0x7ff78c0ba9d0 2021-01-06T22:14:06.848 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff790ca5700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7ff78c0bb3f0 0x7ff78c0669b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50208/0 (socket says 172.21.15.203:50208) 2021-01-06T22:14:06.848 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff790ca5700 1 -- 172.21.15.203:0/3560065373 learned_addr learned my addr 172.21.15.203:0/3560065373 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:14:06.848 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff7914a6700 1 --2- 172.21.15.203:0/3560065373 >> v2:172.21.15.91:3300/0 conn(0x7ff78c0ba9d0 0x7ff78c15cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:06.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 -- 172.21.15.203:0/3560065373 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1141605535 0 0) 0x7ff78c152cc0 con 0x7ff78c0bb3f0 2021-01-06T22:14:06.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 -- 172.21.15.203:0/3560065373 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff770003230 con 0x7ff78c0bb3f0 2021-01-06T22:14:06.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 -- 172.21.15.203:0/3560065373 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 821222643 0 0) 0x7ff770003230 con 0x7ff78c0bb3f0 2021-01-06T22:14:06.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 -- 172.21.15.203:0/3560065373 >> v2:172.21.15.91:3300/0 conn(0x7ff78c0ba9d0 msgr2=0x7ff78c15cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:06.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 --2- 172.21.15.203:0/3560065373 >> v2:172.21.15.91:3300/0 conn(0x7ff78c0ba9d0 0x7ff78c15cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:06.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 -- 172.21.15.203:0/3560065373 >> v1:172.21.15.91:6789/0 conn(0x7ff78c0a5e50 legacy=0x7ff78c0bb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:06.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 -- 172.21.15.203:0/3560065373 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c064450 con 0x7ff78c0bb3f0 2021-01-06T22:14:06.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 -- 172.21.15.203:0/3560065373 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7ff7800014e0 con 0x7ff78c0bb3f0 2021-01-06T22:14:06.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 -- 172.21.15.203:0/3560065373 >> v1:172.21.15.203:6789/0 conn(0x7ff78c0bb3f0 legacy=0x7ff78c0669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:06.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 --2- 172.21.15.203:0/3560065373 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff770003720 0x7ff770003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:06.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 --2- 172.21.15.203:0/3560065373 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff7700044d0 0x7ff770004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:06.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 --2- 172.21.15.203:0/3560065373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770004e40 0x7ff77000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:06.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 -- 172.21.15.203:0/3560065373 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c0670c0 con 0x7ff770003720 2021-01-06T22:14:06.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 -- 172.21.15.203:0/3560065373 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff770004250 con 0x7ff770004e40 2021-01-06T22:14:06.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 -- 172.21.15.203:0/3560065373 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77000bc10 con 0x7ff7700044d0 2021-01-06T22:14:06.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff7914a6700 1 --2- 172.21.15.203:0/3560065373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770004e40 0x7ff77000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:06.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff790ca5700 1 --2- 172.21.15.203:0/3560065373 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff770003720 0x7ff770003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:06.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78bfff700 1 --2- 172.21.15.203:0/3560065373 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff7700044d0 0x7ff770004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:06.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff7914a6700 1 -- 172.21.15.203:0/3560065373 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff7700044d0 msgr2=0x7ff770004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:06.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff7914a6700 1 --2- 172.21.15.203:0/3560065373 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff7700044d0 0x7ff770004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:06.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff7914a6700 1 -- 172.21.15.203:0/3560065373 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff770003720 msgr2=0x7ff770003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:06.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff7914a6700 1 --2- 172.21.15.203:0/3560065373 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff770003720 0x7ff770003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:06.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff7914a6700 1 -- 172.21.15.203:0/3560065373 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff770003230 con 0x7ff770004e40 2021-01-06T22:14:06.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78bfff700 1 --2- 172.21.15.203:0/3560065373 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff7700044d0 0x7ff770004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:14:06.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff7914a6700 1 -- 172.21.15.203:0/3560065373 reap_dead start 2021-01-06T22:14:06.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff7914a6700 1 --2- 172.21.15.203:0/3560065373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770004e40 0x7ff77000b720 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7ff784009e90 tx=0x7ff7840091d0).ready entity=mon.1 client_cookie=a42cbcc4d989d3e4 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:06.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 -- 172.21.15.203:0/3560065373 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78400edc0 con 0x7ff770004e40 2021-01-06T22:14:06.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 -- 172.21.15.203:0/3560065373 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7840123f0 con 0x7ff770004e40 2021-01-06T22:14:06.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff78b7fe700 1 -- 172.21.15.203:0/3560065373 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7840127b0 con 0x7ff770004e40 2021-01-06T22:14:06.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff791ca7700 1 -- 172.21.15.203:0/3560065373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770004e40 msgr2=0x7ff77000b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:06.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff791ca7700 1 --2- 172.21.15.203:0/3560065373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770004e40 0x7ff77000b720 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7ff784009e90 tx=0x7ff7840091d0).stop 2021-01-06T22:14:06.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff791ca7700 1 -- 172.21.15.203:0/3560065373 shutdown_connections 2021-01-06T22:14:06.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff791ca7700 1 --2- 172.21.15.203:0/3560065373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770004e40 0x7ff77000b720 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:06.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff791ca7700 1 -- 172.21.15.203:0/3560065373 >> 172.21.15.203:0/3560065373 conn(0x7ff78c061530 msgr2=0x7ff78c063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:06.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff791ca7700 1 -- 172.21.15.203:0/3560065373 shutdown_connections 2021-01-06T22:14:06.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.842+0000 7ff791ca7700 1 -- 172.21.15.203:0/3560065373 wait complete. 2021-01-06T22:14:06.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff791ca7700 1 Processor -- start 2021-01-06T22:14:06.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff791ca7700 1 -- start start 2021-01-06T22:14:06.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff791ca7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770004e40 0x7ff78c09f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:06.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff791ca7700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff78c0b87e0 0x7ff78c1548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:06.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff791ca7700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff78c0b9e60 0x7ff78c14c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:06.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff791ca7700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c161b20 con 0x7ff78c0b87e0 2021-01-06T22:14:06.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff791ca7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c161dc0 con 0x7ff770004e40 2021-01-06T22:14:06.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff791ca7700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c162060 con 0x7ff78c0b9e60 2021-01-06T22:14:06.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff790ca5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770004e40 0x7ff78c09f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:06.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff790ca5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770004e40 0x7ff78c09f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51626/0 (socket says 172.21.15.203:51626) 2021-01-06T22:14:06.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff790ca5700 1 -- 172.21.15.203:0/239145914 learned_addr learned my addr 172.21.15.203:0/239145914 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:14:06.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff78bfff700 1 --2- 172.21.15.203:0/239145914 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff78c0b87e0 0x7ff78c1548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:06.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff7914a6700 1 --2- 172.21.15.203:0/239145914 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff78c0b9e60 0x7ff78c14c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:06.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff790ca5700 1 -- 172.21.15.203:0/239145914 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff78c0b9e60 msgr2=0x7ff78c14c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:06.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff790ca5700 1 --2- 172.21.15.203:0/239145914 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff78c0b9e60 0x7ff78c14c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:06.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff790ca5700 1 -- 172.21.15.203:0/239145914 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff78c0b87e0 msgr2=0x7ff78c1548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:06.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff790ca5700 1 --2- 172.21.15.203:0/239145914 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff78c0b87e0 0x7ff78c1548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:06.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff790ca5700 1 -- 172.21.15.203:0/239145914 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c162300 con 0x7ff770004e40 2021-01-06T22:14:06.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff790ca5700 1 --2- 172.21.15.203:0/239145914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770004e40 0x7ff78c09f3e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7ff78000ea50 tx=0x7ff78c061aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:06.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff788ff9700 1 -- 172.21.15.203:0/239145914 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000ee20 con 0x7ff770004e40 2021-01-06T22:14:06.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff788ff9700 1 -- 172.21.15.203:0/239145914 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff780010040 con 0x7ff770004e40 2021-01-06T22:14:06.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff791ca7700 1 -- 172.21.15.203:0/239145914 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78c1625a0 con 0x7ff770004e40 2021-01-06T22:14:06.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff788ff9700 1 -- 172.21.15.203:0/239145914 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000a9f0 con 0x7ff770004e40 2021-01-06T22:14:06.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.846+0000 7ff791ca7700 1 -- 172.21.15.203:0/239145914 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78c154de0 con 0x7ff770004e40 2021-01-06T22:14:06.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.850+0000 7ff791ca7700 1 -- 172.21.15.203:0/239145914 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78c154fa0 con 0x7ff770004e40 2021-01-06T22:14:06.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.850+0000 7ff788ff9700 1 -- 172.21.15.203:0/239145914 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff78001b020 con 0x7ff770004e40 2021-01-06T22:14:06.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.850+0000 7ff788ff9700 1 --2- 172.21.15.203:0/239145914 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff76c0563d0 0x7ff76c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:06.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.850+0000 7ff788ff9700 1 -- 172.21.15.203:0/239145914 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6503+0+0 (secure 0 0 0) 0x7ff78006ed40 con 0x7ff770004e40 2021-01-06T22:14:06.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.850+0000 7ff78bfff700 1 --2- 172.21.15.203:0/239145914 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff76c0563d0 0x7ff76c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:06.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.850+0000 7ff78bfff700 1 --2- 172.21.15.203:0/239145914 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff76c0563d0 0x7ff76c058890 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7ff78c150630 tx=0x7ff778009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:06.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:06.858+0000 7ff788ff9700 1 -- 172.21.15.203:0/239145914 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff780044300 con 0x7ff770004e40 2021-01-06T22:14:07.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:07.062+0000 7ff791ca7700 1 -- 172.21.15.203:0/239145914 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff78c0450a0 con 0x7ff770004e40 2021-01-06T22:14:07.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:07.062+0000 7ff788ff9700 1 -- 172.21.15.203:0/239145914 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3566 (secure 0 0 0) 0x7ff780047920 con 0x7ff770004e40 2021-01-06T22:14:07.066 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:14:07.067 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:12:05.529398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4925,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4925":{"gid":4925,"name":"f","rank":0,"incarnation":45,"state":"up:active","state_seq":30,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:14:07.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:07.062+0000 7ff791ca7700 1 -- 172.21.15.203:0/239145914 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff76c0563d0 msgr2=0x7ff76c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:07.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:07.062+0000 7ff791ca7700 1 --2- 172.21.15.203:0/239145914 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff76c0563d0 0x7ff76c058890 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7ff78c150630 tx=0x7ff778009040).stop 2021-01-06T22:14:07.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:07.062+0000 7ff791ca7700 1 -- 172.21.15.203:0/239145914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770004e40 msgr2=0x7ff78c09f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:07.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:07.062+0000 7ff791ca7700 1 --2- 172.21.15.203:0/239145914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770004e40 0x7ff78c09f3e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7ff78000ea50 tx=0x7ff78c061aa0).stop 2021-01-06T22:14:07.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:07.066+0000 7ff791ca7700 1 -- 172.21.15.203:0/239145914 shutdown_connections 2021-01-06T22:14:07.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:07.066+0000 7ff791ca7700 1 --2- 172.21.15.203:0/239145914 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff76c0563d0 0x7ff76c058890 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:07.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:07.066+0000 7ff791ca7700 1 --2- 172.21.15.203:0/239145914 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff78c0b9e60 0x7ff78c14c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:07.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:07.066+0000 7ff791ca7700 1 --2- 172.21.15.203:0/239145914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770004e40 0x7ff78c09f3e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:07.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:07.066+0000 7ff791ca7700 1 --2- 172.21.15.203:0/239145914 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff78c0b87e0 0x7ff78c1548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:07.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:07.066+0000 7ff791ca7700 1 -- 172.21.15.203:0/239145914 >> 172.21.15.203:0/239145914 conn(0x7ff78c061530 msgr2=0x7ff78c1512c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:07.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:07.066+0000 7ff791ca7700 1 -- 172.21.15.203:0/239145914 shutdown_connections 2021-01-06T22:14:07.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:07.066+0000 7ff791ca7700 1 -- 172.21.15.203:0/239145914 wait complete. 2021-01-06T22:14:07.071 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 50 2021-01-06T22:14:07.081 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:14:09.082 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:14:09.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.210+0000 7f371eb42700 1 Processor -- start 2021-01-06T22:14:09.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.210+0000 7f371eb42700 1 -- start start 2021-01-06T22:14:09.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371eb42700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f37180a5680 0x7f371814fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:09.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371eb42700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3718060fe0 0x7f3718064340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:09.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371eb42700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3718067660 con 0x7f37180a5a60 2021-01-06T22:14:09.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371eb42700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3718064ab0 con 0x7f3718060fe0 2021-01-06T22:14:09.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371eb42700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37181500f0 con 0x7f37180a5680 2021-01-06T22:14:09.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371db40700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f37180a5680 0x7f371814fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:09.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371d33f700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f37180a5a60 0x7f37180608d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:44882/0 (socket says 172.21.15.203:44882) 2021-01-06T22:14:09.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371e341700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3718060fe0 0x7f3718064340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:09.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371d33f700 1 -- 172.21.15.203:0/3911789698 learned_addr learned my addr 172.21.15.203:0/3911789698 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:14:09.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371e341700 1 -- 172.21.15.203:0/3911789698 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f37180a5680 msgr2=0x7f371814fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:09.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371e341700 1 --2- 172.21.15.203:0/3911789698 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f37180a5680 0x7f371814fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:09.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371e341700 1 -- 172.21.15.203:0/3911789698 >> v1:172.21.15.91:6789/0 conn(0x7f37180a5a60 legacy=0x7f37180608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:09.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371cb3e700 1 -- 172.21.15.203:0/3911789698 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570701044 0 0) 0x7f3718067660 con 0x7f37180a5a60 2021-01-06T22:14:09.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371e341700 1 -- 172.21.15.203:0/3911789698 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3718064830 con 0x7f3718060fe0 2021-01-06T22:14:09.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371db40700 1 --2- 172.21.15.203:0/3911789698 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f37180a5680 0x7f371814fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:14:09.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371e341700 1 --2- 172.21.15.203:0/3911789698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3718060fe0 0x7f3718064340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f3710009c90 tx=0x7f37100091d0).ready entity=mon.0 client_cookie=4b808cb19b28d3da server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:09.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371cb3e700 1 -- 172.21.15.203:0/3911789698 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371000bb00 con 0x7f3718060fe0 2021-01-06T22:14:09.222 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371cb3e700 1 -- 172.21.15.203:0/3911789698 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371000bc60 con 0x7f3718060fe0 2021-01-06T22:14:09.222 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371cb3e700 1 -- 172.21.15.203:0/3911789698 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710011400 con 0x7f3718060fe0 2021-01-06T22:14:09.222 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371eb42700 1 -- 172.21.15.203:0/3911789698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3718060fe0 msgr2=0x7f3718064340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:09.222 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371eb42700 1 --2- 172.21.15.203:0/3911789698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3718060fe0 0x7f3718064340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f3710009c90 tx=0x7f37100091d0).stop 2021-01-06T22:14:09.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371eb42700 1 -- 172.21.15.203:0/3911789698 shutdown_connections 2021-01-06T22:14:09.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371eb42700 1 --2- 172.21.15.203:0/3911789698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3718060fe0 0x7f3718064340 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:09.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371eb42700 1 --2- 172.21.15.203:0/3911789698 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f37180a5680 0x7f371814fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:09.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371eb42700 1 -- 172.21.15.203:0/3911789698 >> 172.21.15.203:0/3911789698 conn(0x7f3718061530 msgr2=0x7f3718063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:09.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371eb42700 1 -- 172.21.15.203:0/3911789698 shutdown_connections 2021-01-06T22:14:09.224 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.214+0000 7f371eb42700 1 -- 172.21.15.203:0/3911789698 wait complete. 2021-01-06T22:14:09.224 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371eb42700 1 Processor -- start 2021-01-06T22:14:09.226 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371eb42700 1 -- start start 2021-01-06T22:14:09.226 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371eb42700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3718060fe0 0x7f37180b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:09.226 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371eb42700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37180a5680 0x7f37180bef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:09.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371eb42700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f37180a5a60 0x7f371815d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:09.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371eb42700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37180b7910 con 0x7f3718060fe0 2021-01-06T22:14:09.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371eb42700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37180be4e0 con 0x7f37180a5680 2021-01-06T22:14:09.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371eb42700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37180b9560 con 0x7f37180a5a60 2021-01-06T22:14:09.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371d33f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37180a5680 0x7f37180bef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:09.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371d33f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37180a5680 0x7f37180bef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51640/0 (socket says 172.21.15.203:51640) 2021-01-06T22:14:09.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371d33f700 1 -- 172.21.15.203:0/746544539 learned_addr learned my addr 172.21.15.203:0/746544539 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:14:09.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371e341700 1 --2- 172.21.15.203:0/746544539 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f37180a5a60 0x7f371815d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:09.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371db40700 1 --2- 172.21.15.203:0/746544539 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3718060fe0 0x7f37180b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:09.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371d33f700 1 -- 172.21.15.203:0/746544539 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f37180a5a60 msgr2=0x7f371815d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:09.229 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371d33f700 1 --2- 172.21.15.203:0/746544539 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f37180a5a60 0x7f371815d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:09.229 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371d33f700 1 -- 172.21.15.203:0/746544539 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3718060fe0 msgr2=0x7f37180b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:09.229 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371d33f700 1 --2- 172.21.15.203:0/746544539 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3718060fe0 0x7f37180b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:09.229 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371d33f700 1 -- 172.21.15.203:0/746544539 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37180bdfe0 con 0x7f37180a5680 2021-01-06T22:14:09.229 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371d33f700 1 --2- 172.21.15.203:0/746544539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37180a5680 0x7f37180bef50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f370800c760 tx=0x7f3708007c70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:09.230 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f3705ffb700 1 -- 172.21.15.203:0/746544539 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37080072e0 con 0x7f37180a5680 2021-01-06T22:14:09.230 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f3705ffb700 1 -- 172.21.15.203:0/746544539 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3708007440 con 0x7f37180a5680 2021-01-06T22:14:09.230 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371eb42700 1 -- 172.21.15.203:0/746544539 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37180ba1a0 con 0x7f37180a5680 2021-01-06T22:14:09.230 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f3705ffb700 1 -- 172.21.15.203:0/746544539 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37080057f0 con 0x7f37180a5680 2021-01-06T22:14:09.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.218+0000 7f371eb42700 1 -- 172.21.15.203:0/746544539 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37180b8e20 con 0x7f37180a5680 2021-01-06T22:14:09.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.222+0000 7f371eb42700 1 -- 172.21.15.203:0/746544539 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37180450a0 con 0x7f37180a5680 2021-01-06T22:14:09.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.222+0000 7f3705ffb700 1 -- 172.21.15.203:0/746544539 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f3708010070 con 0x7f37180a5680 2021-01-06T22:14:09.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.222+0000 7f3705ffb700 1 --2- 172.21.15.203:0/746544539 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f36f80563d0 0x7f36f8058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:09.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.222+0000 7f3705ffb700 1 -- 172.21.15.203:0/746544539 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6503+0+0 (secure 0 0 0) 0x7f370806eca0 con 0x7f37180a5680 2021-01-06T22:14:09.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.222+0000 7f371db40700 1 --2- 172.21.15.203:0/746544539 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f36f80563d0 0x7f36f8058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:09.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.222+0000 7f371db40700 1 --2- 172.21.15.203:0/746544539 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f36f80563d0 0x7f36f8058890 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f371400bd90 tx=0x7f3714006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:09.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.230+0000 7f3705ffb700 1 -- 172.21.15.203:0/746544539 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f3708073070 con 0x7f37180a5680 2021-01-06T22:14:09.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.426+0000 7f371eb42700 1 -- 172.21.15.203:0/746544539 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f371805ca60 con 0x7f37180a5680 2021-01-06T22:14:09.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.426+0000 7f3705ffb700 1 -- 172.21.15.203:0/746544539 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3566 (secure 0 0 0) 0x7f3708044370 con 0x7f37180a5680 2021-01-06T22:14:09.431 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:14:09.432 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:12:05.529398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4925,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4925":{"gid":4925,"name":"f","rank":0,"incarnation":45,"state":"up:active","state_seq":30,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:14:09.433 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.430+0000 7f371eb42700 1 -- 172.21.15.203:0/746544539 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f36f80563d0 msgr2=0x7f36f8058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:09.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.430+0000 7f371eb42700 1 --2- 172.21.15.203:0/746544539 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f36f80563d0 0x7f36f8058890 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f371400bd90 tx=0x7f3714006de0).stop 2021-01-06T22:14:09.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.430+0000 7f371eb42700 1 -- 172.21.15.203:0/746544539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37180a5680 msgr2=0x7f37180bef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:09.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.430+0000 7f371eb42700 1 --2- 172.21.15.203:0/746544539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37180a5680 0x7f37180bef50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f370800c760 tx=0x7f3708007c70).stop 2021-01-06T22:14:09.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.430+0000 7f371eb42700 1 -- 172.21.15.203:0/746544539 shutdown_connections 2021-01-06T22:14:09.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.430+0000 7f371eb42700 1 --2- 172.21.15.203:0/746544539 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f36f80563d0 0x7f36f8058890 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:09.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.430+0000 7f371eb42700 1 --2- 172.21.15.203:0/746544539 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f37180a5a60 0x7f371815d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:09.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.430+0000 7f371eb42700 1 --2- 172.21.15.203:0/746544539 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3718060fe0 0x7f37180b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:09.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.430+0000 7f371eb42700 1 --2- 172.21.15.203:0/746544539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37180a5680 0x7f37180bef50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:09.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.430+0000 7f371eb42700 1 -- 172.21.15.203:0/746544539 >> 172.21.15.203:0/746544539 conn(0x7f3718061530 msgr2=0x7f37180bd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:09.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.430+0000 7f371eb42700 1 -- 172.21.15.203:0/746544539 shutdown_connections 2021-01-06T22:14:09.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:09.430+0000 7f371eb42700 1 -- 172.21.15.203:0/746544539 wait complete. 2021-01-06T22:14:09.437 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 50 2021-01-06T22:14:09.447 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:14:11.449 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:14:11.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0ce217700 1 Processor -- start 2021-01-06T22:14:11.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0ce217700 1 -- start start 2021-01-06T22:14:11.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0ce217700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff0c80610d0 0x7ff0c814e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:11.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0ce217700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff0c8068670 0x7ff0c80bb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:11.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0ce217700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0c80614b0 0x7ff0c80bbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:11.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0ce217700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c814e5c0 con 0x7ff0c80610d0 2021-01-06T22:14:11.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0ce217700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c80671a0 con 0x7ff0c8068670 2021-01-06T22:14:11.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0ce217700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80643c0 con 0x7ff0c80614b0 2021-01-06T22:14:11.588 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0cca14700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff0c8068670 0x7ff0c80bb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:11.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0cca14700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff0c8068670 0x7ff0c80bb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51648/0 (socket says 172.21.15.203:51648) 2021-01-06T22:14:11.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0cca14700 1 -- 172.21.15.203:0/694861233 learned_addr learned my addr 172.21.15.203:0/694861233 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:14:11.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0cda16700 1 --2- 172.21.15.203:0/694861233 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0c80614b0 0x7ff0c80bbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:11.589 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0cd215700 1 --2- 172.21.15.203:0/694861233 >> v2:172.21.15.91:3300/0 conn(0x7ff0c80610d0 0x7ff0c814e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:11.590 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0cca14700 1 -- 172.21.15.203:0/694861233 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0c80614b0 msgr2=0x7ff0c80bbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:11.590 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0cca14700 1 --2- 172.21.15.203:0/694861233 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0c80614b0 0x7ff0c80bbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:11.592 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0cca14700 1 -- 172.21.15.203:0/694861233 >> v2:172.21.15.91:3300/0 conn(0x7ff0c80610d0 msgr2=0x7ff0c814e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:11.592 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0cca14700 1 --2- 172.21.15.203:0/694861233 >> v2:172.21.15.91:3300/0 conn(0x7ff0c80610d0 0x7ff0c814e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:11.592 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0cca14700 1 -- 172.21.15.203:0/694861233 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c8064660 con 0x7ff0c8068670 2021-01-06T22:14:11.592 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.582+0000 7ff0cd215700 1 --2- 172.21.15.203:0/694861233 >> v2:172.21.15.91:3300/0 conn(0x7ff0c80610d0 0x7ff0c814e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:14:11.593 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.586+0000 7ff0cca14700 1 --2- 172.21.15.203:0/694861233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0c8068670 0x7ff0c80bb9e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7ff0c8066450 tx=0x7ff0b0006ea0).ready entity=mon.1 client_cookie=4be7f214a1edcbe2 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:11.593 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.586+0000 7ff0c7fff700 1 -- 172.21.15.203:0/694861233 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0009810 con 0x7ff0c8068670 2021-01-06T22:14:11.593 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.586+0000 7ff0c7fff700 1 -- 172.21.15.203:0/694861233 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b0009970 con 0x7ff0c8068670 2021-01-06T22:14:11.593 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.586+0000 7ff0c7fff700 1 -- 172.21.15.203:0/694861233 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0009cf0 con 0x7ff0c8068670 2021-01-06T22:14:11.593 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.586+0000 7ff0ce217700 1 -- 172.21.15.203:0/694861233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0c8068670 msgr2=0x7ff0c80bb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:11.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.586+0000 7ff0ce217700 1 --2- 172.21.15.203:0/694861233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0c8068670 0x7ff0c80bb9e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7ff0c8066450 tx=0x7ff0b0006ea0).stop 2021-01-06T22:14:11.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.586+0000 7ff0ce217700 1 -- 172.21.15.203:0/694861233 shutdown_connections 2021-01-06T22:14:11.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.586+0000 7ff0ce217700 1 --2- 172.21.15.203:0/694861233 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0c80614b0 0x7ff0c80bbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:11.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.586+0000 7ff0ce217700 1 --2- 172.21.15.203:0/694861233 >> v2:172.21.15.91:3300/0 conn(0x7ff0c80610d0 0x7ff0c814e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:11.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.586+0000 7ff0ce217700 1 --2- 172.21.15.203:0/694861233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0c8068670 0x7ff0c80bb9e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:11.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.586+0000 7ff0ce217700 1 -- 172.21.15.203:0/694861233 >> 172.21.15.203:0/694861233 conn(0x7ff0c80bd3a0 msgr2=0x7ff0c80bd780 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:11.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.586+0000 7ff0ce217700 1 -- 172.21.15.203:0/694861233 shutdown_connections 2021-01-06T22:14:11.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.586+0000 7ff0ce217700 1 -- 172.21.15.203:0/694861233 wait complete. 2021-01-06T22:14:11.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.586+0000 7ff0ce217700 1 Processor -- start 2021-01-06T22:14:11.596 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.586+0000 7ff0ce217700 1 -- start start 2021-01-06T22:14:11.596 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0ce217700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0c80610d0 0x7ff0c80b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:11.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0ce217700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0c80614b0 0x7ff0c8160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:11.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0ce217700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0c8068670 0x7ff0c81610b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:11.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0ce217700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80c06a0 con 0x7ff0c80610d0 2021-01-06T22:14:11.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0ce217700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80b71d0 con 0x7ff0c80614b0 2021-01-06T22:14:11.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0ce217700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80b3fe0 con 0x7ff0c8068670 2021-01-06T22:14:11.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0cca14700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0c80614b0 0x7ff0c8160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:11.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0cca14700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0c80614b0 0x7ff0c8160bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51656/0 (socket says 172.21.15.203:51656) 2021-01-06T22:14:11.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0cca14700 1 -- 172.21.15.203:0/2007296757 learned_addr learned my addr 172.21.15.203:0/2007296757 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:14:11.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0cd215700 1 --2- 172.21.15.203:0/2007296757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0c80610d0 0x7ff0c80b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:11.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0cda16700 1 --2- 172.21.15.203:0/2007296757 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0c8068670 0x7ff0c81610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:11.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0cca14700 1 -- 172.21.15.203:0/2007296757 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0c8068670 msgr2=0x7ff0c81610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:11.601 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0cca14700 1 --2- 172.21.15.203:0/2007296757 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0c8068670 0x7ff0c81610b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:11.601 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0cca14700 1 -- 172.21.15.203:0/2007296757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0c80610d0 msgr2=0x7ff0c80b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:11.601 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0cca14700 1 --2- 172.21.15.203:0/2007296757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0c80610d0 0x7ff0c80b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:11.601 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0cca14700 1 -- 172.21.15.203:0/2007296757 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c80b4280 con 0x7ff0c80614b0 2021-01-06T22:14:11.602 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0cca14700 1 --2- 172.21.15.203:0/2007296757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0c80614b0 0x7ff0c8160bc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7ff0b0006ee0 tx=0x7ff0b0017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:11.602 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0c57fa700 1 -- 172.21.15.203:0/2007296757 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0010d60 con 0x7ff0c80614b0 2021-01-06T22:14:11.602 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0c57fa700 1 -- 172.21.15.203:0/2007296757 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b001f070 con 0x7ff0c80614b0 2021-01-06T22:14:11.602 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0ce217700 1 -- 172.21.15.203:0/2007296757 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c80b6540 con 0x7ff0c80614b0 2021-01-06T22:14:11.602 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0c57fa700 1 -- 172.21.15.203:0/2007296757 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b001b410 con 0x7ff0c80614b0 2021-01-06T22:14:11.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0ce217700 1 -- 172.21.15.203:0/2007296757 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c80b4da0 con 0x7ff0c80614b0 2021-01-06T22:14:11.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0ce217700 1 -- 172.21.15.203:0/2007296757 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c80b6ba0 con 0x7ff0c80614b0 2021-01-06T22:14:11.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.590+0000 7ff0c57fa700 1 -- 172.21.15.203:0/2007296757 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff0b002b070 con 0x7ff0c80614b0 2021-01-06T22:14:11.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.594+0000 7ff0c57fa700 1 --2- 172.21.15.203:0/2007296757 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0ac0563d0 0x7ff0ac058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:11.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.594+0000 7ff0c57fa700 1 -- 172.21.15.203:0/2007296757 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6503+0+0 (secure 0 0 0) 0x7ff0b0070e30 con 0x7ff0c80614b0 2021-01-06T22:14:11.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.594+0000 7ff0cd215700 1 --2- 172.21.15.203:0/2007296757 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0ac0563d0 0x7ff0ac058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:11.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.594+0000 7ff0cd215700 1 --2- 172.21.15.203:0/2007296757 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0ac0563d0 0x7ff0ac058890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7ff0c0007c60 tx=0x7ff0c0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:11.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.598+0000 7ff0c57fa700 1 -- 172.21.15.203:0/2007296757 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff0b0046500 con 0x7ff0c80614b0 2021-01-06T22:14:11.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.794+0000 7ff0ce217700 1 -- 172.21.15.203:0/2007296757 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0c80450a0 con 0x7ff0c80614b0 2021-01-06T22:14:11.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.794+0000 7ff0c57fa700 1 -- 172.21.15.203:0/2007296757 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3566 (secure 0 0 0) 0x7ff0b0049b20 con 0x7ff0c80614b0 2021-01-06T22:14:11.800 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:14:11.800 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:12:05.529398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4925,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4925":{"gid":4925,"name":"f","rank":0,"incarnation":45,"state":"up:active","state_seq":30,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:14:11.801 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.798+0000 7ff0ce217700 1 -- 172.21.15.203:0/2007296757 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0ac0563d0 msgr2=0x7ff0ac058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:11.801 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.798+0000 7ff0ce217700 1 --2- 172.21.15.203:0/2007296757 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0ac0563d0 0x7ff0ac058890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7ff0c0007c60 tx=0x7ff0c0008040).stop 2021-01-06T22:14:11.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.798+0000 7ff0ce217700 1 -- 172.21.15.203:0/2007296757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0c80614b0 msgr2=0x7ff0c8160bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:11.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.798+0000 7ff0ce217700 1 --2- 172.21.15.203:0/2007296757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0c80614b0 0x7ff0c8160bc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7ff0b0006ee0 tx=0x7ff0b0017000).stop 2021-01-06T22:14:11.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.798+0000 7ff0ce217700 1 -- 172.21.15.203:0/2007296757 shutdown_connections 2021-01-06T22:14:11.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.798+0000 7ff0ce217700 1 --2- 172.21.15.203:0/2007296757 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff0ac0563d0 0x7ff0ac058890 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:11.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.798+0000 7ff0ce217700 1 --2- 172.21.15.203:0/2007296757 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff0c8068670 0x7ff0c81610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:11.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.798+0000 7ff0ce217700 1 --2- 172.21.15.203:0/2007296757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0c80610d0 0x7ff0c80b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:11.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.798+0000 7ff0ce217700 1 --2- 172.21.15.203:0/2007296757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0c80614b0 0x7ff0c8160bc0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:11.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.798+0000 7ff0ce217700 1 -- 172.21.15.203:0/2007296757 >> 172.21.15.203:0/2007296757 conn(0x7ff0c80bd3a0 msgr2=0x7ff0c80bfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:11.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.798+0000 7ff0ce217700 1 -- 172.21.15.203:0/2007296757 shutdown_connections 2021-01-06T22:14:11.804 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:11.798+0000 7ff0ce217700 1 -- 172.21.15.203:0/2007296757 wait complete. 2021-01-06T22:14:11.804 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 50 2021-01-06T22:14:11.821 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:14:13.822 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:14:13.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.954+0000 7f9e5ddaf700 1 Processor -- start 2021-01-06T22:14:13.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.954+0000 7f9e5ddaf700 1 -- start start 2021-01-06T22:14:13.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.954+0000 7f9e5ddaf700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9e580c1a90 0x7f9e58150730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:13.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.954+0000 7f9e5ddaf700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e580c48d0 0x7f9e58150c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:13.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.954+0000 7f9e5ddaf700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e580c0f50 con 0x7f9e580c44f0 2021-01-06T22:14:13.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.954+0000 7f9e5ddaf700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e58066cf0 con 0x7f9e580c1a90 2021-01-06T22:14:13.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.954+0000 7f9e5ddaf700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e58060750 con 0x7f9e580c48d0 2021-01-06T22:14:13.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.954+0000 7f9e5d5ae700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e580c48d0 0x7f9e58150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:13.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.954+0000 7f9e57fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9e580c1a90 0x7f9e58150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:13.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.954+0000 7f9e5d5ae700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e580c48d0 0x7f9e58150c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:56748/0 (socket says 172.21.15.203:56748) 2021-01-06T22:14:13.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.954+0000 7f9e5d5ae700 1 -- 172.21.15.203:0/1295555239 learned_addr learned my addr 172.21.15.203:0/1295555239 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:14:13.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.954+0000 7f9e5cdad700 1 --1- 172.21.15.203:0/1295555239 >> v1:172.21.15.91:6789/0 conn(0x7f9e580c44f0 0x7f9e580c1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:44912/0 (socket says 172.21.15.203:44912) 2021-01-06T22:14:13.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.954+0000 7f9e5d5ae700 1 -- 172.21.15.203:0/1295555239 >> v2:172.21.15.91:3300/0 conn(0x7f9e580c1a90 msgr2=0x7f9e58150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:13.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.954+0000 7f9e5d5ae700 1 --2- 172.21.15.203:0/1295555239 >> v2:172.21.15.91:3300/0 conn(0x7f9e580c1a90 0x7f9e58150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:13.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.954+0000 7f9e5d5ae700 1 -- 172.21.15.203:0/1295555239 >> v1:172.21.15.91:6789/0 conn(0x7f9e580c44f0 legacy=0x7f9e580c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:13.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.954+0000 7f9e5d5ae700 1 -- 172.21.15.203:0/1295555239 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e580609f0 con 0x7f9e580c48d0 2021-01-06T22:14:13.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5d5ae700 1 --2- 172.21.15.203:0/1295555239 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e580c48d0 0x7f9e58150c20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f9e50009d50 tx=0x7f9e50009290).ready entity=mon.2 client_cookie=48c4151e5f7e078e server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:13.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e577fe700 1 -- 172.21.15.203:0/1295555239 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5000bbc0 con 0x7f9e580c48d0 2021-01-06T22:14:13.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e577fe700 1 -- 172.21.15.203:0/1295555239 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5000bd20 con 0x7f9e580c48d0 2021-01-06T22:14:13.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e577fe700 1 -- 172.21.15.203:0/1295555239 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50011610 con 0x7f9e580c48d0 2021-01-06T22:14:13.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5ddaf700 1 -- 172.21.15.203:0/1295555239 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e580c48d0 msgr2=0x7f9e58150c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:13.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5ddaf700 1 --2- 172.21.15.203:0/1295555239 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e580c48d0 0x7f9e58150c20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f9e50009d50 tx=0x7f9e50009290).stop 2021-01-06T22:14:13.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5ddaf700 1 -- 172.21.15.203:0/1295555239 shutdown_connections 2021-01-06T22:14:13.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5ddaf700 1 --2- 172.21.15.203:0/1295555239 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e580c48d0 0x7f9e58150c20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:13.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5ddaf700 1 --2- 172.21.15.203:0/1295555239 >> v2:172.21.15.91:3300/0 conn(0x7f9e580c1a90 0x7f9e58150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:13.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5ddaf700 1 -- 172.21.15.203:0/1295555239 >> 172.21.15.203:0/1295555239 conn(0x7f9e58056cd0 msgr2=0x7f9e58061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:13.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5ddaf700 1 -- 172.21.15.203:0/1295555239 shutdown_connections 2021-01-06T22:14:13.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5ddaf700 1 -- 172.21.15.203:0/1295555239 wait complete. 2021-01-06T22:14:13.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5ddaf700 1 Processor -- start 2021-01-06T22:14:13.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5ddaf700 1 -- start start 2021-01-06T22:14:13.969 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5ddaf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e580c1a90 0x7f9e5815deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:13.969 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5ddaf700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e580c44f0 0x7f9e5814d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:13.969 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5ddaf700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e580c48d0 0x7f9e580bd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:13.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5ddaf700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5815e3a0 con 0x7f9e580c44f0 2021-01-06T22:14:13.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5ddaf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e58150160 con 0x7f9e580c1a90 2021-01-06T22:14:13.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5ddaf700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e580b96e0 con 0x7f9e580c48d0 2021-01-06T22:14:13.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e5cdad700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e580c1a90 0x7f9e5815deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:13.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.958+0000 7f9e57fff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e580c44f0 0x7f9e5814d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:13.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e5cdad700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e580c1a90 0x7f9e5815deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51672/0 (socket says 172.21.15.203:51672) 2021-01-06T22:14:13.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e5cdad700 1 -- 172.21.15.203:0/1115064373 learned_addr learned my addr 172.21.15.203:0/1115064373 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:14:13.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e57fff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e580c44f0 0x7f9e5814d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:59518/0 (socket says 172.21.15.203:59518) 2021-01-06T22:14:13.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e5d5ae700 1 --2- 172.21.15.203:0/1115064373 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e580c48d0 0x7f9e580bd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:13.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e57fff700 1 -- 172.21.15.203:0/1115064373 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e580c48d0 msgr2=0x7f9e580bd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:13.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e57fff700 1 --2- 172.21.15.203:0/1115064373 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e580c48d0 0x7f9e580bd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:13.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e57fff700 1 -- 172.21.15.203:0/1115064373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e580c1a90 msgr2=0x7f9e5815deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:13.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e57fff700 1 --2- 172.21.15.203:0/1115064373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e580c1a90 0x7f9e5815deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:13.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e57fff700 1 -- 172.21.15.203:0/1115064373 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e580b9980 con 0x7f9e580c44f0 2021-01-06T22:14:13.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e57fff700 1 --2- 172.21.15.203:0/1115064373 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e580c44f0 0x7f9e5814d760 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f9e4800e970 tx=0x7f9e48006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:13.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e54ff9700 1 -- 172.21.15.203:0/1115064373 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4800b6a0 con 0x7f9e580c44f0 2021-01-06T22:14:13.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e54ff9700 1 -- 172.21.15.203:0/1115064373 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4800b800 con 0x7f9e580c44f0 2021-01-06T22:14:13.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e5ddaf700 1 -- 172.21.15.203:0/1115064373 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e580b9c20 con 0x7f9e580c44f0 2021-01-06T22:14:13.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e54ff9700 1 -- 172.21.15.203:0/1115064373 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4800bb80 con 0x7f9e580c44f0 2021-01-06T22:14:13.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e5ddaf700 1 -- 172.21.15.203:0/1115064373 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e580ba480 con 0x7f9e580c44f0 2021-01-06T22:14:13.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e5ddaf700 1 -- 172.21.15.203:0/1115064373 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e580ba640 con 0x7f9e580c44f0 2021-01-06T22:14:13.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e54ff9700 1 -- 172.21.15.203:0/1115064373 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f9e4801c460 con 0x7f9e580c44f0 2021-01-06T22:14:13.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.962+0000 7f9e54ff9700 1 --2- 172.21.15.203:0/1115064373 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9e3c0563d0 0x7f9e3c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:13.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.966+0000 7f9e54ff9700 1 -- 172.21.15.203:0/1115064373 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6503+0+0 (secure 0 0 0) 0x7f9e4806f8c0 con 0x7f9e580c44f0 2021-01-06T22:14:13.975 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.966+0000 7f9e5cdad700 1 --2- 172.21.15.203:0/1115064373 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9e3c0563d0 0x7f9e3c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:13.975 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.966+0000 7f9e5cdad700 1 --2- 172.21.15.203:0/1115064373 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9e3c0563d0 0x7f9e3c058890 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f9e40000bc0 tx=0x7f9e40007bf0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:13.979 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:13.974+0000 7f9e54ff9700 1 -- 172.21.15.203:0/1115064373 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f9e48048e60 con 0x7f9e580c44f0 2021-01-06T22:14:14.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:14.182+0000 7f9e5ddaf700 1 -- 172.21.15.203:0/1115064373 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e580450a0 con 0x7f9e580c44f0 2021-01-06T22:14:14.185 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:14.182+0000 7f9e54ff9700 1 -- 172.21.15.203:0/1115064373 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3566 (secure 0 0 0) 0x7f9e48044f30 con 0x7f9e580c44f0 2021-01-06T22:14:14.186 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:14:14.186 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:12:05.529398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4925,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4925":{"gid":4925,"name":"f","rank":0,"incarnation":45,"state":"up:active","state_seq":30,"addr":"172.21.15.91:6837/1310069455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1310069455},{"type":"v1","addr":"172.21.15.91:6837","nonce":1310069455}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:14:14.187 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:14.182+0000 7f9e5ddaf700 1 -- 172.21.15.203:0/1115064373 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9e3c0563d0 msgr2=0x7f9e3c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:14.188 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:14.182+0000 7f9e5ddaf700 1 --2- 172.21.15.203:0/1115064373 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9e3c0563d0 0x7f9e3c058890 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f9e40000bc0 tx=0x7f9e40007bf0).stop 2021-01-06T22:14:14.188 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:14.182+0000 7f9e5ddaf700 1 -- 172.21.15.203:0/1115064373 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e580c44f0 msgr2=0x7f9e5814d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:14.188 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:14.182+0000 7f9e5ddaf700 1 --2- 172.21.15.203:0/1115064373 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e580c44f0 0x7f9e5814d760 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f9e4800e970 tx=0x7f9e48006de0).stop 2021-01-06T22:14:14.188 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:14.182+0000 7f9e5ddaf700 1 -- 172.21.15.203:0/1115064373 shutdown_connections 2021-01-06T22:14:14.188 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:14.182+0000 7f9e5ddaf700 1 --2- 172.21.15.203:0/1115064373 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9e3c0563d0 0x7f9e3c058890 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:14.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:14.182+0000 7f9e5ddaf700 1 --2- 172.21.15.203:0/1115064373 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9e580c48d0 0x7f9e580bd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:14.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:14.182+0000 7f9e5ddaf700 1 --2- 172.21.15.203:0/1115064373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e580c1a90 0x7f9e5815deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:14.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:14.182+0000 7f9e5ddaf700 1 --2- 172.21.15.203:0/1115064373 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e580c44f0 0x7f9e5814d760 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:14.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:14.182+0000 7f9e5ddaf700 1 -- 172.21.15.203:0/1115064373 >> 172.21.15.203:0/1115064373 conn(0x7f9e58056cd0 msgr2=0x7f9e5814f840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:14.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:14.186+0000 7f9e5ddaf700 1 -- 172.21.15.203:0/1115064373 shutdown_connections 2021-01-06T22:14:14.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:14.186+0000 7f9e5ddaf700 1 -- 172.21.15.203:0/1115064373 wait complete. 2021-01-06T22:14:14.198 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 50 2021-01-06T22:14:14.200 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:14:16.202 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:14:16.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.330+0000 7f1ce155e700 1 Processor -- start 2021-01-06T22:14:16.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.330+0000 7f1ce155e700 1 -- start start 2021-01-06T22:14:16.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce155e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1cdc0a5e50 0x7f1cdc0ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:16.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce155e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1cdc0bb7d0 0x7f1cdc14eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:16.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce155e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc0646b0 con 0x7f1cdc0bb3f0 2021-01-06T22:14:16.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce155e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cdc0670c0 con 0x7f1cdc0a5e50 2021-01-06T22:14:16.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce155e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cdc064450 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce0d5d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1cdc0bb7d0 0x7f1cdc14eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:16.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1cdbfff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f1cdc0bb3f0 0x7f1cdc0669b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50276/0 (socket says 172.21.15.203:50276) 2021-01-06T22:14:16.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1cdbfff700 1 -- 172.21.15.203:0/3887722703 learned_addr learned my addr 172.21.15.203:0/3887722703 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:14:16.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1cdb7fe700 1 --2- 172.21.15.203:0/3887722703 >> v2:172.21.15.91:3300/0 conn(0x7f1cdc0a5e50 0x7f1cdc0ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:16.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce0d5d700 1 -- 172.21.15.203:0/3887722703 >> v2:172.21.15.91:3300/0 conn(0x7f1cdc0a5e50 msgr2=0x7f1cdc0ba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:16.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce0d5d700 1 --2- 172.21.15.203:0/3887722703 >> v2:172.21.15.91:3300/0 conn(0x7f1cdc0a5e50 0x7f1cdc0ba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce0d5d700 1 -- 172.21.15.203:0/3887722703 >> v1:172.21.15.203:6789/0 conn(0x7f1cdc0bb3f0 legacy=0x7f1cdc0669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:16.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1cdaffd700 1 -- 172.21.15.203:0/3887722703 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2457119766 0 0) 0x7f1cdc0646b0 con 0x7f1cdc0bb3f0 2021-01-06T22:14:16.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce0d5d700 1 -- 172.21.15.203:0/3887722703 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc152cc0 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce0d5d700 1 --2- 172.21.15.203:0/3887722703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cdc0bb7d0 0x7f1cdc14eef0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f1cdc0bb130 tx=0x7f1ccc006de0).ready entity=mon.1 client_cookie=e476af474603bc01 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:16.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1cdaffd700 1 -- 172.21.15.203:0/3887722703 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0096c0 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1cdaffd700 1 -- 172.21.15.203:0/3887722703 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc009820 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1cdaffd700 1 -- 172.21.15.203:0/3887722703 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc009ba0 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.342 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce155e700 1 -- 172.21.15.203:0/3887722703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cdc0bb7d0 msgr2=0x7f1cdc14eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:16.342 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce155e700 1 --2- 172.21.15.203:0/3887722703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cdc0bb7d0 0x7f1cdc14eef0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f1cdc0bb130 tx=0x7f1ccc006de0).stop 2021-01-06T22:14:16.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce155e700 1 -- 172.21.15.203:0/3887722703 shutdown_connections 2021-01-06T22:14:16.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce155e700 1 --2- 172.21.15.203:0/3887722703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cdc0bb7d0 0x7f1cdc14eef0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce155e700 1 --2- 172.21.15.203:0/3887722703 >> v2:172.21.15.91:3300/0 conn(0x7f1cdc0a5e50 0x7f1cdc0ba9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce155e700 1 -- 172.21.15.203:0/3887722703 >> 172.21.15.203:0/3887722703 conn(0x7f1cdc061530 msgr2=0x7f1cdc063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:16.343 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce155e700 1 -- 172.21.15.203:0/3887722703 shutdown_connections 2021-01-06T22:14:16.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.334+0000 7f1ce155e700 1 -- 172.21.15.203:0/3887722703 wait complete. 2021-01-06T22:14:16.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.338+0000 7f1ce155e700 1 Processor -- start 2021-01-06T22:14:16.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.338+0000 7f1ce155e700 1 -- start start 2021-01-06T22:14:16.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.338+0000 7f1ce155e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1cdc0a5e50 0x7f1cdc0b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:16.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.338+0000 7f1ce155e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1cdc0bb3f0 0x7f1cdc161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:16.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.338+0000 7f1ce155e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cdc0bb7d0 0x7f1cdc158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:16.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.338+0000 7f1ce155e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc1624e0 con 0x7f1cdc0a5e50 2021-01-06T22:14:16.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.338+0000 7f1ce155e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0b75a0 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.338+0000 7f1ce155e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0b7840 con 0x7f1cdc0bb3f0 2021-01-06T22:14:16.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1ce0d5d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cdc0bb7d0 0x7f1cdc158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:16.348 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1ce0d5d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cdc0bb7d0 0x7f1cdc158fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51690/0 (socket says 172.21.15.203:51690) 2021-01-06T22:14:16.348 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1cdbfff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1cdc0a5e50 0x7f1cdc0b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:16.348 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1ce0d5d700 1 -- 172.21.15.203:0/630505350 learned_addr learned my addr 172.21.15.203:0/630505350 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:14:16.348 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1cdb7fe700 1 --2- 172.21.15.203:0/630505350 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1cdc0bb3f0 0x7f1cdc161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:16.348 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1ce0d5d700 1 -- 172.21.15.203:0/630505350 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1cdc0bb3f0 msgr2=0x7f1cdc161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:16.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1ce0d5d700 1 --2- 172.21.15.203:0/630505350 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1cdc0bb3f0 0x7f1cdc161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1ce0d5d700 1 -- 172.21.15.203:0/630505350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1cdc0a5e50 msgr2=0x7f1cdc0b87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:16.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1ce0d5d700 1 --2- 172.21.15.203:0/630505350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1cdc0a5e50 0x7f1cdc0b87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1ce0d5d700 1 -- 172.21.15.203:0/630505350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc0b7ae0 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1cdb7fe700 1 --2- 172.21.15.203:0/630505350 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1cdc0bb3f0 0x7f1cdc161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:14:16.350 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1ce0d5d700 1 --2- 172.21.15.203:0/630505350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cdc0bb7d0 0x7f1cdc158fe0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f1cdc064590 tx=0x7f1ccc016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:16.350 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1cd3fff700 1 -- 172.21.15.203:0/630505350 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc01c630 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.350 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1cd3fff700 1 -- 172.21.15.203:0/630505350 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc01c790 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1ce155e700 1 -- 172.21.15.203:0/630505350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cdc0b7d80 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1cd3fff700 1 -- 172.21.15.203:0/630505350 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc01cb10 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.351 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1ce155e700 1 -- 172.21.15.203:0/630505350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc0b82c0 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1ce155e700 1 -- 172.21.15.203:0/630505350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cdc0450a0 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.342+0000 7f1cd3fff700 1 -- 172.21.15.203:0/630505350 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f1ccc00f070 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.346+0000 7f1cd3fff700 1 --2- 172.21.15.203:0/630505350 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1cb40563d0 0x7f1cb4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:16.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.346+0000 7f1cd3fff700 1 -- 172.21.15.203:0/630505350 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6503+0+0 (secure 0 0 0) 0x7f1ccc071100 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.346+0000 7f1cdbfff700 1 --2- 172.21.15.203:0/630505350 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1cb40563d0 0x7f1cb4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:16.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.346+0000 7f1cdbfff700 1 --2- 172.21.15.203:0/630505350 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1cb40563d0 0x7f1cb4058890 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1cd40011a0 tx=0x7f1cd40079f0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:16.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.350+0000 7f1cd3fff700 1 -- 172.21.15.203:0/630505350 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f1ccc0467d0 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.558+0000 7f1ce155e700 1 -- 172.21.15.203:0/630505350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cdc05ca60 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.558+0000 7f1cd3fff700 1 -- 172.21.15.203:0/630505350 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3212 (secure 0 0 0) 0x7f1ccc049df0 con 0x7f1cdc0bb7d0 2021-01-06T22:14:16.562 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:14:16.562 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:14:16.526363+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:replay","state_seq":2,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:14:16.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.562+0000 7f1ce155e700 1 -- 172.21.15.203:0/630505350 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1cb40563d0 msgr2=0x7f1cb4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:16.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.562+0000 7f1ce155e700 1 --2- 172.21.15.203:0/630505350 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1cb40563d0 0x7f1cb4058890 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1cd40011a0 tx=0x7f1cd40079f0).stop 2021-01-06T22:14:16.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.562+0000 7f1ce155e700 1 -- 172.21.15.203:0/630505350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cdc0bb7d0 msgr2=0x7f1cdc158fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:16.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.562+0000 7f1ce155e700 1 --2- 172.21.15.203:0/630505350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cdc0bb7d0 0x7f1cdc158fe0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f1cdc064590 tx=0x7f1ccc016000).stop 2021-01-06T22:14:16.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.562+0000 7f1ce155e700 1 -- 172.21.15.203:0/630505350 shutdown_connections 2021-01-06T22:14:16.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.562+0000 7f1ce155e700 1 --2- 172.21.15.203:0/630505350 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1cb40563d0 0x7f1cb4058890 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.568 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.562+0000 7f1ce155e700 1 --2- 172.21.15.203:0/630505350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cdc0bb7d0 0x7f1cdc158fe0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.568 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.562+0000 7f1ce155e700 1 --2- 172.21.15.203:0/630505350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1cdc0a5e50 0x7f1cdc0b87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.569 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.562+0000 7f1ce155e700 1 --2- 172.21.15.203:0/630505350 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1cdc0bb3f0 0x7f1cdc161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.569 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.562+0000 7f1ce155e700 1 -- 172.21.15.203:0/630505350 >> 172.21.15.203:0/630505350 conn(0x7f1cdc061530 msgr2=0x7f1cdc0602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:16.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.562+0000 7f1ce155e700 1 -- 172.21.15.203:0/630505350 shutdown_connections 2021-01-06T22:14:16.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.562+0000 7f1ce155e700 1 -- 172.21.15.203:0/630505350 wait complete. 2021-01-06T22:14:16.571 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 51 2021-01-06T22:14:16.582 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2021-01-06T22:14:16.582 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2021-01-06T22:14:16.583 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:14:16.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.690+0000 7efcf8d6e700 1 Processor -- start 2021-01-06T22:14:16.694 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.690+0000 7efcf8d6e700 1 -- start start 2021-01-06T22:14:16.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.690+0000 7efcf8d6e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efcf40c1430 0x7efcf4148140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:16.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.690+0000 7efcf8d6e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efcf40bb230 0x7efcf414b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:16.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.690+0000 7efcf8d6e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf414b950 con 0x7efcf4147d60 2021-01-06T22:14:16.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.690+0000 7efcf8d6e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf40bb7e0 con 0x7efcf40bb230 2021-01-06T22:14:16.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.690+0000 7efcf8d6e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf40bba80 con 0x7efcf40c1430 2021-01-06T22:14:16.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf3fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efcf40bb230 0x7efcf414b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:16.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf37fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7efcf4147d60 0x7efcf40671a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50296/0 (socket says 172.21.15.203:50296) 2021-01-06T22:14:16.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf37fe700 1 -- 172.21.15.203:0/4170743286 learned_addr learned my addr 172.21.15.203:0/4170743286 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:14:16.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf2ffd700 1 --2- 172.21.15.203:0/4170743286 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efcf40c1430 0x7efcf4148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:16.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf3fff700 1 -- 172.21.15.203:0/4170743286 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efcf40c1430 msgr2=0x7efcf4148140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:16.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf3fff700 1 --2- 172.21.15.203:0/4170743286 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efcf40c1430 0x7efcf4148140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf3fff700 1 -- 172.21.15.203:0/4170743286 >> v1:172.21.15.203:6789/0 conn(0x7efcf4147d60 legacy=0x7efcf40671a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:16.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf3fff700 1 -- 172.21.15.203:0/4170743286 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf40bbd20 con 0x7efcf40bb230 2021-01-06T22:14:16.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf27fc700 1 -- 172.21.15.203:0/4170743286 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4109782302 0 0) 0x7efcf414b950 con 0x7efcf4147d60 2021-01-06T22:14:16.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf3fff700 1 --2- 172.21.15.203:0/4170743286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf40bb230 0x7efcf414b460 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7efcec009c90 tx=0x7efcec0091d0).ready entity=mon.1 client_cookie=33e0f57e9ca76a94 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:16.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf27fc700 1 -- 172.21.15.203:0/4170743286 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec00bbc0 con 0x7efcf40bb230 2021-01-06T22:14:16.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf27fc700 1 -- 172.21.15.203:0/4170743286 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec00bd20 con 0x7efcf40bb230 2021-01-06T22:14:16.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf27fc700 1 -- 172.21.15.203:0/4170743286 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec011610 con 0x7efcf40bb230 2021-01-06T22:14:16.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf8d6e700 1 -- 172.21.15.203:0/4170743286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf40bb230 msgr2=0x7efcf414b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:16.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf8d6e700 1 --2- 172.21.15.203:0/4170743286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf40bb230 0x7efcf414b460 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7efcec009c90 tx=0x7efcec0091d0).stop 2021-01-06T22:14:16.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf8d6e700 1 -- 172.21.15.203:0/4170743286 shutdown_connections 2021-01-06T22:14:16.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf8d6e700 1 --2- 172.21.15.203:0/4170743286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf40bb230 0x7efcf414b460 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf8d6e700 1 --2- 172.21.15.203:0/4170743286 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efcf40c1430 0x7efcf4148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf8d6e700 1 -- 172.21.15.203:0/4170743286 >> 172.21.15.203:0/4170743286 conn(0x7efcf40667b0 msgr2=0x7efcf40c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:16.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf8d6e700 1 -- 172.21.15.203:0/4170743286 shutdown_connections 2021-01-06T22:14:16.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf8d6e700 1 -- 172.21.15.203:0/4170743286 wait complete. 2021-01-06T22:14:16.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf8d6e700 1 Processor -- start 2021-01-06T22:14:16.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf8d6e700 1 -- start start 2021-01-06T22:14:16.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf8d6e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efcf40bb230 0x7efcf4060450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:16.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf8d6e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efcf40c1430 0x7efcf415d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:16.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf8d6e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf4147d60 0x7efcf415d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:16.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf8d6e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf415ab60 con 0x7efcf40c1430 2021-01-06T22:14:16.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf8d6e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf4161550 con 0x7efcf4147d60 2021-01-06T22:14:16.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf8d6e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf415db60 con 0x7efcf40bb230 2021-01-06T22:14:16.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf4147d60 0x7efcf415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:16.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf4147d60 0x7efcf415d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51712/0 (socket says 172.21.15.203:51712) 2021-01-06T22:14:16.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf3fff700 1 -- 172.21.15.203:0/957915647 learned_addr learned my addr 172.21.15.203:0/957915647 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:14:16.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf37fe700 1 --2- 172.21.15.203:0/957915647 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efcf40bb230 0x7efcf4060450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:16.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf2ffd700 1 --2- 172.21.15.203:0/957915647 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efcf40c1430 0x7efcf415d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:16.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf3fff700 1 -- 172.21.15.203:0/957915647 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efcf40bb230 msgr2=0x7efcf4060450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:16.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf3fff700 1 --2- 172.21.15.203:0/957915647 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efcf40bb230 0x7efcf4060450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf3fff700 1 -- 172.21.15.203:0/957915647 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efcf40c1430 msgr2=0x7efcf415d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:16.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf3fff700 1 --2- 172.21.15.203:0/957915647 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efcf40c1430 0x7efcf415d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf3fff700 1 -- 172.21.15.203:0/957915647 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf40b86d0 con 0x7efcf4147d60 2021-01-06T22:14:16.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.694+0000 7efcf2ffd700 1 --2- 172.21.15.203:0/957915647 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efcf40c1430 0x7efcf415d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:14:16.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.698+0000 7efcf3fff700 1 --2- 172.21.15.203:0/957915647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf4147d60 0x7efcf415d670 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7efcf405c980 tx=0x7efcec017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:16.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.698+0000 7efcdb7fe700 1 -- 172.21.15.203:0/957915647 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec010800 con 0x7efcf4147d60 2021-01-06T22:14:16.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.698+0000 7efcdb7fe700 1 -- 172.21.15.203:0/957915647 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec010960 con 0x7efcf4147d60 2021-01-06T22:14:16.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.698+0000 7efcf8d6e700 1 -- 172.21.15.203:0/957915647 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf40b8970 con 0x7efcf4147d60 2021-01-06T22:14:16.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.698+0000 7efcdb7fe700 1 -- 172.21.15.203:0/957915647 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec010d20 con 0x7efcf4147d60 2021-01-06T22:14:16.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.698+0000 7efcf8d6e700 1 -- 172.21.15.203:0/957915647 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf40ba380 con 0x7efcf4147d60 2021-01-06T22:14:16.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.698+0000 7efcf8d6e700 1 -- 172.21.15.203:0/957915647 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf40ba540 con 0x7efcf4147d60 2021-01-06T22:14:16.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.698+0000 7efcdb7fe700 1 -- 172.21.15.203:0/957915647 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7efcec02c070 con 0x7efcf4147d60 2021-01-06T22:14:16.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.698+0000 7efcdb7fe700 1 --2- 172.21.15.203:0/957915647 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7efcd40563d0 0x7efcd4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:16.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.698+0000 7efcdb7fe700 1 -- 172.21.15.203:0/957915647 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6589+0+0 (secure 0 0 0) 0x7efcec071de0 con 0x7efcf4147d60 2021-01-06T22:14:16.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.698+0000 7efcf37fe700 1 --2- 172.21.15.203:0/957915647 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7efcd40563d0 0x7efcd4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:16.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.698+0000 7efcf37fe700 1 --2- 172.21.15.203:0/957915647 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7efcd40563d0 0x7efcd4058890 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7efce400c5c0 tx=0x7efce4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:16.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.706+0000 7efcdb7fe700 1 -- 172.21.15.203:0/957915647 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7efcec047460 con 0x7efcf4147d60 2021-01-06T22:14:16.918 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.914+0000 7efcf8d6e700 1 -- 172.21.15.203:0/957915647 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efcf40450a0 con 0x7efcf4147d60 2021-01-06T22:14:16.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.914+0000 7efcdb7fe700 1 -- 172.21.15.203:0/957915647 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3212 (secure 0 0 0) 0x7efcec020030 con 0x7efcf4147d60 2021-01-06T22:14:16.919 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:14:16.919 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:14:16.526363+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:replay","state_seq":2,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:14:16.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.918+0000 7efcf8d6e700 1 -- 172.21.15.203:0/957915647 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7efcd40563d0 msgr2=0x7efcd4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:16.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.918+0000 7efcf8d6e700 1 --2- 172.21.15.203:0/957915647 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7efcd40563d0 0x7efcd4058890 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7efce400c5c0 tx=0x7efce4009040).stop 2021-01-06T22:14:16.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.918+0000 7efcf8d6e700 1 -- 172.21.15.203:0/957915647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf4147d60 msgr2=0x7efcf415d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:16.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.918+0000 7efcf8d6e700 1 --2- 172.21.15.203:0/957915647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf4147d60 0x7efcf415d670 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7efcf405c980 tx=0x7efcec017000).stop 2021-01-06T22:14:16.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.918+0000 7efcf8d6e700 1 -- 172.21.15.203:0/957915647 shutdown_connections 2021-01-06T22:14:16.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.918+0000 7efcf8d6e700 1 --2- 172.21.15.203:0/957915647 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7efcd40563d0 0x7efcd4058890 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.918+0000 7efcf8d6e700 1 --2- 172.21.15.203:0/957915647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf4147d60 0x7efcf415d670 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.918+0000 7efcf8d6e700 1 --2- 172.21.15.203:0/957915647 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efcf40bb230 0x7efcf4060450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.918+0000 7efcf8d6e700 1 --2- 172.21.15.203:0/957915647 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efcf40c1430 0x7efcf415d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:16.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.918+0000 7efcf8d6e700 1 -- 172.21.15.203:0/957915647 >> 172.21.15.203:0/957915647 conn(0x7efcf40667b0 msgr2=0x7efcf40c0820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:16.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.918+0000 7efcf8d6e700 1 -- 172.21.15.203:0/957915647 shutdown_connections 2021-01-06T22:14:16.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:16.918+0000 7efcf8d6e700 1 -- 172.21.15.203:0/957915647 wait complete. 2021-01-06T22:14:16.924 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 51 2021-01-06T22:14:16.934 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 51, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 51, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5068": { "addr": "172.21.15.203:6835/1477297469", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6834", "nonce": 1477297469, "type": "v2" }, { "addr": "172.21.15.203:6835", "nonce": 1477297469, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5068, "incarnation": 33, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 9 }, "gid_5350": { "addr": "172.21.15.203:6837/4277691165", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6836", "nonce": 4277691165, "type": "v2" }, { "addr": "172.21.15.203:6837", "nonce": 4277691165, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5350, "incarnation": 51, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 2 } }, "last_failure": 0, "last_failure_osd_epoch": 29, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2021-01-06T22:14:16.526363+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 5350, "mds_1": 5068 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.91:6836/2428882934", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6834", "nonce": 2428882934, "type": "v2" }, { "addr": "172.21.15.91:6836", "nonce": 2428882934, "type": "v1" } ] }, "epoch": 26, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4724, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2021-01-06T22:14:17.359 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:14:17.362 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:14:18.935 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:14:19.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa02d397700 1 Processor -- start 2021-01-06T22:14:19.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa02d397700 1 -- start start 2021-01-06T22:14:19.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa02d397700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa0280c04c0 0x7fa0280680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:19.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa02d397700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa028155b60 0x7fa028156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:19.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa02d397700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0281531f0 con 0x7fa0280c08a0 2021-01-06T22:14:19.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa02d397700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0281522e0 con 0x7fa0280c04c0 2021-01-06T22:14:19.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa02d397700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0280645f0 con 0x7fa028155b60 2021-01-06T22:14:19.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa0267fc700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fa0280c08a0 0x7fa028063ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50314/0 (socket says 172.21.15.203:50314) 2021-01-06T22:14:19.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa0267fc700 1 -- 172.21.15.203:0/3795493319 learned_addr learned my addr 172.21.15.203:0/3795493319 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:14:19.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa0277fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa028155b60 0x7fa028156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:19.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa026ffd700 1 --2- 172.21.15.203:0/3795493319 >> v2:172.21.15.91:3300/0 conn(0x7fa0280c04c0 0x7fa0280680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:19.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa025ffb700 1 -- 172.21.15.203:0/3795493319 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641810394 0 0) 0x7fa0281531f0 con 0x7fa0280c08a0 2021-01-06T22:14:19.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa025ffb700 1 -- 172.21.15.203:0/3795493319 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa008003230 con 0x7fa0280c08a0 2021-01-06T22:14:19.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa0277fe700 1 -- 172.21.15.203:0/3795493319 >> v2:172.21.15.91:3300/0 conn(0x7fa0280c04c0 msgr2=0x7fa0280680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:19.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa0277fe700 1 --2- 172.21.15.203:0/3795493319 >> v2:172.21.15.91:3300/0 conn(0x7fa0280c04c0 0x7fa0280680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:19.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa0277fe700 1 -- 172.21.15.203:0/3795493319 >> v1:172.21.15.203:6789/0 conn(0x7fa0280c08a0 legacy=0x7fa028063ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:19.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa025ffb700 1 -- 172.21.15.203:0/3795493319 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3770642526 0 0) 0x7fa008003230 con 0x7fa0280c08a0 2021-01-06T22:14:19.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa0277fe700 1 -- 172.21.15.203:0/3795493319 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa028156500 con 0x7fa028155b60 2021-01-06T22:14:19.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa0277fe700 1 --2- 172.21.15.203:0/3795493319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa028155b60 0x7fa028156010 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7fa028062090 tx=0x7fa020006de0).ready entity=mon.1 client_cookie=6e546be3979e53ad server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:19.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa025ffb700 1 -- 172.21.15.203:0/3795493319 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0100014e0 con 0x7fa028155b60 2021-01-06T22:14:19.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa025ffb700 1 -- 172.21.15.203:0/3795493319 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa020011400 con 0x7fa028155b60 2021-01-06T22:14:19.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa025ffb700 1 -- 172.21.15.203:0/3795493319 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa020011780 con 0x7fa028155b60 2021-01-06T22:14:19.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa02d397700 1 -- 172.21.15.203:0/3795493319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa028155b60 msgr2=0x7fa028156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:19.080 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa02d397700 1 --2- 172.21.15.203:0/3795493319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa028155b60 0x7fa028156010 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7fa028062090 tx=0x7fa020006de0).stop 2021-01-06T22:14:19.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa02d397700 1 -- 172.21.15.203:0/3795493319 shutdown_connections 2021-01-06T22:14:19.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa02d397700 1 --2- 172.21.15.203:0/3795493319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa028155b60 0x7fa028156010 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:19.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa02d397700 1 --2- 172.21.15.203:0/3795493319 >> v2:172.21.15.91:3300/0 conn(0x7fa0280c04c0 0x7fa0280680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:19.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa02d397700 1 -- 172.21.15.203:0/3795493319 >> 172.21.15.203:0/3795493319 conn(0x7fa028056cd0 msgr2=0x7fa0280626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:19.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa02d397700 1 -- 172.21.15.203:0/3795493319 shutdown_connections 2021-01-06T22:14:19.083 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.066+0000 7fa02d397700 1 -- 172.21.15.203:0/3795493319 wait complete. 2021-01-06T22:14:19.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa02d397700 1 Processor -- start 2021-01-06T22:14:19.088 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa02d397700 1 -- start start 2021-01-06T22:14:19.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa02d397700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0280c04c0 0x7fa0280b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:19.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa02d397700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0280c08a0 0x7fa0280b90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:19.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa02d397700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa028155b60 0x7fa02814c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:19.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa02d397700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0281626e0 con 0x7fa0280c08a0 2021-01-06T22:14:19.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa02d397700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0280b0530 con 0x7fa0280c04c0 2021-01-06T22:14:19.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa02d397700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0280b07d0 con 0x7fa028155b60 2021-01-06T22:14:19.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa026ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0280c04c0 0x7fa0280b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:19.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa026ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0280c04c0 0x7fa0280b8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51726/0 (socket says 172.21.15.203:51726) 2021-01-06T22:14:19.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa026ffd700 1 -- 172.21.15.203:0/1890714374 learned_addr learned my addr 172.21.15.203:0/1890714374 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:14:19.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa0267fc700 1 --2- 172.21.15.203:0/1890714374 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0280c08a0 0x7fa0280b90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:19.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa0277fe700 1 --2- 172.21.15.203:0/1890714374 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa028155b60 0x7fa02814c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:19.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa026ffd700 1 -- 172.21.15.203:0/1890714374 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa028155b60 msgr2=0x7fa02814c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:19.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa026ffd700 1 --2- 172.21.15.203:0/1890714374 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa028155b60 0x7fa02814c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:19.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa026ffd700 1 -- 172.21.15.203:0/1890714374 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0280c08a0 msgr2=0x7fa0280b90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:19.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa026ffd700 1 --2- 172.21.15.203:0/1890714374 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0280c08a0 0x7fa0280b90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:19.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa026ffd700 1 -- 172.21.15.203:0/1890714374 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0280b0a70 con 0x7fa0280c04c0 2021-01-06T22:14:19.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa0277fe700 1 --2- 172.21.15.203:0/1890714374 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa028155b60 0x7fa02814c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:14:19.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa026ffd700 1 --2- 172.21.15.203:0/1890714374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0280c04c0 0x7fa0280b8b20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fa01800eea0 tx=0x7fa0180091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:19.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa01effd700 1 -- 172.21.15.203:0/1890714374 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0180096e0 con 0x7fa0280c04c0 2021-01-06T22:14:19.096 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa01effd700 1 -- 172.21.15.203:0/1890714374 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01800cc70 con 0x7fa0280c04c0 2021-01-06T22:14:19.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa01effd700 1 -- 172.21.15.203:0/1890714374 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0180123f0 con 0x7fa0280c04c0 2021-01-06T22:14:19.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.070+0000 7fa02d397700 1 -- 172.21.15.203:0/1890714374 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0280b7900 con 0x7fa0280c04c0 2021-01-06T22:14:19.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.074+0000 7fa02d397700 1 -- 172.21.15.203:0/1890714374 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0280bf170 con 0x7fa0280c04c0 2021-01-06T22:14:19.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.074+0000 7fa01effd700 1 -- 172.21.15.203:0/1890714374 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa018022070 con 0x7fa0280c04c0 2021-01-06T22:14:19.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.074+0000 7fa02d397700 1 -- 172.21.15.203:0/1890714374 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02815f750 con 0x7fa0280c04c0 2021-01-06T22:14:19.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.074+0000 7fa01effd700 1 --2- 172.21.15.203:0/1890714374 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa00c0563d0 0x7fa00c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:19.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.074+0000 7fa01effd700 1 -- 172.21.15.203:0/1890714374 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6589+0+0 (secure 0 0 0) 0x7fa01806ffd0 con 0x7fa0280c04c0 2021-01-06T22:14:19.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.074+0000 7fa0267fc700 1 --2- 172.21.15.203:0/1890714374 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa00c0563d0 0x7fa00c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:19.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.074+0000 7fa0267fc700 1 --2- 172.21.15.203:0/1890714374 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa00c0563d0 0x7fa00c058890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fa0280678e0 tx=0x7fa010001e10).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:19.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.078+0000 7fa01effd700 1 -- 172.21.15.203:0/1890714374 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa018045650 con 0x7fa0280c04c0 2021-01-06T22:14:19.287 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.282+0000 7fa02d397700 1 -- 172.21.15.203:0/1890714374 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa02805cfe0 con 0x7fa0280c04c0 2021-01-06T22:14:19.288 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.286+0000 7fa01effd700 1 -- 172.21.15.203:0/1890714374 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+3216 (secure 0 0 0) 0x7fa018016070 con 0x7fa0280c04c0 2021-01-06T22:14:19.288 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:14:19.289 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:14:18.546874+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:reconnect","state_seq":35,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:14:19.291 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.286+0000 7fa02d397700 1 -- 172.21.15.203:0/1890714374 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa00c0563d0 msgr2=0x7fa00c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:19.292 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.286+0000 7fa02d397700 1 --2- 172.21.15.203:0/1890714374 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa00c0563d0 0x7fa00c058890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fa0280678e0 tx=0x7fa010001e10).stop 2021-01-06T22:14:19.292 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.286+0000 7fa02d397700 1 -- 172.21.15.203:0/1890714374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0280c04c0 msgr2=0x7fa0280b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:19.292 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.286+0000 7fa02d397700 1 --2- 172.21.15.203:0/1890714374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0280c04c0 0x7fa0280b8b20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fa01800eea0 tx=0x7fa0180091d0).stop 2021-01-06T22:14:19.292 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.286+0000 7fa02d397700 1 -- 172.21.15.203:0/1890714374 shutdown_connections 2021-01-06T22:14:19.293 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.286+0000 7fa02d397700 1 --2- 172.21.15.203:0/1890714374 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa00c0563d0 0x7fa00c058890 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:19.293 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.286+0000 7fa02d397700 1 --2- 172.21.15.203:0/1890714374 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa028155b60 0x7fa02814c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:19.293 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.286+0000 7fa02d397700 1 --2- 172.21.15.203:0/1890714374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0280c04c0 0x7fa0280b8b20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:19.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.286+0000 7fa02d397700 1 --2- 172.21.15.203:0/1890714374 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0280c08a0 0x7fa0280b90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:19.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.286+0000 7fa02d397700 1 -- 172.21.15.203:0/1890714374 >> 172.21.15.203:0/1890714374 conn(0x7fa028056cd0 msgr2=0x7fa028150280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:19.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.290+0000 7fa02d397700 1 -- 172.21.15.203:0/1890714374 shutdown_connections 2021-01-06T22:14:19.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:19.290+0000 7fa02d397700 1 -- 172.21.15.203:0/1890714374 wait complete. 2021-01-06T22:14:19.295 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 53 2021-01-06T22:14:19.307 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:14:21.309 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:14:21.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.438+0000 7fa9d2a13700 1 Processor -- start 2021-01-06T22:14:21.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.438+0000 7fa9d2a13700 1 -- start start 2021-01-06T22:14:21.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d2a13700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa9cc0bf100 0x7fa9cc0bf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:21.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d2a13700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9cc151560 0x7fa9cc147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:21.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d2a13700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9cc152460 con 0x7fa9cc064bc0 2021-01-06T22:14:21.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d2a13700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9cc14e740 con 0x7fa9cc0bf100 2021-01-06T22:14:21.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d2a13700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc14e9e0 con 0x7fa9cc151560 2021-01-06T22:14:21.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d0fb0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9cc151560 0x7fa9cc147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:21.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9cbfff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa9cc0bf100 0x7fa9cc0bf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:21.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9cbfff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa9cc0bf100 0x7fa9cc0bf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:59588/0 (socket says 172.21.15.203:59588) 2021-01-06T22:14:21.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9cbfff700 1 -- 172.21.15.203:0/3945572012 learned_addr learned my addr 172.21.15.203:0/3945572012 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:14:21.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d0fb0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9cc151560 0x7fa9cc147bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:56826/0 (socket says 172.21.15.203:56826) 2021-01-06T22:14:21.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d0fb0700 1 -- 172.21.15.203:0/3945572012 >> v2:172.21.15.91:3300/0 conn(0x7fa9cc0bf100 msgr2=0x7fa9cc0bf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:21.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d0fb0700 1 --2- 172.21.15.203:0/3945572012 >> v2:172.21.15.91:3300/0 conn(0x7fa9cc0bf100 0x7fa9cc0bf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:21.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d0fb0700 1 -- 172.21.15.203:0/3945572012 >> v1:172.21.15.91:6789/0 conn(0x7fa9cc064bc0 legacy=0x7fa9cc14df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:21.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d0fb0700 1 -- 172.21.15.203:0/3945572012 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc14ec80 con 0x7fa9cc151560 2021-01-06T22:14:21.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9cbfff700 1 --2- 172.21.15.203:0/3945572012 >> v2:172.21.15.91:3300/0 conn(0x7fa9cc0bf100 0x7fa9cc0bf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:14:21.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d0fb0700 1 --2- 172.21.15.203:0/3945572012 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9cc151560 0x7fa9cc147bf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fa9c4009c90 tx=0x7fa9c40091d0).ready entity=mon.2 client_cookie=9a8e8e90a02bd2ff server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:21.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9caffd700 1 -- 172.21.15.203:0/3945572012 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400ba70 con 0x7fa9cc151560 2021-01-06T22:14:21.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9caffd700 1 -- 172.21.15.203:0/3945572012 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c400bbd0 con 0x7fa9cc151560 2021-01-06T22:14:21.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9caffd700 1 -- 172.21.15.203:0/3945572012 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4011400 con 0x7fa9cc151560 2021-01-06T22:14:21.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d2a13700 1 -- 172.21.15.203:0/3945572012 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9cc151560 msgr2=0x7fa9cc147bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:21.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d2a13700 1 --2- 172.21.15.203:0/3945572012 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9cc151560 0x7fa9cc147bf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fa9c4009c90 tx=0x7fa9c40091d0).stop 2021-01-06T22:14:21.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d2a13700 1 -- 172.21.15.203:0/3945572012 shutdown_connections 2021-01-06T22:14:21.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d2a13700 1 --2- 172.21.15.203:0/3945572012 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9cc151560 0x7fa9cc147bf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:21.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d2a13700 1 --2- 172.21.15.203:0/3945572012 >> v2:172.21.15.91:3300/0 conn(0x7fa9cc0bf100 0x7fa9cc0bf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:21.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d2a13700 1 -- 172.21.15.203:0/3945572012 >> 172.21.15.203:0/3945572012 conn(0x7fa9cc0602b0 msgr2=0x7fa9cc0606c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:21.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d2a13700 1 -- 172.21.15.203:0/3945572012 shutdown_connections 2021-01-06T22:14:21.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d2a13700 1 -- 172.21.15.203:0/3945572012 wait complete. 2021-01-06T22:14:21.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d2a13700 1 Processor -- start 2021-01-06T22:14:21.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.442+0000 7fa9d2a13700 1 -- start start 2021-01-06T22:14:21.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9d2a13700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa9cc064bc0 0x7fa9cc15f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:21.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9d2a13700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9cc0bf100 0x7fa9cc15fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:21.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9d2a13700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9cc151560 0x7fa9cc147a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:21.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9d2a13700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc15fff0 con 0x7fa9cc064bc0 2021-01-06T22:14:21.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9d2a13700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc161580 con 0x7fa9cc0bf100 2021-01-06T22:14:21.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9d2a13700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc161820 con 0x7fa9cc151560 2021-01-06T22:14:21.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9cb7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9cc0bf100 0x7fa9cc15fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:21.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9d0fb0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9cc151560 0x7fa9cc147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:21.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9cb7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9cc0bf100 0x7fa9cc15fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51748/0 (socket says 172.21.15.203:51748) 2021-01-06T22:14:21.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9d0fb0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9cc151560 0x7fa9cc147a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:56832/0 (socket says 172.21.15.203:56832) 2021-01-06T22:14:21.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9cb7fe700 1 -- 172.21.15.203:0/652194309 learned_addr learned my addr 172.21.15.203:0/652194309 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:14:21.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9cbfff700 1 --2- 172.21.15.203:0/652194309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa9cc064bc0 0x7fa9cc15f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:21.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9cb7fe700 1 -- 172.21.15.203:0/652194309 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9cc151560 msgr2=0x7fa9cc147a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:21.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9cb7fe700 1 --2- 172.21.15.203:0/652194309 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9cc151560 0x7fa9cc147a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:21.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9cb7fe700 1 -- 172.21.15.203:0/652194309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa9cc064bc0 msgr2=0x7fa9cc15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:21.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9cb7fe700 1 --2- 172.21.15.203:0/652194309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa9cc064bc0 0x7fa9cc15f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:21.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9cb7fe700 1 -- 172.21.15.203:0/652194309 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc15ec30 con 0x7fa9cc0bf100 2021-01-06T22:14:21.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9cb7fe700 1 --2- 172.21.15.203:0/652194309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9cc0bf100 0x7fa9cc15fb00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fa9c000c740 tx=0x7fa9c0007be0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:21.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9b3fff700 1 -- 172.21.15.203:0/652194309 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0007020 con 0x7fa9cc0bf100 2021-01-06T22:14:21.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9b3fff700 1 -- 172.21.15.203:0/652194309 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c0007180 con 0x7fa9cc0bf100 2021-01-06T22:14:21.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9d2a13700 1 -- 172.21.15.203:0/652194309 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9cc15eed0 con 0x7fa9cc0bf100 2021-01-06T22:14:21.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9b3fff700 1 -- 172.21.15.203:0/652194309 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c00064e0 con 0x7fa9cc0bf100 2021-01-06T22:14:21.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.446+0000 7fa9d2a13700 1 -- 172.21.15.203:0/652194309 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9cc0bfad0 con 0x7fa9cc0bf100 2021-01-06T22:14:21.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.450+0000 7fa9b3fff700 1 -- 172.21.15.203:0/652194309 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa9c001a060 con 0x7fa9cc0bf100 2021-01-06T22:14:21.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.450+0000 7fa9d2a13700 1 -- 172.21.15.203:0/652194309 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9cc14ee60 con 0x7fa9cc0bf100 2021-01-06T22:14:21.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.450+0000 7fa9b3fff700 1 --2- 172.21.15.203:0/652194309 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa9ac0563d0 0x7fa9ac058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:21.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.450+0000 7fa9b3fff700 1 -- 172.21.15.203:0/652194309 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6589+0+0 (secure 0 0 0) 0x7fa9c006ef60 con 0x7fa9cc0bf100 2021-01-06T22:14:21.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.450+0000 7fa9cbfff700 1 --2- 172.21.15.203:0/652194309 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa9ac0563d0 0x7fa9ac058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:21.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.450+0000 7fa9cbfff700 1 --2- 172.21.15.203:0/652194309 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa9ac0563d0 0x7fa9ac058890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fa9bc0069e0 tx=0x7fa9bc006ea0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:21.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.454+0000 7fa9b3fff700 1 -- 172.21.15.203:0/652194309 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa9c00445e0 con 0x7fa9cc0bf100 2021-01-06T22:14:21.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.678+0000 7fa9d2a13700 1 -- 172.21.15.203:0/652194309 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9cc05d600 con 0x7fa9cc0bf100 2021-01-06T22:14:21.684 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.682+0000 7fa9b3fff700 1 -- 172.21.15.203:0/652194309 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+3213 (secure 0 0 0) 0x7fa9c0018030 con 0x7fa9cc0bf100 2021-01-06T22:14:21.684 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:14:21.685 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:14:20.559082+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:14:21.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.682+0000 7fa9d2a13700 1 -- 172.21.15.203:0/652194309 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa9ac0563d0 msgr2=0x7fa9ac058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:21.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.682+0000 7fa9d2a13700 1 --2- 172.21.15.203:0/652194309 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa9ac0563d0 0x7fa9ac058890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fa9bc0069e0 tx=0x7fa9bc006ea0).stop 2021-01-06T22:14:21.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.682+0000 7fa9d2a13700 1 -- 172.21.15.203:0/652194309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9cc0bf100 msgr2=0x7fa9cc15fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:21.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.682+0000 7fa9d2a13700 1 --2- 172.21.15.203:0/652194309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9cc0bf100 0x7fa9cc15fb00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fa9c000c740 tx=0x7fa9c0007be0).stop 2021-01-06T22:14:21.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.682+0000 7fa9d2a13700 1 -- 172.21.15.203:0/652194309 shutdown_connections 2021-01-06T22:14:21.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.682+0000 7fa9d2a13700 1 --2- 172.21.15.203:0/652194309 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa9ac0563d0 0x7fa9ac058890 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:21.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.682+0000 7fa9d2a13700 1 --2- 172.21.15.203:0/652194309 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9cc151560 0x7fa9cc147a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:21.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.682+0000 7fa9d2a13700 1 --2- 172.21.15.203:0/652194309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa9cc064bc0 0x7fa9cc15f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:21.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.682+0000 7fa9d2a13700 1 --2- 172.21.15.203:0/652194309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9cc0bf100 0x7fa9cc15fb00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:21.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.682+0000 7fa9d2a13700 1 -- 172.21.15.203:0/652194309 >> 172.21.15.203:0/652194309 conn(0x7fa9cc0602b0 msgr2=0x7fa9cc14fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:21.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.682+0000 7fa9d2a13700 1 -- 172.21.15.203:0/652194309 shutdown_connections 2021-01-06T22:14:21.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:21.682+0000 7fa9d2a13700 1 -- 172.21.15.203:0/652194309 wait complete. 2021-01-06T22:14:21.690 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 55 2021-01-06T22:14:21.701 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=4925 2021-01-06T22:14:21.701 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.f 2021-01-06T22:14:22.844 INFO:tasks.ceph.mds.a.smithi203.stderr:2021-01-06T22:14:22.842+0000 7f8e8c4a9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2021-01-06T22:14:22.844 INFO:tasks.ceph.mds.a.smithi203.stderr:2021-01-06T22:14:22.842+0000 7f8e8c4a9700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2021-01-06T22:14:26.701 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2021-01-06T22:14:26.702 INFO:tasks.ceph.mds.f:Restarting daemon 2021-01-06T22:14:26.702 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2021-01-06T22:14:26.708 INFO:tasks.ceph.mds.f:Started 2021-01-06T22:14:26.710 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:14:26.786 INFO:tasks.ceph.mds.f.smithi091.stdout:starting mds.f at 2021-01-06T22:14:26.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.850+0000 7fa2e9439700 1 Processor -- start 2021-01-06T22:14:26.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.850+0000 7fa2e9439700 1 -- start start 2021-01-06T22:14:26.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e9439700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa2e40c44f0 0x7fa2e40c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:26.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e9439700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2e4060fc0 0x7fa2e4150e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:26.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e9439700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e40c0f50 con 0x7fa2e40c48d0 2021-01-06T22:14:26.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e9439700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e4066cf0 con 0x7fa2e40c44f0 2021-01-06T22:14:26.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e9439700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e4060750 con 0x7fa2e4060fc0 2021-01-06T22:14:26.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e3fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa2e40c44f0 0x7fa2e40c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:26.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e3fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa2e40c44f0 0x7fa2e40c1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51754/0 (socket says 172.21.15.203:51754) 2021-01-06T22:14:26.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e3fff700 1 -- 172.21.15.203:0/3403662598 learned_addr learned my addr 172.21.15.203:0/3403662598 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:14:26.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e8c38700 1 --2- 172.21.15.203:0/3403662598 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2e4060fc0 0x7fa2e4150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:26.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e3fff700 1 -- 172.21.15.203:0/3403662598 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2e4060fc0 msgr2=0x7fa2e4150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:26.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e3fff700 1 --2- 172.21.15.203:0/3403662598 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2e4060fc0 0x7fa2e4150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:26.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e3fff700 1 -- 172.21.15.203:0/3403662598 >> v1:172.21.15.91:6789/0 conn(0x7fa2e40c48d0 legacy=0x7fa2e4150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:26.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e3fff700 1 -- 172.21.15.203:0/3403662598 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e40609f0 con 0x7fa2e40c44f0 2021-01-06T22:14:26.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e3fff700 1 -- 172.21.15.203:0/3403662598 >> v2:172.21.15.203:3300/0 conn(0x7fa2e40c44f0 msgr2=0x7fa2e40c1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2021-01-06T22:14:26.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e3fff700 1 -- 172.21.15.203:0/3403662598 >> v2:172.21.15.203:3300/0 conn(0x7fa2e40c44f0 msgr2=0x7fa2e40c1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:14:26.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e3fff700 1 --2- 172.21.15.203:0/3403662598 >> v2:172.21.15.203:3300/0 conn(0x7fa2e40c44f0 0x7fa2e40c1380 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0x7fa2d400b9e0 tx=0x7fa2d4006ea0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:14:26.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:26.854+0000 7fa2e3fff700 1 --2- 172.21.15.203:0/3403662598 >> v2:172.21.15.203:3300/0 conn(0x7fa2e40c44f0 0x7fa2e40c1380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-01-06T22:14:27.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e3fff700 1 --2- 172.21.15.203:0/3403662598 >> v2:172.21.15.203:3300/0 conn(0x7fa2e40c44f0 0x7fa2e40c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:27.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e3fff700 0 --2- 172.21.15.203:0/3403662598 >> v2:172.21.15.203:3300/0 conn(0x7fa2e40c44f0 0x7fa2e40c1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2021-01-06T22:14:27.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e3fff700 1 --2- 172.21.15.203:0/3403662598 >> v2:172.21.15.203:3300/0 conn(0x7fa2e40c44f0 0x7fa2e40c1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:27.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e37fe700 1 -- 172.21.15.203:0/3403662598 >> v2:172.21.15.203:3300/0 conn(0x7fa2e40c44f0 msgr2=0x7fa2e40c1380 unknown :-1 s=STATE_CLOSED l=0).mark_down 2021-01-06T22:14:27.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e37fe700 1 --2- 172.21.15.203:0/3403662598 >> v2:172.21.15.203:3300/0 conn(0x7fa2e40c44f0 0x7fa2e40c1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:27.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e37fe700 1 --2- 172.21.15.203:0/3403662598 >> v2:172.21.15.91:3300/0 conn(0x7fa2c4003210 0x7fa2c4005690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:27.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e37fe700 1 -- 172.21.15.203:0/3403662598 --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c4009b80 con 0x7fa2c4005dc0 2021-01-06T22:14:27.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e37fe700 1 -- 172.21.15.203:0/3403662598 --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c400ad80 con 0x7fa2c4003210 2021-01-06T22:14:27.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e3fff700 1 --2- 172.21.15.203:0/3403662598 >> v2:172.21.15.91:3300/0 conn(0x7fa2c4003210 0x7fa2c4005690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:27.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e37fe700 1 -- 172.21.15.203:0/3403662598 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337284441 0 0) 0x7fa2c4009b80 con 0x7fa2c4005dc0 2021-01-06T22:14:27.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e37fe700 1 -- 172.21.15.203:0/3403662598 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c4009df0 con 0x7fa2c4005dc0 2021-01-06T22:14:27.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e3fff700 1 -- 172.21.15.203:0/3403662598 >> v1:172.21.15.203:6789/0 conn(0x7fa2c4005dc0 legacy=0x7fa2c4008280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:27.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e3fff700 1 -- 172.21.15.203:0/3403662598 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c400afb0 con 0x7fa2c4003210 2021-01-06T22:14:27.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e3fff700 1 --2- 172.21.15.203:0/3403662598 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2c4003210 0x7fa2c4005690 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fa2e40c1f40 tx=0x7fa2d4006ea0).ready entity=mon.0 client_cookie=fb175c3788f4e23c server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:27.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e37fe700 1 -- 172.21.15.203:0/3403662598 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4018070 con 0x7fa2c4003210 2021-01-06T22:14:27.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e37fe700 1 -- 172.21.15.203:0/3403662598 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d4009ea0 con 0x7fa2c4003210 2021-01-06T22:14:27.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.058+0000 7fa2e37fe700 1 -- 172.21.15.203:0/3403662598 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4012650 con 0x7fa2c4003210 2021-01-06T22:14:27.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.062+0000 7fa2e9439700 1 -- 172.21.15.203:0/3403662598 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2c4003210 msgr2=0x7fa2c4005690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:27.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.062+0000 7fa2e9439700 1 --2- 172.21.15.203:0/3403662598 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2c4003210 0x7fa2c4005690 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fa2e40c1f40 tx=0x7fa2d4006ea0).stop 2021-01-06T22:14:27.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.062+0000 7fa2e8c38700 1 -- 172.21.15.203:0/3403662598 reap_dead start 2021-01-06T22:14:27.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.062+0000 7fa2e9439700 1 -- 172.21.15.203:0/3403662598 shutdown_connections 2021-01-06T22:14:27.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.062+0000 7fa2e9439700 1 --2- 172.21.15.203:0/3403662598 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2c4003210 0x7fa2c4005690 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:27.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.062+0000 7fa2e9439700 1 -- 172.21.15.203:0/3403662598 >> 172.21.15.203:0/3403662598 conn(0x7fa2e4056cd0 msgr2=0x7fa2e4061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:27.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.062+0000 7fa2e9439700 1 -- 172.21.15.203:0/3403662598 shutdown_connections 2021-01-06T22:14:27.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.062+0000 7fa2e9439700 1 -- 172.21.15.203:0/3403662598 wait complete. 2021-01-06T22:14:27.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.062+0000 7fa2e9439700 1 Processor -- start 2021-01-06T22:14:27.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.062+0000 7fa2e9439700 1 -- start start 2021-01-06T22:14:27.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e9439700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2c4003210 0x7fa2e4161a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:27.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e9439700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2e414d9e0 0x7fa2e40ba040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:27.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e9439700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2e414de60 0x7fa2e40ba530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:27.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e9439700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e4161f30 con 0x7fa2e414d9e0 2021-01-06T22:14:27.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e9439700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e40baa20 con 0x7fa2c4003210 2021-01-06T22:14:27.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e9439700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e40bc8e0 con 0x7fa2e414de60 2021-01-06T22:14:27.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2c4003210 0x7fa2e4161a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:27.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2c4003210 0x7fa2e4161a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51766/0 (socket says 172.21.15.203:51766) 2021-01-06T22:14:27.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e3fff700 1 -- 172.21.15.203:0/56650282 learned_addr learned my addr 172.21.15.203:0/56650282 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:14:27.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2db7fe700 1 --2- 172.21.15.203:0/56650282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2e414d9e0 0x7fa2e40ba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:27.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e8c38700 1 --2- 172.21.15.203:0/56650282 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2e414de60 0x7fa2e40ba530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:27.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e3fff700 1 -- 172.21.15.203:0/56650282 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2e414de60 msgr2=0x7fa2e40ba530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:27.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e3fff700 1 --2- 172.21.15.203:0/56650282 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2e414de60 0x7fa2e40ba530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:27.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e3fff700 1 -- 172.21.15.203:0/56650282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2e414d9e0 msgr2=0x7fa2e40ba040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:27.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e3fff700 1 --2- 172.21.15.203:0/56650282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2e414d9e0 0x7fa2e40ba040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:27.078 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e3fff700 1 -- 172.21.15.203:0/56650282 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e4153cf0 con 0x7fa2c4003210 2021-01-06T22:14:27.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e3fff700 1 --2- 172.21.15.203:0/56650282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2c4003210 0x7fa2e4161a40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fa2d4007fd0 tx=0x7fa2d4014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:27.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e0ff9700 1 -- 172.21.15.203:0/56650282 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4028440 con 0x7fa2c4003210 2021-01-06T22:14:27.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e0ff9700 1 -- 172.21.15.203:0/56650282 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d4023040 con 0x7fa2c4003210 2021-01-06T22:14:27.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e9439700 1 -- 172.21.15.203:0/56650282 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e4153f90 con 0x7fa2c4003210 2021-01-06T22:14:27.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e0ff9700 1 -- 172.21.15.203:0/56650282 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40287f0 con 0x7fa2c4003210 2021-01-06T22:14:27.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.066+0000 7fa2e9439700 1 -- 172.21.15.203:0/56650282 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e40b9900 con 0x7fa2c4003210 2021-01-06T22:14:27.083 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.070+0000 7fa2e0ff9700 1 -- 172.21.15.203:0/56650282 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa2d400f070 con 0x7fa2c4003210 2021-01-06T22:14:27.083 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.070+0000 7fa2e9439700 1 -- 172.21.15.203:0/56650282 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e40450a0 con 0x7fa2c4003210 2021-01-06T22:14:27.083 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.070+0000 7fa2e0ff9700 1 --2- 172.21.15.203:0/56650282 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa2c80563d0 0x7fa2c8058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:27.084 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.070+0000 7fa2e0ff9700 1 -- 172.21.15.203:0/56650282 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6589+0+0 (secure 0 0 0) 0x7fa2d40715c0 con 0x7fa2c4003210 2021-01-06T22:14:27.084 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.070+0000 7fa2db7fe700 1 --2- 172.21.15.203:0/56650282 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa2c80563d0 0x7fa2c8058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:27.084 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.070+0000 7fa2db7fe700 1 --2- 172.21.15.203:0/56650282 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa2c80563d0 0x7fa2c8058890 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fa2e40b8020 tx=0x7fa2cc008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:27.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.074+0000 7fa2e0ff9700 1 -- 172.21.15.203:0/56650282 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa2d404a720 con 0x7fa2c4003210 2021-01-06T22:14:27.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.298+0000 7fa2e9439700 1 -- 172.21.15.203:0/56650282 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2e41503f0 con 0x7fa2c4003210 2021-01-06T22:14:27.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.298+0000 7fa2e0ff9700 1 -- 172.21.15.203:0/56650282 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+3213 (secure 0 0 0) 0x7fa2d404a310 con 0x7fa2c4003210 2021-01-06T22:14:27.303 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:14:27.303 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:14:20.559082+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:14:27.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.302+0000 7fa2e9439700 1 -- 172.21.15.203:0/56650282 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa2c80563d0 msgr2=0x7fa2c8058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:27.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.302+0000 7fa2e9439700 1 --2- 172.21.15.203:0/56650282 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa2c80563d0 0x7fa2c8058890 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fa2e40b8020 tx=0x7fa2cc008040).stop 2021-01-06T22:14:27.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.302+0000 7fa2e9439700 1 -- 172.21.15.203:0/56650282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2c4003210 msgr2=0x7fa2e4161a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:27.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.302+0000 7fa2e9439700 1 --2- 172.21.15.203:0/56650282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2c4003210 0x7fa2e4161a40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fa2d4007fd0 tx=0x7fa2d4014000).stop 2021-01-06T22:14:27.305 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.302+0000 7fa2e9439700 1 -- 172.21.15.203:0/56650282 shutdown_connections 2021-01-06T22:14:27.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.302+0000 7fa2e9439700 1 --2- 172.21.15.203:0/56650282 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa2c80563d0 0x7fa2c8058890 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:27.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.302+0000 7fa2e9439700 1 --2- 172.21.15.203:0/56650282 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2e414de60 0x7fa2e40ba530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:27.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.302+0000 7fa2e9439700 1 --2- 172.21.15.203:0/56650282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2c4003210 0x7fa2e4161a40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:27.306 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.302+0000 7fa2e9439700 1 --2- 172.21.15.203:0/56650282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2e414d9e0 0x7fa2e40ba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:27.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.302+0000 7fa2e9439700 1 -- 172.21.15.203:0/56650282 >> 172.21.15.203:0/56650282 conn(0x7fa2e4056cd0 msgr2=0x7fa2e4067f10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:27.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.302+0000 7fa2e9439700 1 -- 172.21.15.203:0/56650282 shutdown_connections 2021-01-06T22:14:27.307 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:27.302+0000 7fa2e9439700 1 -- 172.21.15.203:0/56650282 wait complete. 2021-01-06T22:14:27.308 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 55 2021-01-06T22:14:27.323 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2021-01-06T22:14:29.324 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:14:29.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2dbbfff700 1 Processor -- start 2021-01-06T22:14:29.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2dbbfff700 1 -- start start 2021-01-06T22:14:29.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2dbbfff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2dbc14e310 0x7f2dbc1471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:29.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2dbbfff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dbc0c16b0 con 0x7f2dbc064380 2021-01-06T22:14:29.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2dbbfff700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dbc0bb2d0 con 0x7f2dbc064760 2021-01-06T22:14:29.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2dbbfff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc14a560 con 0x7f2dbc14e310 2021-01-06T22:14:29.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2dbb7fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f2dbc064760 0x7f2dbc0bb730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50372/0 (socket says 172.21.15.203:50372) 2021-01-06T22:14:29.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2dbb7fe700 1 -- 172.21.15.203:0/405754737 learned_addr learned my addr 172.21.15.203:0/405754737 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:14:29.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2dba7fc700 1 --2- 172.21.15.203:0/405754737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2dbc14e310 0x7f2dbc1471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:29.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2db9ffb700 1 -- 172.21.15.203:0/405754737 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2366779237 0 0) 0x7f2dbc0bb2d0 con 0x7f2dbc064760 2021-01-06T22:14:29.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2db9ffb700 1 -- 172.21.15.203:0/405754737 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2da8003230 con 0x7f2dbc064760 2021-01-06T22:14:29.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2db9ffb700 1 -- 172.21.15.203:0/405754737 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1765477173 0 0) 0x7f2da8003230 con 0x7f2dbc064760 2021-01-06T22:14:29.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2db9ffb700 1 -- 172.21.15.203:0/405754737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2dbc14e310 msgr2=0x7f2dbc1471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:29.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2db9ffb700 1 --2- 172.21.15.203:0/405754737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2dbc14e310 0x7f2dbc1471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:29.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2db9ffb700 1 -- 172.21.15.203:0/405754737 >> v1:172.21.15.91:6789/0 conn(0x7f2dbc064380 legacy=0x7f2dbc0c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:29.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2db9ffb700 1 -- 172.21.15.203:0/405754737 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc14a800 con 0x7f2dbc064760 2021-01-06T22:14:29.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2db9ffb700 1 -- 172.21.15.203:0/405754737 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f2db4001880 con 0x7f2dbc064760 2021-01-06T22:14:29.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.446+0000 7f2db9ffb700 1 -- 172.21.15.203:0/405754737 >> v1:172.21.15.203:6789/0 conn(0x7f2dbc064760 legacy=0x7f2dbc0bb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:29.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2db9ffb700 1 --2- 172.21.15.203:0/405754737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2dbc14e310 0x7f2da8003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:29.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2db9ffb700 1 --2- 172.21.15.203:0/405754737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2da8003c10 0x7f2da80044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:29.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2db9ffb700 1 --2- 172.21.15.203:0/405754737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2da80049c0 0x7f2da8004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:29.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2db9ffb700 1 -- 172.21.15.203:0/405754737 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc14a560 con 0x7f2da80049c0 2021-01-06T22:14:29.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2db9ffb700 1 -- 172.21.15.203:0/405754737 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da8004130 con 0x7f2da8003c10 2021-01-06T22:14:29.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2db9ffb700 1 -- 172.21.15.203:0/405754737 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da80042f0 con 0x7f2dbc14e310 2021-01-06T22:14:29.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dba7fc700 1 --2- 172.21.15.203:0/405754737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2da8003c10 0x7f2da80044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:29.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dbaffd700 1 --2- 172.21.15.203:0/405754737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2dbc14e310 0x7f2da8003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:29.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dbb7fe700 1 --2- 172.21.15.203:0/405754737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2da80049c0 0x7f2da8004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:29.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dba7fc700 1 -- 172.21.15.203:0/405754737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2dbc14e310 msgr2=0x7f2da8003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:29.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dba7fc700 1 --2- 172.21.15.203:0/405754737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2dbc14e310 0x7f2da8003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:29.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dba7fc700 1 -- 172.21.15.203:0/405754737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2da80049c0 msgr2=0x7f2da8004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:14:29.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dba7fc700 1 --2- 172.21.15.203:0/405754737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2da80049c0 0x7f2da8004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:29.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dba7fc700 1 -- 172.21.15.203:0/405754737 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da8003230 con 0x7f2da8003c10 2021-01-06T22:14:29.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dba7fc700 1 --2- 172.21.15.203:0/405754737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2da8003c10 0x7f2da80044d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f2db000bb60 tx=0x7f2db0008040).ready entity=mon.1 client_cookie=cce22012efffb9 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:29.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2db9ffb700 1 -- 172.21.15.203:0/405754737 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db000b2b0 con 0x7f2da8003c10 2021-01-06T22:14:29.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2db9ffb700 1 -- 172.21.15.203:0/405754737 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db000ea90 con 0x7f2da8003c10 2021-01-06T22:14:29.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2db9ffb700 1 -- 172.21.15.203:0/405754737 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db000ee30 con 0x7f2da8003c10 2021-01-06T22:14:29.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dbbfff700 1 -- 172.21.15.203:0/405754737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2da8003c10 msgr2=0x7f2da80044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:29.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dbbfff700 1 --2- 172.21.15.203:0/405754737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2da8003c10 0x7f2da80044d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f2db000bb60 tx=0x7f2db0008040).stop 2021-01-06T22:14:29.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dbb7fe700 1 -- 172.21.15.203:0/405754737 reap_dead start 2021-01-06T22:14:29.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dbbfff700 1 -- 172.21.15.203:0/405754737 shutdown_connections 2021-01-06T22:14:29.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dbbfff700 1 -- 172.21.15.203:0/405754737 >> 172.21.15.203:0/405754737 conn(0x7f2dbc0667b0 msgr2=0x7f2dbc0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:29.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dbbfff700 1 -- 172.21.15.203:0/405754737 shutdown_connections 2021-01-06T22:14:29.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dbbfff700 1 -- 172.21.15.203:0/405754737 wait complete. 2021-01-06T22:14:29.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dbbfff700 1 Processor -- start 2021-01-06T22:14:29.464 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.450+0000 7f2dbbfff700 1 -- start start 2021-01-06T22:14:29.464 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dbbfff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2da8003c10 0x7f2dbc15c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:29.464 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dbbfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2dbc0b8200 0x7f2dbc159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:29.464 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dbbfff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2dbc0b98a0 0x7f2dbc0b8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:29.464 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dbbfff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc15ca00 con 0x7f2dbc0b98a0 2021-01-06T22:14:29.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dbbfff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc15cca0 con 0x7f2dbc0b8200 2021-01-06T22:14:29.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dbbfff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc15cf40 con 0x7f2da8003c10 2021-01-06T22:14:29.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dba7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2dbc0b8200 0x7f2dbc159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:29.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dba7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2dbc0b8200 0x7f2dbc159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51788/0 (socket says 172.21.15.203:51788) 2021-01-06T22:14:29.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dba7fc700 1 -- 172.21.15.203:0/3791648780 learned_addr learned my addr 172.21.15.203:0/3791648780 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:14:29.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dbb7fe700 1 --2- 172.21.15.203:0/3791648780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2dbc0b98a0 0x7f2dbc0b8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:29.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dbaffd700 1 --2- 172.21.15.203:0/3791648780 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2da8003c10 0x7f2dbc15c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:29.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dba7fc700 1 -- 172.21.15.203:0/3791648780 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2da8003c10 msgr2=0x7f2dbc15c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:29.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dba7fc700 1 --2- 172.21.15.203:0/3791648780 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2da8003c10 0x7f2dbc15c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:29.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dba7fc700 1 -- 172.21.15.203:0/3791648780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2dbc0b98a0 msgr2=0x7f2dbc0b8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:29.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dba7fc700 1 --2- 172.21.15.203:0/3791648780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2dbc0b98a0 0x7f2dbc0b8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:29.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dba7fc700 1 -- 172.21.15.203:0/3791648780 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc15d1e0 con 0x7f2dbc0b8200 2021-01-06T22:14:29.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dba7fc700 1 --2- 172.21.15.203:0/3791648780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2dbc0b8200 0x7f2dbc159730 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f2db000c010 tx=0x7f2db001a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:29.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2da2ffd700 1 -- 172.21.15.203:0/3791648780 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0025430 con 0x7f2dbc0b8200 2021-01-06T22:14:29.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2da2ffd700 1 -- 172.21.15.203:0/3791648780 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db00255b0 con 0x7f2dbc0b8200 2021-01-06T22:14:29.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dbbfff700 1 -- 172.21.15.203:0/3791648780 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dbc159290 con 0x7f2dbc0b8200 2021-01-06T22:14:29.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2da2ffd700 1 -- 172.21.15.203:0/3791648780 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0025970 con 0x7f2dbc0b8200 2021-01-06T22:14:29.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2dbbfff700 1 -- 172.21.15.203:0/3791648780 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dbc0b87d0 con 0x7f2dbc0b8200 2021-01-06T22:14:29.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.454+0000 7f2da2ffd700 1 -- 172.21.15.203:0/3791648780 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f2db001b420 con 0x7f2dbc0b8200 2021-01-06T22:14:29.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.458+0000 7f2da2ffd700 1 --2- 172.21.15.203:0/3791648780 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2d9c0563d0 0x7f2d9c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:14:29.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.458+0000 7f2da2ffd700 1 -- 172.21.15.203:0/3791648780 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6589+0+0 (secure 0 0 0) 0x7f2db0071500 con 0x7f2dbc0b8200 2021-01-06T22:14:29.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.458+0000 7f2dbaffd700 1 --2- 172.21.15.203:0/3791648780 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2d9c0563d0 0x7f2d9c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:14:29.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.458+0000 7f2dbbfff700 1 -- 172.21.15.203:0/3791648780 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dbc159420 con 0x7f2dbc0b8200 2021-01-06T22:14:29.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.458+0000 7f2dbaffd700 1 --2- 172.21.15.203:0/3791648780 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2d9c0563d0 0x7f2d9c058890 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f2dac001230 tx=0x7f2dac009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:14:29.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.462+0000 7f2da2ffd700 1 -- 172.21.15.203:0/3791648780 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f2db002a070 con 0x7f2dbc0b8200 2021-01-06T22:14:29.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.710+0000 7f2dbbfff700 1 -- 172.21.15.203:0/3791648780 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2dbc05ca60 con 0x7f2dbc0b8200 2021-01-06T22:14:29.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.710+0000 7f2da2ffd700 1 -- 172.21.15.203:0/3791648780 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3563 (secure 0 0 0) 0x7f2db0016030 con 0x7f2dbc0b8200 2021-01-06T22:14:29.713 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:14:29.713 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5364,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:14:20.559082+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:14:29.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.710+0000 7f2dbbfff700 1 -- 172.21.15.203:0/3791648780 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2d9c0563d0 msgr2=0x7f2d9c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:29.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.710+0000 7f2dbbfff700 1 --2- 172.21.15.203:0/3791648780 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2d9c0563d0 0x7f2d9c058890 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f2dac001230 tx=0x7f2dac009040).stop 2021-01-06T22:14:29.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.710+0000 7f2dbbfff700 1 -- 172.21.15.203:0/3791648780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2dbc0b8200 msgr2=0x7f2dbc159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:14:29.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.710+0000 7f2dbbfff700 1 --2- 172.21.15.203:0/3791648780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2dbc0b8200 0x7f2dbc159730 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f2db000c010 tx=0x7f2db001a000).stop 2021-01-06T22:14:29.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.710+0000 7f2dbbfff700 1 -- 172.21.15.203:0/3791648780 shutdown_connections 2021-01-06T22:14:29.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.710+0000 7f2dbbfff700 1 --2- 172.21.15.203:0/3791648780 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2d9c0563d0 0x7f2d9c058890 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:29.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.710+0000 7f2dbbfff700 1 --2- 172.21.15.203:0/3791648780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2dbc0b98a0 0x7f2dbc0b8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:29.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.710+0000 7f2dbbfff700 1 --2- 172.21.15.203:0/3791648780 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2da8003c10 0x7f2dbc15c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:29.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.710+0000 7f2dbbfff700 1 --2- 172.21.15.203:0/3791648780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2dbc0b8200 0x7f2dbc159730 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:14:29.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.710+0000 7f2dbbfff700 1 -- 172.21.15.203:0/3791648780 >> 172.21.15.203:0/3791648780 conn(0x7f2dbc0667b0 msgr2=0x7f2dbc0bede0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:14:29.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.710+0000 7f2dbbfff700 1 -- 172.21.15.203:0/3791648780 shutdown_connections 2021-01-06T22:14:29.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:14:29.710+0000 7f2dbbfff700 1 -- 172.21.15.203:0/3791648780 wait complete. 2021-01-06T22:14:29.718 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 56 2021-01-06T22:14:29.733 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2021-01-06T22:14:29.733 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 90 secs before thrashing 2021-01-06T22:14:47.398 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:14:47.401 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:15:17.496 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:15:17.498 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:15:47.702 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:15:47.705 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:15:59.734 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:15:59.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.856+0000 7f5f4cfde700 1 Processor -- start 2021-01-06T22:15:59.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.856+0000 7f5f4cfde700 1 -- start start 2021-01-06T22:15:59.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.856+0000 7f5f4cfde700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5f480bf100 0x7f5f480bf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:15:59.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.856+0000 7f5f4cfde700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5f48151560 0x7f5f48147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:15:59.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.856+0000 7f5f4cfde700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f48152460 con 0x7f5f48064bc0 2021-01-06T22:15:59.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.856+0000 7f5f4cfde700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4814e740 con 0x7f5f48151560 2021-01-06T22:15:59.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.856+0000 7f5f4cfde700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4814e9e0 con 0x7f5f480bf100 2021-01-06T22:15:59.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.856+0000 7f5f4659c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5f480bf100 0x7f5f480bf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:15:59.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.856+0000 7f5f4659c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5f480bf100 0x7f5f480bf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51864/0 (socket says 172.21.15.203:51864) 2021-01-06T22:15:59.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.856+0000 7f5f4659c700 1 -- 172.21.15.203:0/499735585 learned_addr learned my addr 172.21.15.203:0/499735585 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:15:59.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f46d9d700 1 --2- 172.21.15.203:0/499735585 >> v2:172.21.15.91:3300/0 conn(0x7f5f48151560 0x7f5f48147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:15:59.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f4659c700 1 -- 172.21.15.203:0/499735585 >> v2:172.21.15.91:3300/0 conn(0x7f5f48151560 msgr2=0x7f5f48147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:15:59.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f4659c700 1 --2- 172.21.15.203:0/499735585 >> v2:172.21.15.91:3300/0 conn(0x7f5f48151560 0x7f5f48147bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:15:59.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f4659c700 1 -- 172.21.15.203:0/499735585 >> v1:172.21.15.203:6789/0 conn(0x7f5f48064bc0 legacy=0x7f5f4814df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:15:59.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f4659c700 1 -- 172.21.15.203:0/499735585 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4814ec80 con 0x7f5f480bf100 2021-01-06T22:15:59.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f4659c700 1 --2- 172.21.15.203:0/499735585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f480bf100 0x7f5f480bf4e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f5f48068760 tx=0x7f5f38006ea0).ready entity=mon.1 client_cookie=85cafae68b12cee9 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:15:59.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f45d9b700 1 -- 172.21.15.203:0/499735585 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f380098d0 con 0x7f5f480bf100 2021-01-06T22:15:59.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f45d9b700 1 -- 172.21.15.203:0/499735585 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f38009a30 con 0x7f5f480bf100 2021-01-06T22:15:59.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f45d9b700 1 -- 172.21.15.203:0/499735585 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38009db0 con 0x7f5f480bf100 2021-01-06T22:15:59.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f4cfde700 1 -- 172.21.15.203:0/499735585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f480bf100 msgr2=0x7f5f480bf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:15:59.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f4cfde700 1 --2- 172.21.15.203:0/499735585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f480bf100 0x7f5f480bf4e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f5f48068760 tx=0x7f5f38006ea0).stop 2021-01-06T22:15:59.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f4cfde700 1 -- 172.21.15.203:0/499735585 shutdown_connections 2021-01-06T22:15:59.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f4cfde700 1 --2- 172.21.15.203:0/499735585 >> v2:172.21.15.91:3300/0 conn(0x7f5f48151560 0x7f5f48147bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:15:59.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f4cfde700 1 --2- 172.21.15.203:0/499735585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f480bf100 0x7f5f480bf4e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:15:59.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f4cfde700 1 -- 172.21.15.203:0/499735585 >> 172.21.15.203:0/499735585 conn(0x7f5f480602b0 msgr2=0x7f5f480606c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:15:59.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f4cfde700 1 -- 172.21.15.203:0/499735585 shutdown_connections 2021-01-06T22:15:59.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f4cfde700 1 -- 172.21.15.203:0/499735585 wait complete. 2021-01-06T22:15:59.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f4cfde700 1 Processor -- start 2021-01-06T22:15:59.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.860+0000 7f5f4cfde700 1 -- start start 2021-01-06T22:15:59.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f4cfde700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5f48064bc0 0x7f5f4815e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:15:59.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f4cfde700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f480bf100 0x7f5f4814cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:15:59.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f4cfde700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5f48151560 0x7f5f480b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:15:59.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f4cfde700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4815ebc0 con 0x7f5f48064bc0 2021-01-06T22:15:59.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f4cfde700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4814c3c0 con 0x7f5f480bf100 2021-01-06T22:15:59.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f4cfde700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4814c620 con 0x7f5f48151560 2021-01-06T22:15:59.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f3ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f480bf100 0x7f5f4814cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:15:59.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f3ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f480bf100 0x7f5f4814cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51872/0 (socket says 172.21.15.203:51872) 2021-01-06T22:15:59.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f3ffff700 1 -- 172.21.15.203:0/2014815993 learned_addr learned my addr 172.21.15.203:0/2014815993 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:15:59.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f4659c700 1 --2- 172.21.15.203:0/2014815993 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5f48064bc0 0x7f5f4815e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:15:59.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f46d9d700 1 --2- 172.21.15.203:0/2014815993 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5f48151560 0x7f5f480b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:15:59.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f3ffff700 1 -- 172.21.15.203:0/2014815993 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5f48151560 msgr2=0x7f5f480b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:15:59.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f3ffff700 1 --2- 172.21.15.203:0/2014815993 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5f48151560 0x7f5f480b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:15:59.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f3ffff700 1 -- 172.21.15.203:0/2014815993 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5f48064bc0 msgr2=0x7f5f4815e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:15:59.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f3ffff700 1 --2- 172.21.15.203:0/2014815993 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5f48064bc0 0x7f5f4815e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:15:59.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f3ffff700 1 -- 172.21.15.203:0/2014815993 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4814c8c0 con 0x7f5f480bf100 2021-01-06T22:15:59.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f46d9d700 1 --2- 172.21.15.203:0/2014815993 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5f48151560 0x7f5f480b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:15:59.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f3ffff700 1 --2- 172.21.15.203:0/2014815993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f480bf100 0x7f5f4814cef0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f5f3000c740 tx=0x7f5f30007be0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:15:59.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f3e7fc700 1 -- 172.21.15.203:0/2014815993 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30007020 con 0x7f5f480bf100 2021-01-06T22:15:59.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f3e7fc700 1 -- 172.21.15.203:0/2014815993 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f30007180 con 0x7f5f480bf100 2021-01-06T22:15:59.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f4cfde700 1 -- 172.21.15.203:0/2014815993 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f480b3a50 con 0x7f5f480bf100 2021-01-06T22:15:59.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f3e7fc700 1 -- 172.21.15.203:0/2014815993 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f300064e0 con 0x7f5f480bf100 2021-01-06T22:15:59.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f4cfde700 1 -- 172.21.15.203:0/2014815993 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f48150940 con 0x7f5f480bf100 2021-01-06T22:15:59.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.864+0000 7f5f4cfde700 1 -- 172.21.15.203:0/2014815993 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4805d600 con 0x7f5f480bf100 2021-01-06T22:15:59.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.868+0000 7f5f3e7fc700 1 -- 172.21.15.203:0/2014815993 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f5f3001a060 con 0x7f5f480bf100 2021-01-06T22:15:59.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.868+0000 7f5f3e7fc700 1 --2- 172.21.15.203:0/2014815993 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5f2c0563d0 0x7f5f2c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:15:59.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.868+0000 7f5f3e7fc700 1 -- 172.21.15.203:0/2014815993 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6589+0+0 (secure 0 0 0) 0x7f5f3006edc0 con 0x7f5f480bf100 2021-01-06T22:15:59.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.868+0000 7f5f4659c700 1 --2- 172.21.15.203:0/2014815993 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5f2c0563d0 0x7f5f2c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:15:59.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.868+0000 7f5f4659c700 1 --2- 172.21.15.203:0/2014815993 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5f2c0563d0 0x7f5f2c058890 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f5f3800b190 tx=0x7f5f38016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:15:59.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:15:59.872+0000 7f5f3e7fc700 1 -- 172.21.15.203:0/2014815993 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f5f30044440 con 0x7f5f480bf100 2021-01-06T22:16:00.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.076+0000 7f5f4cfde700 1 -- 172.21.15.203:0/2014815993 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f481495e0 con 0x7f5f480bf100 2021-01-06T22:16:00.083 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.076+0000 7f5f3e7fc700 1 -- 172.21.15.203:0/2014815993 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3563 (secure 0 0 0) 0x7f5f30018030 con 0x7f5f480bf100 2021-01-06T22:16:00.083 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:00.083 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5364,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:14:20.559082+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:00.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.080+0000 7f5f4cfde700 1 -- 172.21.15.203:0/2014815993 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5f2c0563d0 msgr2=0x7f5f2c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:00.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.080+0000 7f5f4cfde700 1 --2- 172.21.15.203:0/2014815993 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5f2c0563d0 0x7f5f2c058890 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f5f3800b190 tx=0x7f5f38016000).stop 2021-01-06T22:16:00.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.080+0000 7f5f4cfde700 1 -- 172.21.15.203:0/2014815993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f480bf100 msgr2=0x7f5f4814cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:00.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.080+0000 7f5f4cfde700 1 --2- 172.21.15.203:0/2014815993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f480bf100 0x7f5f4814cef0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f5f3000c740 tx=0x7f5f30007be0).stop 2021-01-06T22:16:00.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.080+0000 7f5f4cfde700 1 -- 172.21.15.203:0/2014815993 shutdown_connections 2021-01-06T22:16:00.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.080+0000 7f5f4cfde700 1 --2- 172.21.15.203:0/2014815993 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5f2c0563d0 0x7f5f2c058890 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:00.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.080+0000 7f5f4cfde700 1 --2- 172.21.15.203:0/2014815993 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5f48151560 0x7f5f480b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:00.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.080+0000 7f5f4cfde700 1 --2- 172.21.15.203:0/2014815993 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5f48064bc0 0x7f5f4815e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:00.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.080+0000 7f5f4cfde700 1 --2- 172.21.15.203:0/2014815993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f480bf100 0x7f5f4814cef0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:00.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.080+0000 7f5f4cfde700 1 -- 172.21.15.203:0/2014815993 >> 172.21.15.203:0/2014815993 conn(0x7f5f480602b0 msgr2=0x7f5f480c26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:00.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.080+0000 7f5f4cfde700 1 -- 172.21.15.203:0/2014815993 shutdown_connections 2021-01-06T22:16:00.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.080+0000 7f5f4cfde700 1 -- 172.21.15.203:0/2014815993 wait complete. 2021-01-06T22:16:00.088 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 56 2021-01-06T22:16:00.098 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=1) 2021-01-06T22:16:00.098 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-01-06T22:16:00.098 INFO:teuthology.orchestra.run:waiting for 300 2021-01-06T22:16:00.110 INFO:tasks.ceph.mds.c:Stopped 2021-01-06T22:16:00.110 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:00.274 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.268+0000 7f5b237ea700 1 Processor -- start 2021-01-06T22:16:00.275 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.268+0000 7f5b237ea700 1 -- start start 2021-01-06T22:16:00.278 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b237ea700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5b1c14e680 0x7f5b1c063730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:00.278 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b237ea700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5b1c0bf670 0x7f5b1c0c1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:00.278 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b237ea700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1c14eb50 con 0x7f5b1c0bf290 2021-01-06T22:16:00.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b237ea700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1c0c0f80 con 0x7f5b1c14e680 2021-01-06T22:16:00.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b237ea700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1c0c1220 con 0x7f5b1c0bf670 2021-01-06T22:16:00.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b21d87700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5b1c0bf670 0x7f5b1c0c1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:00.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b21d87700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5b1c0bf670 0x7f5b1c0c1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51884/0 (socket says 172.21.15.203:51884) 2021-01-06T22:16:00.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b21d87700 1 -- 172.21.15.203:0/341499067 learned_addr learned my addr 172.21.15.203:0/341499067 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:00.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b20d85700 1 --2- 172.21.15.203:0/341499067 >> v2:172.21.15.91:3300/0 conn(0x7f5b1c14e680 0x7f5b1c063730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:00.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b21d87700 1 -- 172.21.15.203:0/341499067 >> v2:172.21.15.91:3300/0 conn(0x7f5b1c14e680 msgr2=0x7f5b1c063730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:00.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b21d87700 1 --2- 172.21.15.203:0/341499067 >> v2:172.21.15.91:3300/0 conn(0x7f5b1c14e680 0x7f5b1c063730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:00.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b21d87700 1 -- 172.21.15.203:0/341499067 >> v1:172.21.15.91:6789/0 conn(0x7f5b1c0bf290 legacy=0x7f5b1c14df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:00.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b21d87700 1 -- 172.21.15.203:0/341499067 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c0c14c0 con 0x7f5b1c0bf670 2021-01-06T22:16:00.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b20d85700 1 --2- 172.21.15.203:0/341499067 >> v2:172.21.15.91:3300/0 conn(0x7f5b1c14e680 0x7f5b1c063730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:16:00.282 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b21d87700 1 --2- 172.21.15.203:0/341499067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b1c0bf670 0x7f5b1c0c1c00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f5b1c068790 tx=0x7f5b14009290).ready entity=mon.1 client_cookie=1e24fef5a529f5af server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:00.282 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b0bfff700 1 -- 172.21.15.203:0/341499067 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1400bc50 con 0x7f5b1c0bf670 2021-01-06T22:16:00.282 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b0bfff700 1 -- 172.21.15.203:0/341499067 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1400bdb0 con 0x7f5b1c0bf670 2021-01-06T22:16:00.283 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b0bfff700 1 -- 172.21.15.203:0/341499067 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14011610 con 0x7f5b1c0bf670 2021-01-06T22:16:00.283 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b237ea700 1 -- 172.21.15.203:0/341499067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b1c0bf670 msgr2=0x7f5b1c0c1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:00.284 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b237ea700 1 --2- 172.21.15.203:0/341499067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b1c0bf670 0x7f5b1c0c1c00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f5b1c068790 tx=0x7f5b14009290).stop 2021-01-06T22:16:00.284 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b237ea700 1 -- 172.21.15.203:0/341499067 shutdown_connections 2021-01-06T22:16:00.284 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b237ea700 1 --2- 172.21.15.203:0/341499067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b1c0bf670 0x7f5b1c0c1c00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:00.285 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b237ea700 1 --2- 172.21.15.203:0/341499067 >> v2:172.21.15.91:3300/0 conn(0x7f5b1c14e680 0x7f5b1c063730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:00.285 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b237ea700 1 -- 172.21.15.203:0/341499067 >> 172.21.15.203:0/341499067 conn(0x7f5b1c0626e0 msgr2=0x7f5b1c061a30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:00.286 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b237ea700 1 -- 172.21.15.203:0/341499067 shutdown_connections 2021-01-06T22:16:00.287 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.272+0000 7f5b237ea700 1 -- 172.21.15.203:0/341499067 wait complete. 2021-01-06T22:16:00.290 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b237ea700 1 Processor -- start 2021-01-06T22:16:00.290 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b237ea700 1 -- start start 2021-01-06T22:16:00.290 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b237ea700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5b1c0bf290 0x7f5b1c0ba2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:00.290 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b237ea700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b1c0bf670 0x7f5b1c162f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:00.291 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b237ea700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b1c14e680 0x7f5b1c1624b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:00.291 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b237ea700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c148030 con 0x7f5b1c0bf670 2021-01-06T22:16:00.291 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b237ea700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c1629a0 con 0x7f5b1c14e680 2021-01-06T22:16:00.291 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b237ea700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c147d90 con 0x7f5b1c0bf290 2021-01-06T22:16:00.292 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b21d87700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b1c14e680 0x7f5b1c1624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:00.292 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b21d87700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b1c14e680 0x7f5b1c1624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51890/0 (socket says 172.21.15.203:51890) 2021-01-06T22:16:00.292 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b21d87700 1 -- 172.21.15.203:0/34802787 learned_addr learned my addr 172.21.15.203:0/34802787 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:00.292 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b21586700 1 --2- 172.21.15.203:0/34802787 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5b1c0bf290 0x7f5b1c0ba2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:00.293 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b20d85700 1 --2- 172.21.15.203:0/34802787 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b1c0bf670 0x7f5b1c162f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:00.293 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b21d87700 1 -- 172.21.15.203:0/34802787 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5b1c0bf290 msgr2=0x7f5b1c0ba2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:00.293 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b21d87700 1 --2- 172.21.15.203:0/34802787 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5b1c0bf290 0x7f5b1c0ba2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:00.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b21d87700 1 -- 172.21.15.203:0/34802787 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b1c0bf670 msgr2=0x7f5b1c162f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:00.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b21d87700 1 --2- 172.21.15.203:0/34802787 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b1c0bf670 0x7f5b1c162f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:00.294 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b21d87700 1 -- 172.21.15.203:0/34802787 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c0b9640 con 0x7f5b1c14e680 2021-01-06T22:16:00.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b21d87700 1 --2- 172.21.15.203:0/34802787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b1c14e680 0x7f5b1c1624b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f5b1c0c1360 tx=0x7f5b14017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:00.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b097fa700 1 -- 172.21.15.203:0/34802787 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14011e70 con 0x7f5b1c14e680 2021-01-06T22:16:00.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b097fa700 1 -- 172.21.15.203:0/34802787 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b140032a0 con 0x7f5b1c14e680 2021-01-06T22:16:00.295 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b237ea700 1 -- 172.21.15.203:0/34802787 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1c0b98e0 con 0x7f5b1c14e680 2021-01-06T22:16:00.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b097fa700 1 -- 172.21.15.203:0/34802787 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14010c90 con 0x7f5b1c14e680 2021-01-06T22:16:00.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.276+0000 7f5b237ea700 1 -- 172.21.15.203:0/34802787 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1c0b9e20 con 0x7f5b1c14e680 2021-01-06T22:16:00.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.280+0000 7f5b237ea700 1 -- 172.21.15.203:0/34802787 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1c05cfe0 con 0x7f5b1c14e680 2021-01-06T22:16:00.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.280+0000 7f5b097fa700 1 -- 172.21.15.203:0/34802787 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f5b1400f070 con 0x7f5b1c14e680 2021-01-06T22:16:00.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.280+0000 7f5b097fa700 1 --2- 172.21.15.203:0/34802787 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5afc0563d0 0x7f5afc058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:00.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.280+0000 7f5b097fa700 1 -- 172.21.15.203:0/34802787 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6589+0+0 (secure 0 0 0) 0x7f5b14071680 con 0x7f5b1c14e680 2021-01-06T22:16:00.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.280+0000 7f5b21586700 1 --2- 172.21.15.203:0/34802787 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5afc0563d0 0x7f5afc058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:00.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.280+0000 7f5b21586700 1 --2- 172.21.15.203:0/34802787 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5afc0563d0 0x7f5afc058890 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f5b0c0035f0 tx=0x7f5b0c007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:00.298 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.284+0000 7f5b097fa700 1 -- 172.21.15.203:0/34802787 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f5b14046d00 con 0x7f5b1c14e680 2021-01-06T22:16:00.515 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.508+0000 7f5b237ea700 1 -- 172.21.15.203:0/34802787 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b1c064db0 con 0x7f5b1c14e680 2021-01-06T22:16:00.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.512+0000 7f5b097fa700 1 -- 172.21.15.203:0/34802787 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3563 (secure 0 0 0) 0x7f5b1404a320 con 0x7f5b1c14e680 2021-01-06T22:16:00.520 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:00.520 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5364,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:14:20.559082+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:00.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.512+0000 7f5b237ea700 1 -- 172.21.15.203:0/34802787 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5afc0563d0 msgr2=0x7f5afc058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:00.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.512+0000 7f5b237ea700 1 --2- 172.21.15.203:0/34802787 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5afc0563d0 0x7f5afc058890 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f5b0c0035f0 tx=0x7f5b0c007b80).stop 2021-01-06T22:16:00.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.516+0000 7f5b237ea700 1 -- 172.21.15.203:0/34802787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b1c14e680 msgr2=0x7f5b1c1624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:00.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.516+0000 7f5b237ea700 1 --2- 172.21.15.203:0/34802787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b1c14e680 0x7f5b1c1624b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f5b1c0c1360 tx=0x7f5b14017000).stop 2021-01-06T22:16:00.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.516+0000 7f5b237ea700 1 -- 172.21.15.203:0/34802787 shutdown_connections 2021-01-06T22:16:00.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.516+0000 7f5b237ea700 1 --2- 172.21.15.203:0/34802787 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5afc0563d0 0x7f5afc058890 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:00.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.516+0000 7f5b237ea700 1 --2- 172.21.15.203:0/34802787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b1c14e680 0x7f5b1c1624b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:00.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.516+0000 7f5b237ea700 1 --2- 172.21.15.203:0/34802787 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5b1c0bf290 0x7f5b1c0ba2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:00.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.516+0000 7f5b237ea700 1 --2- 172.21.15.203:0/34802787 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b1c0bf670 0x7f5b1c162f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:00.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.516+0000 7f5b237ea700 1 -- 172.21.15.203:0/34802787 >> 172.21.15.203:0/34802787 conn(0x7f5b1c0626e0 msgr2=0x7f5b1c0c4620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:00.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.516+0000 7f5b237ea700 1 -- 172.21.15.203:0/34802787 shutdown_connections 2021-01-06T22:16:00.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:00.516+0000 7f5b237ea700 1 -- 172.21.15.203:0/34802787 wait complete. 2021-01-06T22:16:00.525 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 56 2021-01-06T22:16:00.536 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:16:02.537 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:02.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.668+0000 7f74c02cb700 1 Processor -- start 2021-01-06T22:16:02.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.668+0000 7f74c02cb700 1 -- start start 2021-01-06T22:16:02.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74c02cb700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74b80676d0 0x7f74b80682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:02.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74c02cb700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f74b8067ab0 0x7f74b814ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:02.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74c02cb700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b814a7b0 con 0x7f74b80602b0 2021-01-06T22:16:02.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74c02cb700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b8061110 con 0x7f74b8067ab0 2021-01-06T22:16:02.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74c02cb700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80613b0 con 0x7f74b80676d0 2021-01-06T22:16:02.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74be868700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f74b80602b0 0x7f74b81476a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:45164/0 (socket says 172.21.15.203:45164) 2021-01-06T22:16:02.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74bd866700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f74b8067ab0 0x7f74b814ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:02.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74be868700 1 -- 172.21.15.203:0/1143378885 learned_addr learned my addr 172.21.15.203:0/1143378885 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:16:02.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74be067700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74b80676d0 0x7f74b80682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:02.679 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74bd065700 1 -- 172.21.15.203:0/1143378885 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625373723 0 0) 0x7f74b0001450 con 0x7f74b80602b0 2021-01-06T22:16:02.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74bd065700 1 -- 172.21.15.203:0/1143378885 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74a0003230 con 0x7f74b80602b0 2021-01-06T22:16:02.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74bd866700 1 -- 172.21.15.203:0/1143378885 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74b80676d0 msgr2=0x7f74b80682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:02.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74bd866700 1 --2- 172.21.15.203:0/1143378885 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74b80676d0 0x7f74b80682c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:02.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74bd866700 1 -- 172.21.15.203:0/1143378885 >> v1:172.21.15.91:6789/0 conn(0x7f74b80602b0 legacy=0x7f74b81476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:02.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74bd866700 1 -- 172.21.15.203:0/1143378885 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b8061650 con 0x7f74b8067ab0 2021-01-06T22:16:02.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74be067700 1 --2- 172.21.15.203:0/1143378885 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74b80676d0 0x7f74b80682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:16:02.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74bd866700 1 --2- 172.21.15.203:0/1143378885 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74b8067ab0 0x7f74b814ae90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f74a800b9b0 tx=0x7f74a8006de0).ready entity=mon.0 client_cookie=436358bad1206121 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:02.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74bd065700 1 -- 172.21.15.203:0/1143378885 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a800f770 con 0x7f74b8067ab0 2021-01-06T22:16:02.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74bd065700 1 -- 172.21.15.203:0/1143378885 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a800f8d0 con 0x7f74b8067ab0 2021-01-06T22:16:02.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74bd065700 1 -- 172.21.15.203:0/1143378885 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a800fc50 con 0x7f74b8067ab0 2021-01-06T22:16:02.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74c02cb700 1 -- 172.21.15.203:0/1143378885 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74b8067ab0 msgr2=0x7f74b814ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:02.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74c02cb700 1 --2- 172.21.15.203:0/1143378885 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74b8067ab0 0x7f74b814ae90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f74a800b9b0 tx=0x7f74a8006de0).stop 2021-01-06T22:16:02.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74c02cb700 1 -- 172.21.15.203:0/1143378885 shutdown_connections 2021-01-06T22:16:02.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74c02cb700 1 --2- 172.21.15.203:0/1143378885 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74b80676d0 0x7f74b80682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:02.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74c02cb700 1 --2- 172.21.15.203:0/1143378885 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74b8067ab0 0x7f74b814ae90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:02.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74c02cb700 1 -- 172.21.15.203:0/1143378885 >> 172.21.15.203:0/1143378885 conn(0x7f74b80c0c50 msgr2=0x7f74b80c1030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:02.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74c02cb700 1 -- 172.21.15.203:0/1143378885 shutdown_connections 2021-01-06T22:16:02.684 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.672+0000 7f74c02cb700 1 -- 172.21.15.203:0/1143378885 wait complete. 2021-01-06T22:16:02.684 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74c02cb700 1 Processor -- start 2021-01-06T22:16:02.684 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74c02cb700 1 -- start start 2021-01-06T22:16:02.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74c02cb700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74b80602b0 0x7f74b815e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:02.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74c02cb700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74b80676d0 0x7f74b814f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:02.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74c02cb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74b8067ab0 0x7f74b80b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:02.687 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74c02cb700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80b5fd0 con 0x7f74b80602b0 2021-01-06T22:16:02.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74c02cb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b814d060 con 0x7f74b8067ab0 2021-01-06T22:16:02.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74c02cb700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b814e1b0 con 0x7f74b80676d0 2021-01-06T22:16:02.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74be868700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74b8067ab0 0x7f74b80b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:02.688 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74bd866700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74b80676d0 0x7f74b814f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:02.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74be868700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74b8067ab0 0x7f74b80b6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51922/0 (socket says 172.21.15.203:51922) 2021-01-06T22:16:02.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74bd866700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74b80676d0 0x7f74b814f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:57002/0 (socket says 172.21.15.203:57002) 2021-01-06T22:16:02.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74be067700 1 --2- 172.21.15.203:0/1126584704 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74b80602b0 0x7f74b815e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:02.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74be868700 1 -- 172.21.15.203:0/1126584704 learned_addr learned my addr 172.21.15.203:0/1126584704 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:02.689 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74be868700 1 -- 172.21.15.203:0/1126584704 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74b80676d0 msgr2=0x7f74b814f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:02.690 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74be868700 1 --2- 172.21.15.203:0/1126584704 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74b80676d0 0x7f74b814f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:02.690 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74be868700 1 -- 172.21.15.203:0/1126584704 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74b80602b0 msgr2=0x7f74b815e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:02.690 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74be868700 1 --2- 172.21.15.203:0/1126584704 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74b80602b0 0x7f74b815e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:02.690 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74be868700 1 -- 172.21.15.203:0/1126584704 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b814d290 con 0x7f74b8067ab0 2021-01-06T22:16:02.691 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74be868700 1 --2- 172.21.15.203:0/1126584704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74b8067ab0 0x7f74b80b6330 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f74b000c9b0 tx=0x7f74b0007c70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:02.691 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74a67fc700 1 -- 172.21.15.203:0/1126584704 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0005b70 con 0x7f74b8067ab0 2021-01-06T22:16:02.691 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74a67fc700 1 -- 172.21.15.203:0/1126584704 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b0005cd0 con 0x7f74b8067ab0 2021-01-06T22:16:02.691 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74c02cb700 1 -- 172.21.15.203:0/1126584704 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b814d530 con 0x7f74b8067ab0 2021-01-06T22:16:02.691 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74a67fc700 1 -- 172.21.15.203:0/1126584704 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0007320 con 0x7f74b8067ab0 2021-01-06T22:16:02.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.676+0000 7f74c02cb700 1 -- 172.21.15.203:0/1126584704 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b814da70 con 0x7f74b8067ab0 2021-01-06T22:16:02.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.680+0000 7f74c02cb700 1 -- 172.21.15.203:0/1126584704 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b805d600 con 0x7f74b8067ab0 2021-01-06T22:16:02.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.680+0000 7f74a67fc700 1 -- 172.21.15.203:0/1126584704 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f74b0010030 con 0x7f74b8067ab0 2021-01-06T22:16:02.692 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.680+0000 7f74a67fc700 1 --2- 172.21.15.203:0/1126584704 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7498056400 0x7f74980588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:02.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.680+0000 7f74a67fc700 1 -- 172.21.15.203:0/1126584704 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6589+0+0 (secure 0 0 0) 0x7f74b006f4a0 con 0x7f74b8067ab0 2021-01-06T22:16:02.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.680+0000 7f74be067700 1 --2- 172.21.15.203:0/1126584704 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7498056400 0x7f74980588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:02.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.680+0000 7f74be067700 1 --2- 172.21.15.203:0/1126584704 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7498056400 0x7f74980588c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f74b400bd90 tx=0x7f74b4006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:02.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.688+0000 7f74a67fc700 1 -- 172.21.15.203:0/1126584704 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f74b0044b20 con 0x7f74b8067ab0 2021-01-06T22:16:02.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.920+0000 7f74c02cb700 1 -- 172.21.15.203:0/1126584704 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74b814a7b0 con 0x7f74b8067ab0 2021-01-06T22:16:02.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.920+0000 7f74a67fc700 1 -- 172.21.15.203:0/1126584704 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3563 (secure 0 0 0) 0x7f74b814a7b0 con 0x7f74b8067ab0 2021-01-06T22:16:02.925 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:02.925 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5364,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:14:20.559082+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:02.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.920+0000 7f74c02cb700 1 -- 172.21.15.203:0/1126584704 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7498056400 msgr2=0x7f74980588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:02.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.920+0000 7f74c02cb700 1 --2- 172.21.15.203:0/1126584704 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7498056400 0x7f74980588c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f74b400bd90 tx=0x7f74b4006de0).stop 2021-01-06T22:16:02.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.920+0000 7f74c02cb700 1 -- 172.21.15.203:0/1126584704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74b8067ab0 msgr2=0x7f74b80b6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:02.928 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.920+0000 7f74c02cb700 1 --2- 172.21.15.203:0/1126584704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74b8067ab0 0x7f74b80b6330 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f74b000c9b0 tx=0x7f74b0007c70).stop 2021-01-06T22:16:02.928 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.920+0000 7f74c02cb700 1 -- 172.21.15.203:0/1126584704 shutdown_connections 2021-01-06T22:16:02.928 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.920+0000 7f74c02cb700 1 --2- 172.21.15.203:0/1126584704 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7498056400 0x7f74980588c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:02.928 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.920+0000 7f74c02cb700 1 --2- 172.21.15.203:0/1126584704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74b8067ab0 0x7f74b80b6330 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:02.928 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.920+0000 7f74c02cb700 1 --2- 172.21.15.203:0/1126584704 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74b80602b0 0x7f74b815e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:02.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.920+0000 7f74c02cb700 1 --2- 172.21.15.203:0/1126584704 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74b80676d0 0x7f74b814f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:02.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.920+0000 7f74c02cb700 1 -- 172.21.15.203:0/1126584704 >> 172.21.15.203:0/1126584704 conn(0x7f74b80c0c50 msgr2=0x7f74b80bbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:02.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.920+0000 7f74c02cb700 1 -- 172.21.15.203:0/1126584704 shutdown_connections 2021-01-06T22:16:02.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:02.920+0000 7f74c02cb700 1 -- 172.21.15.203:0/1126584704 wait complete. 2021-01-06T22:16:02.930 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 56 2021-01-06T22:16:02.943 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:16:04.945 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:05.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.072+0000 7f58c96d4700 1 Processor -- start 2021-01-06T22:16:05.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.072+0000 7f58c96d4700 1 -- start start 2021-01-06T22:16:05.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c96d4700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58c40c1430 0x7f58c4148140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:05.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c96d4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f58c40bb230 0x7f58c414b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:05.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c96d4700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c414b960 con 0x7f58c4147d60 2021-01-06T22:16:05.081 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c96d4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c4061b50 con 0x7f58c40bb230 2021-01-06T22:16:05.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c96d4700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c40bb7e0 con 0x7f58c40c1430 2021-01-06T22:16:05.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c8ed3700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f58c40bb230 0x7f58c414b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:05.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c8ed3700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f58c40bb230 0x7f58c414b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51936/0 (socket says 172.21.15.203:51936) 2021-01-06T22:16:05.082 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c8ed3700 1 -- 172.21.15.203:0/1677600470 learned_addr learned my addr 172.21.15.203:0/1677600470 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:05.083 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c37fe700 1 --2- 172.21.15.203:0/1677600470 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58c40c1430 0x7f58c4148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:05.083 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c8ed3700 1 -- 172.21.15.203:0/1677600470 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58c40c1430 msgr2=0x7f58c4148140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:05.083 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c8ed3700 1 --2- 172.21.15.203:0/1677600470 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58c40c1430 0x7f58c4148140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:05.083 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c8ed3700 1 -- 172.21.15.203:0/1677600470 >> v1:172.21.15.91:6789/0 conn(0x7f58c4147d60 legacy=0x7f58c40671a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:05.083 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c8ed3700 1 -- 172.21.15.203:0/1677600470 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c40bba80 con 0x7f58c40bb230 2021-01-06T22:16:05.084 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c8ed3700 1 --2- 172.21.15.203:0/1677600470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58c40bb230 0x7f58c414b470 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f58c414baf0 tx=0x7f58ac006ea0).ready entity=mon.1 client_cookie=36f532325b9115c5 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:05.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c2ffd700 1 -- 172.21.15.203:0/1677600470 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac009960 con 0x7f58c40bb230 2021-01-06T22:16:05.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c2ffd700 1 -- 172.21.15.203:0/1677600470 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac009ac0 con 0x7f58c40bb230 2021-01-06T22:16:05.085 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c2ffd700 1 -- 172.21.15.203:0/1677600470 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac009e40 con 0x7f58c40bb230 2021-01-06T22:16:05.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c96d4700 1 -- 172.21.15.203:0/1677600470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58c40bb230 msgr2=0x7f58c414b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:05.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c96d4700 1 --2- 172.21.15.203:0/1677600470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58c40bb230 0x7f58c414b470 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f58c414baf0 tx=0x7f58ac006ea0).stop 2021-01-06T22:16:05.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c96d4700 1 -- 172.21.15.203:0/1677600470 shutdown_connections 2021-01-06T22:16:05.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c96d4700 1 --2- 172.21.15.203:0/1677600470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58c40bb230 0x7f58c414b470 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:05.086 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c96d4700 1 --2- 172.21.15.203:0/1677600470 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58c40c1430 0x7f58c4148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:05.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c96d4700 1 -- 172.21.15.203:0/1677600470 >> 172.21.15.203:0/1677600470 conn(0x7f58c40667b0 msgr2=0x7f58c40c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:05.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c96d4700 1 -- 172.21.15.203:0/1677600470 shutdown_connections 2021-01-06T22:16:05.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.076+0000 7f58c96d4700 1 -- 172.21.15.203:0/1677600470 wait complete. 2021-01-06T22:16:05.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c96d4700 1 Processor -- start 2021-01-06T22:16:05.087 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c96d4700 1 -- start start 2021-01-06T22:16:05.089 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c96d4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58c40bb230 0x7f58c414cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:05.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c96d4700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58c40c1430 0x7f58c414d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:05.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c96d4700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58c4147d60 0x7f58c4141680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:05.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c96d4700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c414c5b0 con 0x7f58c4147d60 2021-01-06T22:16:05.090 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c96d4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c40b92e0 con 0x7f58c40bb230 2021-01-06T22:16:05.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c96d4700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c414fd70 con 0x7f58c40c1430 2021-01-06T22:16:05.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58c40bb230 0x7f58c414cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:05.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58c40bb230 0x7f58c414cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51938/0 (socket says 172.21.15.203:51938) 2021-01-06T22:16:05.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c3fff700 1 -- 172.21.15.203:0/911639765 learned_addr learned my addr 172.21.15.203:0/911639765 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:05.091 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c37fe700 1 --2- 172.21.15.203:0/911639765 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58c40c1430 0x7f58c414d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:05.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c8ed3700 1 --2- 172.21.15.203:0/911639765 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58c4147d60 0x7f58c4141680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:05.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c3fff700 1 -- 172.21.15.203:0/911639765 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58c40c1430 msgr2=0x7f58c414d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:05.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c3fff700 1 --2- 172.21.15.203:0/911639765 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58c40c1430 0x7f58c414d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:05.092 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c3fff700 1 -- 172.21.15.203:0/911639765 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58c4147d60 msgr2=0x7f58c4141680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:05.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c3fff700 1 --2- 172.21.15.203:0/911639765 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58c4147d60 0x7f58c4141680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:05.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c3fff700 1 -- 172.21.15.203:0/911639765 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c4150010 con 0x7f58c40bb230 2021-01-06T22:16:05.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c3fff700 1 --2- 172.21.15.203:0/911639765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58c40bb230 0x7f58c414cc80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f58bc00ca90 tx=0x7f58bc0013c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:05.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58bbfff700 1 -- 172.21.15.203:0/911639765 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc006860 con 0x7f58c40bb230 2021-01-06T22:16:05.093 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58bbfff700 1 -- 172.21.15.203:0/911639765 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58bc0069c0 con 0x7f58c40bb230 2021-01-06T22:16:05.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58c96d4700 1 -- 172.21.15.203:0/911639765 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c41502b0 con 0x7f58c40bb230 2021-01-06T22:16:05.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.080+0000 7f58bbfff700 1 -- 172.21.15.203:0/911639765 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc006d60 con 0x7f58c40bb230 2021-01-06T22:16:05.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.084+0000 7f58c96d4700 1 -- 172.21.15.203:0/911639765 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c4066e80 con 0x7f58c40bb230 2021-01-06T22:16:05.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.084+0000 7f58c96d4700 1 -- 172.21.15.203:0/911639765 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c40450a0 con 0x7f58c40bb230 2021-01-06T22:16:05.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.084+0000 7f58bbfff700 1 -- 172.21.15.203:0/911639765 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f58bc010070 con 0x7f58c40bb230 2021-01-06T22:16:05.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.084+0000 7f58bbfff700 1 --2- 172.21.15.203:0/911639765 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58a8056400 0x7f58a80588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:05.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.084+0000 7f58bbfff700 1 -- 172.21.15.203:0/911639765 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6589+0+0 (secure 0 0 0) 0x7f58bc06ec70 con 0x7f58c40bb230 2021-01-06T22:16:05.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.084+0000 7f58c37fe700 1 --2- 172.21.15.203:0/911639765 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58a8056400 0x7f58a80588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:05.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.084+0000 7f58c37fe700 1 --2- 172.21.15.203:0/911639765 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58a8056400 0x7f58a80588c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f58c4150740 tx=0x7f58b4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:05.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.092+0000 7f58bbfff700 1 -- 172.21.15.203:0/911639765 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f58bc0442f0 con 0x7f58c40bb230 2021-01-06T22:16:05.321 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.316+0000 7f58c96d4700 1 -- 172.21.15.203:0/911639765 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58c405ca60 con 0x7f58c40bb230 2021-01-06T22:16:05.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.316+0000 7f58bbfff700 1 -- 172.21.15.203:0/911639765 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3563 (secure 0 0 0) 0x7f58bc047910 con 0x7f58c40bb230 2021-01-06T22:16:05.322 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:05.322 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5364,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:14:20.559082+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:05.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.320+0000 7f58c96d4700 1 -- 172.21.15.203:0/911639765 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58a8056400 msgr2=0x7f58a80588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:05.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.320+0000 7f58c96d4700 1 --2- 172.21.15.203:0/911639765 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58a8056400 0x7f58a80588c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f58c4150740 tx=0x7f58b4008040).stop 2021-01-06T22:16:05.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.320+0000 7f58c96d4700 1 -- 172.21.15.203:0/911639765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58c40bb230 msgr2=0x7f58c414cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:05.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.320+0000 7f58c96d4700 1 --2- 172.21.15.203:0/911639765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58c40bb230 0x7f58c414cc80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f58bc00ca90 tx=0x7f58bc0013c0).stop 2021-01-06T22:16:05.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.320+0000 7f58c96d4700 1 -- 172.21.15.203:0/911639765 shutdown_connections 2021-01-06T22:16:05.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.320+0000 7f58c96d4700 1 --2- 172.21.15.203:0/911639765 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58a8056400 0x7f58a80588c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:05.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.320+0000 7f58c96d4700 1 --2- 172.21.15.203:0/911639765 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58c4147d60 0x7f58c4141680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:05.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.320+0000 7f58c96d4700 1 --2- 172.21.15.203:0/911639765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58c40bb230 0x7f58c414cc80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:05.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.320+0000 7f58c96d4700 1 --2- 172.21.15.203:0/911639765 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58c40c1430 0x7f58c414d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:05.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.320+0000 7f58c96d4700 1 -- 172.21.15.203:0/911639765 >> 172.21.15.203:0/911639765 conn(0x7f58c40667b0 msgr2=0x7f58c41475d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:05.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.320+0000 7f58c96d4700 1 -- 172.21.15.203:0/911639765 shutdown_connections 2021-01-06T22:16:05.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:05.320+0000 7f58c96d4700 1 -- 172.21.15.203:0/911639765 wait complete. 2021-01-06T22:16:05.327 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 56 2021-01-06T22:16:05.336 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:16:07.337 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:07.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.448+0000 7f73221a6700 1 Processor -- start 2021-01-06T22:16:07.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.448+0000 7f73221a6700 1 -- start start 2021-01-06T22:16:07.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.448+0000 7f73221a6700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f731c148d70 0x7f731c0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:07.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.448+0000 7f73221a6700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f731c14f0a0 con 0x7f731c149150 2021-01-06T22:16:07.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.448+0000 7f73221a6700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f731c1495a0 con 0x7f731c0602b0 2021-01-06T22:16:07.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.448+0000 7f73221a6700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f731c14c3c0 con 0x7f731c148d70 2021-01-06T22:16:07.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f73209a3700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f731c0602b0 0x7f731c14fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50548/0 (socket says 172.21.15.203:50548) 2021-01-06T22:16:07.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f73209a3700 1 -- 172.21.15.203:0/3703833497 learned_addr learned my addr 172.21.15.203:0/3703833497 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:16:07.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f73211a4700 1 --2- 172.21.15.203:0/3703833497 >> v2:172.21.15.91:3300/0 conn(0x7f731c148d70 0x7f731c0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:07.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f730bfff700 1 -- 172.21.15.203:0/3703833497 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4209699929 0 0) 0x7f730c0010d0 con 0x7f731c0602b0 2021-01-06T22:16:07.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f730bfff700 1 -- 172.21.15.203:0/3703833497 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7304003230 con 0x7f731c0602b0 2021-01-06T22:16:07.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f730bfff700 1 -- 172.21.15.203:0/3703833497 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256135245 0 0) 0x7f731c14f0a0 con 0x7f731c149150 2021-01-06T22:16:07.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f730bfff700 1 -- 172.21.15.203:0/3703833497 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f730c0010d0 con 0x7f731c149150 2021-01-06T22:16:07.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f730bfff700 1 -- 172.21.15.203:0/3703833497 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2714405258 0 0) 0x7f7304003230 con 0x7f731c0602b0 2021-01-06T22:16:07.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f73211a4700 1 -- 172.21.15.203:0/3703833497 >> v1:172.21.15.203:6789/0 conn(0x7f731c0602b0 legacy=0x7f731c14fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:07.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f73211a4700 1 -- 172.21.15.203:0/3703833497 >> v1:172.21.15.91:6789/0 conn(0x7f731c149150 legacy=0x7f731c14bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:07.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f73211a4700 1 -- 172.21.15.203:0/3703833497 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c14c660 con 0x7f731c148d70 2021-01-06T22:16:07.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f73211a4700 1 --2- 172.21.15.203:0/3703833497 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f731c148d70 0x7f731c0c0fa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f731800b900 tx=0x7f7318006de0).ready entity=mon.0 client_cookie=4a6718d5c383c121 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:07.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f730bfff700 1 -- 172.21.15.203:0/3703833497 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c001670 con 0x7f731c148d70 2021-01-06T22:16:07.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f730bfff700 1 -- 172.21.15.203:0/3703833497 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7318011400 con 0x7f731c148d70 2021-01-06T22:16:07.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f730bfff700 1 -- 172.21.15.203:0/3703833497 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318011780 con 0x7f731c148d70 2021-01-06T22:16:07.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f73221a6700 1 -- 172.21.15.203:0/3703833497 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f731c148d70 msgr2=0x7f731c0c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:07.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f73221a6700 1 --2- 172.21.15.203:0/3703833497 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f731c148d70 0x7f731c0c0fa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f731800b900 tx=0x7f7318006de0).stop 2021-01-06T22:16:07.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f73221a6700 1 -- 172.21.15.203:0/3703833497 shutdown_connections 2021-01-06T22:16:07.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f73221a6700 1 --2- 172.21.15.203:0/3703833497 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f731c148d70 0x7f731c0c0fa0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:07.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f73221a6700 1 -- 172.21.15.203:0/3703833497 >> 172.21.15.203:0/3703833497 conn(0x7f731c0667b0 msgr2=0x7f731c0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:07.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f73221a6700 1 -- 172.21.15.203:0/3703833497 shutdown_connections 2021-01-06T22:16:07.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.452+0000 7f73221a6700 1 -- 172.21.15.203:0/3703833497 wait complete. 2021-01-06T22:16:07.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73221a6700 1 Processor -- start 2021-01-06T22:16:07.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73221a6700 1 -- start start 2021-01-06T22:16:07.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73221a6700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f731c0602b0 0x7f731c14fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:07.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73221a6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f731c148d70 0x7f731c14aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:07.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73221a6700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f731c149150 0x7f731c145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:07.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73221a6700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0be0e0 con 0x7f731c0602b0 2021-01-06T22:16:07.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73221a6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c14b8c0 con 0x7f731c148d70 2021-01-06T22:16:07.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73221a6700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0b7d40 con 0x7f731c149150 2021-01-06T22:16:07.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73209a3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f731c148d70 0x7f731c14aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:07.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73209a3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f731c148d70 0x7f731c14aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51958/0 (socket says 172.21.15.203:51958) 2021-01-06T22:16:07.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73209a3700 1 -- 172.21.15.203:0/648482709 learned_addr learned my addr 172.21.15.203:0/648482709 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:07.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73211a4700 1 --2- 172.21.15.203:0/648482709 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f731c0602b0 0x7f731c14fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:07.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73219a5700 1 --2- 172.21.15.203:0/648482709 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f731c149150 0x7f731c145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:07.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73209a3700 1 -- 172.21.15.203:0/648482709 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f731c149150 msgr2=0x7f731c145d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:07.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73209a3700 1 --2- 172.21.15.203:0/648482709 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f731c149150 0x7f731c145d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:07.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73209a3700 1 -- 172.21.15.203:0/648482709 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f731c0602b0 msgr2=0x7f731c14fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:07.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73209a3700 1 --2- 172.21.15.203:0/648482709 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f731c0602b0 0x7f731c14fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:07.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73209a3700 1 -- 172.21.15.203:0/648482709 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c15e3c0 con 0x7f731c148d70 2021-01-06T22:16:07.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73209a3700 1 --2- 172.21.15.203:0/648482709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f731c148d70 0x7f731c14aca0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f730c0017a0 tx=0x7f730c0014c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:07.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73097fa700 1 -- 172.21.15.203:0/648482709 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c007320 con 0x7f731c148d70 2021-01-06T22:16:07.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73097fa700 1 -- 172.21.15.203:0/648482709 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730c007480 con 0x7f731c148d70 2021-01-06T22:16:07.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73097fa700 1 -- 172.21.15.203:0/648482709 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c005830 con 0x7f731c148d70 2021-01-06T22:16:07.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73221a6700 1 -- 172.21.15.203:0/648482709 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731c15e660 con 0x7f731c148d70 2021-01-06T22:16:07.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.456+0000 7f73221a6700 1 -- 172.21.15.203:0/648482709 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731c0bdde0 con 0x7f731c148d70 2021-01-06T22:16:07.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.460+0000 7f73221a6700 1 -- 172.21.15.203:0/648482709 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731c14f750 con 0x7f731c148d70 2021-01-06T22:16:07.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.460+0000 7f73097fa700 1 -- 172.21.15.203:0/648482709 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f730c010030 con 0x7f731c148d70 2021-01-06T22:16:07.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.460+0000 7f73097fa700 1 --2- 172.21.15.203:0/648482709 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72fc0563d0 0x7f72fc058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:07.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.460+0000 7f73211a4700 1 --2- 172.21.15.203:0/648482709 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72fc0563d0 0x7f72fc058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:07.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.460+0000 7f73097fa700 1 -- 172.21.15.203:0/648482709 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6589+0+0 (secure 0 0 0) 0x7f730c06f630 con 0x7f731c148d70 2021-01-06T22:16:07.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.460+0000 7f73211a4700 1 --2- 172.21.15.203:0/648482709 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72fc0563d0 0x7f72fc058890 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f7318006e20 tx=0x7f7318016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:07.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.468+0000 7f73097fa700 1 -- 172.21.15.203:0/648482709 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f731c14f750 con 0x7f731c148d70 2021-01-06T22:16:07.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.692+0000 7f73221a6700 1 -- 172.21.15.203:0/648482709 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f731c1495a0 con 0x7f731c148d70 2021-01-06T22:16:07.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.692+0000 7f73097fa700 1 -- 172.21.15.203:0/648482709 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3563 (secure 0 0 0) 0x7f731c1495a0 con 0x7f731c148d70 2021-01-06T22:16:07.696 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:07.697 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5364,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:14:20.559082+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:07.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.692+0000 7f73221a6700 1 -- 172.21.15.203:0/648482709 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72fc0563d0 msgr2=0x7f72fc058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:07.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.692+0000 7f73221a6700 1 --2- 172.21.15.203:0/648482709 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72fc0563d0 0x7f72fc058890 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f7318006e20 tx=0x7f7318016000).stop 2021-01-06T22:16:07.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.692+0000 7f73221a6700 1 -- 172.21.15.203:0/648482709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f731c148d70 msgr2=0x7f731c14aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:07.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.692+0000 7f73221a6700 1 --2- 172.21.15.203:0/648482709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f731c148d70 0x7f731c14aca0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f730c0017a0 tx=0x7f730c0014c0).stop 2021-01-06T22:16:07.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.692+0000 7f73221a6700 1 -- 172.21.15.203:0/648482709 shutdown_connections 2021-01-06T22:16:07.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.692+0000 7f73221a6700 1 --2- 172.21.15.203:0/648482709 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f72fc0563d0 0x7f72fc058890 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:07.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.692+0000 7f73221a6700 1 --2- 172.21.15.203:0/648482709 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f731c149150 0x7f731c145d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:07.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.692+0000 7f73221a6700 1 --2- 172.21.15.203:0/648482709 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f731c0602b0 0x7f731c14fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:07.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.692+0000 7f73221a6700 1 --2- 172.21.15.203:0/648482709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f731c148d70 0x7f731c14aca0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:07.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.692+0000 7f73221a6700 1 -- 172.21.15.203:0/648482709 >> 172.21.15.203:0/648482709 conn(0x7f731c0667b0 msgr2=0x7f731c0c0870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:07.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.692+0000 7f73221a6700 1 -- 172.21.15.203:0/648482709 shutdown_connections 2021-01-06T22:16:07.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:07.692+0000 7f73221a6700 1 -- 172.21.15.203:0/648482709 wait complete. 2021-01-06T22:16:07.705 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 56 2021-01-06T22:16:07.711 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:16:09.713 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:09.845 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.840+0000 7f2ccb5d0700 1 Processor -- start 2021-01-06T22:16:09.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.840+0000 7f2ccb5d0700 1 -- start start 2021-01-06T22:16:09.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.840+0000 7f2ccb5d0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cc40bb1e0 0x7f2cc414bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:09.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.840+0000 7f2ccb5d0700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2cc4067ae0 0x7f2cc4067f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:09.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.840+0000 7f2ccb5d0700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc4068460 con 0x7f2cc40bae00 2021-01-06T22:16:09.849 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.840+0000 7f2ccb5d0700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc40bcd60 con 0x7f2cc4067ae0 2021-01-06T22:16:09.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.840+0000 7f2ccb5d0700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc4061030 con 0x7f2cc40bb1e0 2021-01-06T22:16:09.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.840+0000 7f2cc936c700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f2cc40bae00 0x7f2cc4060920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:45212/0 (socket says 172.21.15.203:45212) 2021-01-06T22:16:09.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.840+0000 7f2cc936c700 1 -- 172.21.15.203:0/3466683865 learned_addr learned my addr 172.21.15.203:0/3466683865 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:16:09.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.840+0000 7f2cc8b6b700 1 --2- 172.21.15.203:0/3466683865 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cc40bb1e0 0x7f2cc414bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:09.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.840+0000 7f2cb3fff700 1 -- 172.21.15.203:0/3466683865 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 216861513 0 0) 0x7f2cc4068460 con 0x7f2cc40bae00 2021-01-06T22:16:09.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.840+0000 7f2cb3fff700 1 -- 172.21.15.203:0/3466683865 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cac003230 con 0x7f2cc40bae00 2021-01-06T22:16:09.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.844+0000 7f2cc8b6b700 1 -- 172.21.15.203:0/3466683865 >> v2:172.21.15.203:3300/0 conn(0x7f2cc4067ae0 msgr2=0x7f2cc4067f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-01-06T22:16:09.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.844+0000 7f2cc8b6b700 1 --2- 172.21.15.203:0/3466683865 >> v2:172.21.15.203:3300/0 conn(0x7f2cc4067ae0 0x7f2cc4067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:16:09.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.844+0000 7f2cc8b6b700 1 -- 172.21.15.203:0/3466683865 >> v1:172.21.15.91:6789/0 conn(0x7f2cc40bae00 legacy=0x7f2cc4060920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:09.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.844+0000 7f2cc8b6b700 1 -- 172.21.15.203:0/3466683865 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc414c610 con 0x7f2cc40bb1e0 2021-01-06T22:16:09.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.844+0000 7f2cc8b6b700 1 --2- 172.21.15.203:0/3466683865 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cc40bb1e0 0x7f2cc414bf80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f2cb400b9b0 tx=0x7f2cb4006de0).ready entity=mon.2 client_cookie=666782471197aecf server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:09.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.844+0000 7f2cb3fff700 1 -- 172.21.15.203:0/3466683865 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4009800 con 0x7f2cc40bb1e0 2021-01-06T22:16:09.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.844+0000 7f2cb3fff700 1 -- 172.21.15.203:0/3466683865 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb4009960 con 0x7f2cc40bb1e0 2021-01-06T22:16:09.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.844+0000 7f2cb3fff700 1 -- 172.21.15.203:0/3466683865 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4009ce0 con 0x7f2cc40bb1e0 2021-01-06T22:16:09.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.844+0000 7f2ccb5d0700 1 -- 172.21.15.203:0/3466683865 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cc40bb1e0 msgr2=0x7f2cc414bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:09.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.844+0000 7f2ccb5d0700 1 --2- 172.21.15.203:0/3466683865 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cc40bb1e0 0x7f2cc414bf80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f2cb400b9b0 tx=0x7f2cb4006de0).stop 2021-01-06T22:16:09.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.844+0000 7f2ccb5d0700 1 -- 172.21.15.203:0/3466683865 shutdown_connections 2021-01-06T22:16:09.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.844+0000 7f2ccb5d0700 1 --2- 172.21.15.203:0/3466683865 >> v2:172.21.15.203:3300/0 conn(0x7f2cc4067ae0 0x7f2cc4067f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:16:09.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.844+0000 7f2ccb5d0700 1 --2- 172.21.15.203:0/3466683865 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cc40bb1e0 0x7f2cc414bf80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:09.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.844+0000 7f2ccb5d0700 1 -- 172.21.15.203:0/3466683865 >> 172.21.15.203:0/3466683865 conn(0x7f2cc4056cd0 msgr2=0x7f2cc40570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:09.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.844+0000 7f2ccb5d0700 1 -- 172.21.15.203:0/3466683865 shutdown_connections 2021-01-06T22:16:09.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.844+0000 7f2ccb5d0700 1 -- 172.21.15.203:0/3466683865 wait complete. 2021-01-06T22:16:09.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2ccb5d0700 1 Processor -- start 2021-01-06T22:16:09.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2ccb5d0700 1 -- start start 2021-01-06T22:16:09.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2ccb5d0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cc4067ae0 0x7f2cc415e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:09.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2ccb5d0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cc40bae00 0x7f2cc415f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:09.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2ccb5d0700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cc40bb1e0 0x7f2cc4140a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:09.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2ccb5d0700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc4150630 con 0x7f2cc40bb1e0 2021-01-06T22:16:09.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2ccb5d0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc41508d0 con 0x7f2cc40bae00 2021-01-06T22:16:09.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2ccb5d0700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc4143a10 con 0x7f2cc4067ae0 2021-01-06T22:16:09.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2cc8b6b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cc40bae00 0x7f2cc415f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:09.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2cc8b6b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cc40bae00 0x7f2cc415f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51970/0 (socket says 172.21.15.203:51970) 2021-01-06T22:16:09.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2cc8b6b700 1 -- 172.21.15.203:0/861873328 learned_addr learned my addr 172.21.15.203:0/861873328 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:09.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2cc936c700 1 --2- 172.21.15.203:0/861873328 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cc4067ae0 0x7f2cc415e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:09.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2cc9b6d700 1 --2- 172.21.15.203:0/861873328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cc40bb1e0 0x7f2cc4140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:09.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2cc8b6b700 1 -- 172.21.15.203:0/861873328 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cc4067ae0 msgr2=0x7f2cc415e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:09.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2cc8b6b700 1 --2- 172.21.15.203:0/861873328 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cc4067ae0 0x7f2cc415e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:09.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2cc8b6b700 1 -- 172.21.15.203:0/861873328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cc40bb1e0 msgr2=0x7f2cc4140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:09.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2cc8b6b700 1 --2- 172.21.15.203:0/861873328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cc40bb1e0 0x7f2cc4140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:09.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2cc8b6b700 1 -- 172.21.15.203:0/861873328 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc4143cb0 con 0x7f2cc40bae00 2021-01-06T22:16:09.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2cc8b6b700 1 --2- 172.21.15.203:0/861873328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cc40bae00 0x7f2cc415f130 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f2cb4006e20 tx=0x7f2cb4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:09.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.848+0000 7f2cb17fa700 1 -- 172.21.15.203:0/861873328 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4012e60 con 0x7f2cc40bae00 2021-01-06T22:16:09.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.852+0000 7f2cb17fa700 1 -- 172.21.15.203:0/861873328 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb401c420 con 0x7f2cc40bae00 2021-01-06T22:16:09.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.852+0000 7f2cb17fa700 1 -- 172.21.15.203:0/861873328 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb401c7c0 con 0x7f2cc40bae00 2021-01-06T22:16:09.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.852+0000 7f2ccb5d0700 1 -- 172.21.15.203:0/861873328 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc4143f50 con 0x7f2cc40bae00 2021-01-06T22:16:09.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.852+0000 7f2ccb5d0700 1 -- 172.21.15.203:0/861873328 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc415ee50 con 0x7f2cc40bae00 2021-01-06T22:16:09.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.852+0000 7f2ccb5d0700 1 -- 172.21.15.203:0/861873328 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc41443f0 con 0x7f2cc40bae00 2021-01-06T22:16:09.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.852+0000 7f2cb17fa700 1 -- 172.21.15.203:0/861873328 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f2cb400f070 con 0x7f2cc40bae00 2021-01-06T22:16:09.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.852+0000 7f2cb17fa700 1 --2- 172.21.15.203:0/861873328 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2ca40563d0 0x7f2ca4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:09.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.852+0000 7f2cb17fa700 1 -- 172.21.15.203:0/861873328 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6589+0+0 (secure 0 0 0) 0x7f2cb4070eb0 con 0x7f2cc40bae00 2021-01-06T22:16:09.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.852+0000 7f2cc936c700 1 --2- 172.21.15.203:0/861873328 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2ca40563d0 0x7f2ca4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:09.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.852+0000 7f2cc936c700 1 --2- 172.21.15.203:0/861873328 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2ca40563d0 0x7f2ca4058890 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f2cc0005590 tx=0x7f2cc0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:09.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:09.860+0000 7f2cb17fa700 1 -- 172.21.15.203:0/861873328 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f2cc41443f0 con 0x7f2cc40bae00 2021-01-06T22:16:10.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:10.092+0000 7f2ccb5d0700 1 -- 172.21.15.203:0/861873328 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cc405d600 con 0x7f2cc40bae00 2021-01-06T22:16:10.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:10.092+0000 7f2cb17fa700 1 -- 172.21.15.203:0/861873328 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3563 (secure 0 0 0) 0x7f2cb4046530 con 0x7f2cc40bae00 2021-01-06T22:16:10.098 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:10.099 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5364,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:14:20.559082+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:10.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:10.096+0000 7f2ccb5d0700 1 -- 172.21.15.203:0/861873328 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2ca40563d0 msgr2=0x7f2ca4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:10.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:10.096+0000 7f2ccb5d0700 1 --2- 172.21.15.203:0/861873328 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2ca40563d0 0x7f2ca4058890 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f2cc0005590 tx=0x7f2cc0009040).stop 2021-01-06T22:16:10.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:10.096+0000 7f2ccb5d0700 1 -- 172.21.15.203:0/861873328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cc40bae00 msgr2=0x7f2cc415f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:10.101 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:10.096+0000 7f2ccb5d0700 1 --2- 172.21.15.203:0/861873328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cc40bae00 0x7f2cc415f130 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f2cb4006e20 tx=0x7f2cb4016000).stop 2021-01-06T22:16:10.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:10.096+0000 7f2ccb5d0700 1 -- 172.21.15.203:0/861873328 shutdown_connections 2021-01-06T22:16:10.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:10.096+0000 7f2ccb5d0700 1 --2- 172.21.15.203:0/861873328 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2ca40563d0 0x7f2ca4058890 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:10.102 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:10.096+0000 7f2ccb5d0700 1 --2- 172.21.15.203:0/861873328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cc40bb1e0 0x7f2cc4140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:10.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:10.096+0000 7f2ccb5d0700 1 --2- 172.21.15.203:0/861873328 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cc4067ae0 0x7f2cc415e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:10.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:10.096+0000 7f2ccb5d0700 1 --2- 172.21.15.203:0/861873328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cc40bae00 0x7f2cc415f130 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:10.103 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:10.096+0000 7f2ccb5d0700 1 -- 172.21.15.203:0/861873328 >> 172.21.15.203:0/861873328 conn(0x7f2cc4056cd0 msgr2=0x7f2cc40be0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:10.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:10.096+0000 7f2ccb5d0700 1 -- 172.21.15.203:0/861873328 shutdown_connections 2021-01-06T22:16:10.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:10.096+0000 7f2ccb5d0700 1 -- 172.21.15.203:0/861873328 wait complete. 2021-01-06T22:16:10.104 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 56 2021-01-06T22:16:10.115 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:16:12.117 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.236+0000 7f79819ba700 1 Processor -- start 2021-01-06T22:16:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.236+0000 7f79819ba700 1 -- start start 2021-01-06T22:16:12.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f79819ba700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f797c0676d0 0x7f797c0682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:12.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f79819ba700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c14a7b0 con 0x7f797c067ab0 2021-01-06T22:16:12.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f79819ba700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c061110 con 0x7f797c0602b0 2021-01-06T22:16:12.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f79819ba700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f797c0613b0 con 0x7f797c0676d0 2021-01-06T22:16:12.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f797b7fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f797c0602b0 0x7f797c1476a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50574/0 (socket says 172.21.15.203:50574) 2021-01-06T22:16:12.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f797b7fe700 1 -- 172.21.15.203:0/663323253 learned_addr learned my addr 172.21.15.203:0/663323253 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:16:12.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f797affd700 1 --2- 172.21.15.203:0/663323253 >> v2:172.21.15.91:3300/0 conn(0x7f797c0676d0 0x7f797c0682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:12.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 -- 172.21.15.203:0/663323253 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899491862 0 0) 0x7f797c061110 con 0x7f797c0602b0 2021-01-06T22:16:12.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 -- 172.21.15.203:0/663323253 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7968003230 con 0x7f797c0602b0 2021-01-06T22:16:12.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 -- 172.21.15.203:0/663323253 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895870724 0 0) 0x7f79700010d0 con 0x7f797c067ab0 2021-01-06T22:16:12.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 -- 172.21.15.203:0/663323253 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f797c061110 con 0x7f797c067ab0 2021-01-06T22:16:12.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 -- 172.21.15.203:0/663323253 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 445124914 0 0) 0x7f7968003230 con 0x7f797c0602b0 2021-01-06T22:16:12.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 -- 172.21.15.203:0/663323253 >> v2:172.21.15.91:3300/0 conn(0x7f797c0676d0 msgr2=0x7f797c0682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:12.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 --2- 172.21.15.203:0/663323253 >> v2:172.21.15.91:3300/0 conn(0x7f797c0676d0 0x7f797c0682c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:12.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 -- 172.21.15.203:0/663323253 >> v1:172.21.15.91:6789/0 conn(0x7f797c067ab0 legacy=0x7f797c14ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:12.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 -- 172.21.15.203:0/663323253 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c061650 con 0x7f797c0602b0 2021-01-06T22:16:12.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 -- 172.21.15.203:0/663323253 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f7974001860 con 0x7f797c0602b0 2021-01-06T22:16:12.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 -- 172.21.15.203:0/663323253 >> v1:172.21.15.203:6789/0 conn(0x7f797c0602b0 legacy=0x7f797c1476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:12.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 --2- 172.21.15.203:0/663323253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7968003900 0x7f7968003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:12.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 --2- 172.21.15.203:0/663323253 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79680044d0 0x7f7968004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:12.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 --2- 172.21.15.203:0/663323253 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7968004e40 0x7f796800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:12.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 -- 172.21.15.203:0/663323253 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0613b0 con 0x7f79680044d0 2021-01-06T22:16:12.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 -- 172.21.15.203:0/663323253 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796800de20 con 0x7f7968003900 2021-01-06T22:16:12.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f7979ffb700 1 -- 172.21.15.203:0/663323253 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796800bc10 con 0x7f7968004e40 2021-01-06T22:16:12.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f797affd700 1 --2- 172.21.15.203:0/663323253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7968003900 0x7f7968003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:12.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f797a7fc700 1 --2- 172.21.15.203:0/663323253 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79680044d0 0x7f7968004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:12.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f797affd700 1 -- 172.21.15.203:0/663323253 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7968004e40 msgr2=0x7f796800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:12.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f797affd700 1 --2- 172.21.15.203:0/663323253 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7968004e40 0x7f796800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:16:12.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f797affd700 1 -- 172.21.15.203:0/663323253 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79680044d0 msgr2=0x7f7968004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:12.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f797affd700 1 --2- 172.21.15.203:0/663323253 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79680044d0 0x7f7968004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:12.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f797affd700 1 -- 172.21.15.203:0/663323253 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7968003230 con 0x7f7968003900 2021-01-06T22:16:12.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f797b7fe700 1 -- 172.21.15.203:0/663323253 reap_dead start 2021-01-06T22:16:12.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.240+0000 7f797affd700 1 --2- 172.21.15.203:0/663323253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7968003900 0x7f7968003d00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f796c00cb80 tx=0x7f796c006de0).ready entity=mon.1 client_cookie=2e41d2af8dd767fa server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:12.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f7979ffb700 1 -- 172.21.15.203:0/663323253 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c00eb30 con 0x7f7968003900 2021-01-06T22:16:12.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f7979ffb700 1 -- 172.21.15.203:0/663323253 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c00ec90 con 0x7f7968003900 2021-01-06T22:16:12.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f7979ffb700 1 -- 172.21.15.203:0/663323253 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c0113f0 con 0x7f7968003900 2021-01-06T22:16:12.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f79819ba700 1 -- 172.21.15.203:0/663323253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7968003900 msgr2=0x7f7968003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:12.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f79819ba700 1 --2- 172.21.15.203:0/663323253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7968003900 0x7f7968003d00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f796c00cb80 tx=0x7f796c006de0).stop 2021-01-06T22:16:12.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f79819ba700 1 -- 172.21.15.203:0/663323253 shutdown_connections 2021-01-06T22:16:12.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f79819ba700 1 --2- 172.21.15.203:0/663323253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7968003900 0x7f7968003d00 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:12.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f79819ba700 1 -- 172.21.15.203:0/663323253 >> 172.21.15.203:0/663323253 conn(0x7f797c0c0c50 msgr2=0x7f797c0c1030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:12.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f79819ba700 1 -- 172.21.15.203:0/663323253 shutdown_connections 2021-01-06T22:16:12.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f79819ba700 1 -- 172.21.15.203:0/663323253 wait complete. 2021-01-06T22:16:12.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f79819ba700 1 Processor -- start 2021-01-06T22:16:12.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f79819ba700 1 -- start start 2021-01-06T22:16:12.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f79819ba700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7968003900 0x7f797c15d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:12.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f79819ba700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f797c159820 0x7f797c1464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:12.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f79819ba700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c0bf620 0x7f797c14f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:12.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f79819ba700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c15dc70 con 0x7f7968003900 2021-01-06T22:16:12.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f79819ba700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0b4e00 con 0x7f797c0bf620 2021-01-06T22:16:12.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.244+0000 7f79819ba700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0bda10 con 0x7f797c159820 2021-01-06T22:16:12.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f797b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c0bf620 0x7f797c14f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:12.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f797b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c0bf620 0x7f797c14f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51992/0 (socket says 172.21.15.203:51992) 2021-01-06T22:16:12.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f797b7fe700 1 -- 172.21.15.203:0/184875384 learned_addr learned my addr 172.21.15.203:0/184875384 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:12.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f797affd700 1 --2- 172.21.15.203:0/184875384 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7968003900 0x7f797c15d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:12.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f797a7fc700 1 --2- 172.21.15.203:0/184875384 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f797c159820 0x7f797c1464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:12.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f797b7fe700 1 -- 172.21.15.203:0/184875384 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f797c159820 msgr2=0x7f797c1464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:12.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f797b7fe700 1 --2- 172.21.15.203:0/184875384 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f797c159820 0x7f797c1464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:12.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f797b7fe700 1 -- 172.21.15.203:0/184875384 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7968003900 msgr2=0x7f797c15d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:12.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f797b7fe700 1 --2- 172.21.15.203:0/184875384 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7968003900 0x7f797c15d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:12.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f797b7fe700 1 -- 172.21.15.203:0/184875384 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c0b5060 con 0x7f797c0bf620 2021-01-06T22:16:12.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f797b7fe700 1 --2- 172.21.15.203:0/184875384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c0bf620 0x7f797c14f040 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f797c066090 tx=0x7f797c0617e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:12.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f7962ffd700 1 -- 172.21.15.203:0/184875384 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974004a80 con 0x7f797c0bf620 2021-01-06T22:16:12.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f7962ffd700 1 -- 172.21.15.203:0/184875384 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7974004c20 con 0x7f797c0bf620 2021-01-06T22:16:12.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f79819ba700 1 -- 172.21.15.203:0/184875384 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797c0bf180 con 0x7f797c0bf620 2021-01-06T22:16:12.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f7962ffd700 1 -- 172.21.15.203:0/184875384 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79740070d0 con 0x7f797c0bf620 2021-01-06T22:16:12.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f79819ba700 1 -- 172.21.15.203:0/184875384 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c0bdcb0 con 0x7f797c0bf620 2021-01-06T22:16:12.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f7962ffd700 1 -- 172.21.15.203:0/184875384 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f797401c070 con 0x7f797c0bf620 2021-01-06T22:16:12.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f79819ba700 1 -- 172.21.15.203:0/184875384 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797c05d600 con 0x7f797c0bf620 2021-01-06T22:16:12.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.248+0000 7f7962ffd700 1 --2- 172.21.15.203:0/184875384 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f795c0563d0 0x7f795c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:12.260 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.252+0000 7f797affd700 1 --2- 172.21.15.203:0/184875384 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f795c0563d0 0x7f795c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:12.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.252+0000 7f7962ffd700 1 -- 172.21.15.203:0/184875384 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6589+0+0 (secure 0 0 0) 0x7f797406e9d0 con 0x7f797c0bf620 2021-01-06T22:16:12.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.252+0000 7f797affd700 1 --2- 172.21.15.203:0/184875384 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f795c0563d0 0x7f795c058890 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f796c006cf0 tx=0x7f796c01a000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:12.261 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.256+0000 7f7962ffd700 1 -- 172.21.15.203:0/184875384 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f7974044050 con 0x7f797c0bf620 2021-01-06T22:16:12.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.496+0000 7f79819ba700 1 -- 172.21.15.203:0/184875384 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f797c0646f0 con 0x7f797c0bf620 2021-01-06T22:16:12.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.496+0000 7f7962ffd700 1 -- 172.21.15.203:0/184875384 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3563 (secure 0 0 0) 0x7f797401a030 con 0x7f797c0bf620 2021-01-06T22:16:12.497 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:12.498 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5364,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:14:20.559082+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:12.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.496+0000 7f79819ba700 1 -- 172.21.15.203:0/184875384 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f795c0563d0 msgr2=0x7f795c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:12.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.496+0000 7f79819ba700 1 --2- 172.21.15.203:0/184875384 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f795c0563d0 0x7f795c058890 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f796c006cf0 tx=0x7f796c01a000).stop 2021-01-06T22:16:12.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.496+0000 7f79819ba700 1 -- 172.21.15.203:0/184875384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c0bf620 msgr2=0x7f797c14f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:12.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.496+0000 7f79819ba700 1 --2- 172.21.15.203:0/184875384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c0bf620 0x7f797c14f040 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f797c066090 tx=0x7f797c0617e0).stop 2021-01-06T22:16:12.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.496+0000 7f79819ba700 1 -- 172.21.15.203:0/184875384 shutdown_connections 2021-01-06T22:16:12.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.496+0000 7f79819ba700 1 --2- 172.21.15.203:0/184875384 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f795c0563d0 0x7f795c058890 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:12.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.496+0000 7f79819ba700 1 --2- 172.21.15.203:0/184875384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c0bf620 0x7f797c14f040 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:12.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.496+0000 7f79819ba700 1 --2- 172.21.15.203:0/184875384 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7968003900 0x7f797c15d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:12.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.496+0000 7f79819ba700 1 --2- 172.21.15.203:0/184875384 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f797c159820 0x7f797c1464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:12.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.496+0000 7f79819ba700 1 -- 172.21.15.203:0/184875384 >> 172.21.15.203:0/184875384 conn(0x7f797c0c0c50 msgr2=0x7f797c062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:12.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.496+0000 7f79819ba700 1 -- 172.21.15.203:0/184875384 shutdown_connections 2021-01-06T22:16:12.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:12.496+0000 7f79819ba700 1 -- 172.21.15.203:0/184875384 wait complete. 2021-01-06T22:16:12.503 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 56 2021-01-06T22:16:12.511 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:16:14.513 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:14.655 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.651+0000 7f818e1ba700 1 Processor -- start 2021-01-06T22:16:14.655 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.651+0000 7f818e1ba700 1 -- start start 2021-01-06T22:16:14.657 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818e1ba700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81880c1a90 0x7f8188150730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:14.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818e1ba700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f81880c48d0 0x7f8188150c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:14.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818e1ba700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81880c0f50 con 0x7f81880c44f0 2021-01-06T22:16:14.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818e1ba700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8188066cf0 con 0x7f81880c48d0 2021-01-06T22:16:14.659 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818e1ba700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8188060750 con 0x7f81880c1a90 2021-01-06T22:16:14.659 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818d1b8700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f81880c44f0 0x7f81880c1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50600/0 (socket says 172.21.15.203:50600) 2021-01-06T22:16:14.659 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818d1b8700 1 -- 172.21.15.203:0/1244997490 learned_addr learned my addr 172.21.15.203:0/1244997490 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:16:14.659 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818d9b9700 1 --2- 172.21.15.203:0/1244997490 >> v2:172.21.15.91:3300/0 conn(0x7f81880c48d0 0x7f8188150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:14.660 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818c9b7700 1 --2- 172.21.15.203:0/1244997490 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81880c1a90 0x7f8188150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:14.662 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f817ffff700 1 -- 172.21.15.203:0/1244997490 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155531096 0 0) 0x7f81880c0f50 con 0x7f81880c44f0 2021-01-06T22:16:14.663 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f817ffff700 1 -- 172.21.15.203:0/1244997490 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f816c003230 con 0x7f81880c44f0 2021-01-06T22:16:14.663 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818d1b8700 1 -- 172.21.15.203:0/1244997490 >> v1:172.21.15.203:6789/0 conn(0x7f81880c44f0 legacy=0x7f81880c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2021-01-06T22:16:14.663 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818d1b8700 1 -- 172.21.15.203:0/1244997490 >> v1:172.21.15.203:6789/0 conn(0x7f81880c44f0 legacy=0x7f81880c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2021-01-06T22:16:14.664 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818d1b8700 1 --1- 172.21.15.203:0/1244997490 >> v1:172.21.15.203:6789/0 conn(0x7f81880c44f0 0x7f81880c1380 :-1 s=OPENED pgs=536 cs=1 l=1).handle_message read tag failed 2021-01-06T22:16:14.664 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818d1b8700 1 --1- 172.21.15.203:0/1244997490 >> v1:172.21.15.203:6789/0 conn(0x7f81880c44f0 0x7f81880c1380 :-1 s=OPENED pgs=536 cs=1 l=1).fault on lossy channel, failing 2021-01-06T22:16:14.664 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818c9b7700 1 -- 172.21.15.203:0/1244997490 >> v2:172.21.15.91:3300/0 conn(0x7f81880c48d0 msgr2=0x7f8188150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:14.664 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818c9b7700 1 --2- 172.21.15.203:0/1244997490 >> v2:172.21.15.91:3300/0 conn(0x7f81880c48d0 0x7f8188150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:14.665 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818c9b7700 1 -- 172.21.15.203:0/1244997490 >> v1:172.21.15.203:6789/0 conn(0x7f81880c44f0 legacy=0x7f81880c1380 unknown :-1 s=STATE_CLOSED l=1).mark_down 2021-01-06T22:16:14.665 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818c9b7700 1 -- 172.21.15.203:0/1244997490 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81880609f0 con 0x7f81880c1a90 2021-01-06T22:16:14.665 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818c9b7700 1 --2- 172.21.15.203:0/1244997490 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81880c1a90 0x7f8188150730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f81880c54b0 tx=0x7f8178006de0).ready entity=mon.2 client_cookie=fc8423409c4048f2 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:14.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f817ffff700 1 -- 172.21.15.203:0/1244997490 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81780095c0 con 0x7f81880c1a90 2021-01-06T22:16:14.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f817ffff700 1 -- 172.21.15.203:0/1244997490 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8178009720 con 0x7f81880c1a90 2021-01-06T22:16:14.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f817ffff700 1 -- 172.21.15.203:0/1244997490 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178009aa0 con 0x7f81880c1a90 2021-01-06T22:16:14.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.655+0000 7f818e1ba700 1 -- 172.21.15.203:0/1244997490 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81880c1a90 msgr2=0x7f8188150730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:14.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.659+0000 7f818e1ba700 1 --2- 172.21.15.203:0/1244997490 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81880c1a90 0x7f8188150730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f81880c54b0 tx=0x7f8178006de0).stop 2021-01-06T22:16:14.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.659+0000 7f818e1ba700 1 -- 172.21.15.203:0/1244997490 shutdown_connections 2021-01-06T22:16:14.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.659+0000 7f818e1ba700 1 --2- 172.21.15.203:0/1244997490 >> v2:172.21.15.91:3300/0 conn(0x7f81880c48d0 0x7f8188150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:14.668 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.659+0000 7f818e1ba700 1 --2- 172.21.15.203:0/1244997490 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81880c1a90 0x7f8188150730 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:14.668 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.659+0000 7f818e1ba700 1 -- 172.21.15.203:0/1244997490 >> 172.21.15.203:0/1244997490 conn(0x7f8188056cd0 msgr2=0x7f8188061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:14.668 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.659+0000 7f818e1ba700 1 -- 172.21.15.203:0/1244997490 shutdown_connections 2021-01-06T22:16:14.668 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.659+0000 7f818e1ba700 1 -- 172.21.15.203:0/1244997490 wait complete. 2021-01-06T22:16:14.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.659+0000 7f818e1ba700 1 Processor -- start 2021-01-06T22:16:14.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.659+0000 7f818e1ba700 1 -- start start 2021-01-06T22:16:14.672 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818e1ba700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81880c1a90 0x7f8188152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:14.672 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818e1ba700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81880c44f0 0x7f81880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:14.673 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818e1ba700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81880c48d0 0x7f8188161620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:14.673 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818e1ba700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8188161b10 con 0x7f81880c1a90 2021-01-06T22:16:14.673 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818e1ba700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81880a53b0 con 0x7f81880c48d0 2021-01-06T22:16:14.673 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818e1ba700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81880a5650 con 0x7f81880c44f0 2021-01-06T22:16:14.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818d9b9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81880c48d0 0x7f8188161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:14.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818d9b9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81880c48d0 0x7f8188161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52010/0 (socket says 172.21.15.203:52010) 2021-01-06T22:16:14.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818d9b9700 1 -- 172.21.15.203:0/4262718860 learned_addr learned my addr 172.21.15.203:0/4262718860 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:14.675 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818d1b8700 1 --2- 172.21.15.203:0/4262718860 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81880c1a90 0x7f8188152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:14.675 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818c9b7700 1 --2- 172.21.15.203:0/4262718860 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81880c44f0 0x7f81880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:14.675 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818d9b9700 1 -- 172.21.15.203:0/4262718860 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81880c44f0 msgr2=0x7f81880a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:14.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818d9b9700 1 --2- 172.21.15.203:0/4262718860 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81880c44f0 0x7f81880a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:14.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818d9b9700 1 -- 172.21.15.203:0/4262718860 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81880c1a90 msgr2=0x7f8188152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:14.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818d9b9700 1 --2- 172.21.15.203:0/4262718860 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81880c1a90 0x7f8188152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:14.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818d9b9700 1 -- 172.21.15.203:0/4262718860 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81880a2d90 con 0x7f81880c48d0 2021-01-06T22:16:14.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818d9b9700 1 --2- 172.21.15.203:0/4262718860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81880c48d0 0x7f8188161620 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f818000ef20 tx=0x7f81800091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:14.677 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f817d7fa700 1 -- 172.21.15.203:0/4262718860 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81800096a0 con 0x7f81880c48d0 2021-01-06T22:16:14.678 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f817d7fa700 1 -- 172.21.15.203:0/4262718860 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818000cb90 con 0x7f81880c48d0 2021-01-06T22:16:14.678 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818e1ba700 1 -- 172.21.15.203:0/4262718860 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81880a3030 con 0x7f81880c48d0 2021-01-06T22:16:14.678 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f817d7fa700 1 -- 172.21.15.203:0/4262718860 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81800123f0 con 0x7f81880c48d0 2021-01-06T22:16:14.679 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818e1ba700 1 -- 172.21.15.203:0/4262718860 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8188151590 con 0x7f81880c48d0 2021-01-06T22:16:14.679 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f817d7fa700 1 -- 172.21.15.203:0/4262718860 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f8180022070 con 0x7f81880c48d0 2021-01-06T22:16:14.679 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.663+0000 7f818e1ba700 1 -- 172.21.15.203:0/4262718860 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81880a34d0 con 0x7f81880c48d0 2021-01-06T22:16:14.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.667+0000 7f817d7fa700 1 --2- 172.21.15.203:0/4262718860 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f81680563d0 0x7f8168058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:14.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.667+0000 7f817d7fa700 1 -- 172.21.15.203:0/4262718860 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6589+0+0 (secure 0 0 0) 0x7f818006ff10 con 0x7f81880c48d0 2021-01-06T22:16:14.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.667+0000 7f818d1b8700 1 --2- 172.21.15.203:0/4262718860 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f81680563d0 0x7f8168058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:14.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.667+0000 7f818d1b8700 1 --2- 172.21.15.203:0/4262718860 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f81680563d0 0x7f8168058890 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f81880be5d0 tx=0x7f8184009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:14.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.671+0000 7f817d7fa700 1 -- 172.21.15.203:0/4262718860 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f81880a34d0 con 0x7f81880c48d0 2021-01-06T22:16:14.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.887+0000 7f818e1ba700 1 -- 172.21.15.203:0/4262718860 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81880a34d0 con 0x7f81880c48d0 2021-01-06T22:16:14.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.887+0000 7f817d7fa700 1 -- 172.21.15.203:0/4262718860 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3563 (secure 0 0 0) 0x7f81880a34d0 con 0x7f81880c48d0 2021-01-06T22:16:14.891 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:14.892 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5364,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:14:20.559082+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5068},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5068":{"gid":5068,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6835/1477297469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1477297469},{"type":"v1","addr":"172.21.15.203:6835","nonce":1477297469}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:14.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.891+0000 7f818e1ba700 1 -- 172.21.15.203:0/4262718860 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f81680563d0 msgr2=0x7f8168058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:14.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.891+0000 7f818e1ba700 1 --2- 172.21.15.203:0/4262718860 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f81680563d0 0x7f8168058890 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f81880be5d0 tx=0x7f8184009040).stop 2021-01-06T22:16:14.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.891+0000 7f818e1ba700 1 -- 172.21.15.203:0/4262718860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81880c48d0 msgr2=0x7f8188161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:14.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.891+0000 7f818e1ba700 1 --2- 172.21.15.203:0/4262718860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81880c48d0 0x7f8188161620 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f818000ef20 tx=0x7f81800091d0).stop 2021-01-06T22:16:14.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.891+0000 7f818e1ba700 1 -- 172.21.15.203:0/4262718860 shutdown_connections 2021-01-06T22:16:14.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.891+0000 7f818e1ba700 1 --2- 172.21.15.203:0/4262718860 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f81680563d0 0x7f8168058890 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:14.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.891+0000 7f818e1ba700 1 --2- 172.21.15.203:0/4262718860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81880c48d0 0x7f8188161620 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:14.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.891+0000 7f818e1ba700 1 --2- 172.21.15.203:0/4262718860 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81880c1a90 0x7f8188152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:14.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.891+0000 7f818e1ba700 1 --2- 172.21.15.203:0/4262718860 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f81880c44f0 0x7f81880a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:14.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.891+0000 7f818e1ba700 1 -- 172.21.15.203:0/4262718860 >> 172.21.15.203:0/4262718860 conn(0x7f8188056cd0 msgr2=0x7f818814fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:14.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.891+0000 7f818e1ba700 1 -- 172.21.15.203:0/4262718860 shutdown_connections 2021-01-06T22:16:14.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:14.891+0000 7f818e1ba700 1 -- 172.21.15.203:0/4262718860 wait complete. 2021-01-06T22:16:14.896 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 56 2021-01-06T22:16:14.907 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:16:16.909 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:17.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b84981700 1 Processor -- start 2021-01-06T22:16:17.044 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b84981700 1 -- start start 2021-01-06T22:16:17.045 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b84981700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b80061130 0x7f8b800c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:17.045 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b84981700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8b8014bb30 0x7f8b80148d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:17.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b84981700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8014c1c0 con 0x7f8b800bc700 2021-01-06T22:16:17.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b84981700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8014ff90 con 0x7f8b8014bb30 2021-01-06T22:16:17.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b84981700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8014bfc0 con 0x7f8b80061130 2021-01-06T22:16:17.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b7f7fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b80061130 0x7f8b800c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:17.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b7f7fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b80061130 0x7f8b800c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:57098/0 (socket says 172.21.15.203:57098) 2021-01-06T22:16:17.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b7ffff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8b8014bb30 0x7f8b80148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:17.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b7f7fe700 1 -- 172.21.15.203:0/2339820938 learned_addr learned my addr 172.21.15.203:0/2339820938 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:17.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b7f7fe700 1 -- 172.21.15.203:0/2339820938 >> v2:172.21.15.91:3300/0 conn(0x7f8b8014bb30 msgr2=0x7f8b80148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:17.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b7f7fe700 1 --2- 172.21.15.203:0/2339820938 >> v2:172.21.15.91:3300/0 conn(0x7f8b8014bb30 0x7f8b80148d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:17.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b7f7fe700 1 -- 172.21.15.203:0/2339820938 >> v1:172.21.15.91:6789/0 conn(0x7f8b800bc700 legacy=0x7f8b8014f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:17.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b7f7fe700 1 -- 172.21.15.203:0/2339820938 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8014c460 con 0x7f8b80061130 2021-01-06T22:16:17.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b7f7fe700 1 --2- 172.21.15.203:0/2339820938 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b80061130 0x7f8b800c0fa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f8b7000b9e0 tx=0x7f8b70006ea0).ready entity=mon.2 client_cookie=73d641e51fa8fa42 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:17.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b7e7fc700 1 -- 172.21.15.203:0/2339820938 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b700098a0 con 0x7f8b80061130 2021-01-06T22:16:17.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b7e7fc700 1 -- 172.21.15.203:0/2339820938 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b70009a00 con 0x7f8b80061130 2021-01-06T22:16:17.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b7e7fc700 1 -- 172.21.15.203:0/2339820938 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70009d80 con 0x7f8b80061130 2021-01-06T22:16:17.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b84981700 1 -- 172.21.15.203:0/2339820938 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b80061130 msgr2=0x7f8b800c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:17.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b84981700 1 --2- 172.21.15.203:0/2339820938 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b80061130 0x7f8b800c0fa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f8b7000b9e0 tx=0x7f8b70006ea0).stop 2021-01-06T22:16:17.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b84981700 1 -- 172.21.15.203:0/2339820938 shutdown_connections 2021-01-06T22:16:17.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b84981700 1 --2- 172.21.15.203:0/2339820938 >> v2:172.21.15.91:3300/0 conn(0x7f8b8014bb30 0x7f8b80148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:17.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b84981700 1 --2- 172.21.15.203:0/2339820938 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b80061130 0x7f8b800c0fa0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:17.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b84981700 1 -- 172.21.15.203:0/2339820938 >> 172.21.15.203:0/2339820938 conn(0x7f8b800667b0 msgr2=0x7f8b800c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:17.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b84981700 1 -- 172.21.15.203:0/2339820938 shutdown_connections 2021-01-06T22:16:17.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.043+0000 7f8b84981700 1 -- 172.21.15.203:0/2339820938 wait complete. 2021-01-06T22:16:17.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b84981700 1 Processor -- start 2021-01-06T22:16:17.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b84981700 1 -- start start 2021-01-06T22:16:17.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b84981700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b80061130 0x7f8b800b8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:17.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b84981700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b800bc700 0x7f8b80143b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:17.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b84981700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b8014bb30 0x7f8b800c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:17.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b84981700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800b88a0 con 0x7f8b800bc700 2021-01-06T22:16:17.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b84981700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b80142e30 con 0x7f8b80061130 2021-01-06T22:16:17.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b84981700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b801430d0 con 0x7f8b8014bb30 2021-01-06T22:16:17.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b7f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b80061130 0x7f8b800b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:17.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b7f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b80061130 0x7f8b800b8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52022/0 (socket says 172.21.15.203:52022) 2021-01-06T22:16:17.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b7f7fe700 1 -- 172.21.15.203:0/2486142588 learned_addr learned my addr 172.21.15.203:0/2486142588 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:17.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b7ffff700 1 --2- 172.21.15.203:0/2486142588 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b8014bb30 0x7f8b800c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:17.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b7effd700 1 --2- 172.21.15.203:0/2486142588 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b800bc700 0x7f8b80143b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:17.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b7f7fe700 1 -- 172.21.15.203:0/2486142588 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b8014bb30 msgr2=0x7f8b800c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:17.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b7f7fe700 1 --2- 172.21.15.203:0/2486142588 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b8014bb30 0x7f8b800c0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:17.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b7f7fe700 1 -- 172.21.15.203:0/2486142588 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b800bc700 msgr2=0x7f8b80143b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:17.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b7f7fe700 1 --2- 172.21.15.203:0/2486142588 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b800bc700 0x7f8b80143b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:17.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b7f7fe700 1 -- 172.21.15.203:0/2486142588 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b80143370 con 0x7f8b80061130 2021-01-06T22:16:17.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b7f7fe700 1 --2- 172.21.15.203:0/2486142588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b80061130 0x7f8b800b8280 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f8b80061ac0 tx=0x7f8b70016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:17.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b677fe700 1 -- 172.21.15.203:0/2486142588 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70012ce0 con 0x7f8b80061130 2021-01-06T22:16:17.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b677fe700 1 -- 172.21.15.203:0/2486142588 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b70012e40 con 0x7f8b80061130 2021-01-06T22:16:17.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b84981700 1 -- 172.21.15.203:0/2486142588 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b80143610 con 0x7f8b80061130 2021-01-06T22:16:17.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b84981700 1 -- 172.21.15.203:0/2486142588 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b800bf850 con 0x7f8b80061130 2021-01-06T22:16:17.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.047+0000 7f8b677fe700 1 -- 172.21.15.203:0/2486142588 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7001c650 con 0x7f8b80061130 2021-01-06T22:16:17.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.051+0000 7f8b84981700 1 -- 172.21.15.203:0/2486142588 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b800450a0 con 0x7f8b80061130 2021-01-06T22:16:17.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.051+0000 7f8b677fe700 1 -- 172.21.15.203:0/2486142588 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f8b7000f070 con 0x7f8b80061130 2021-01-06T22:16:17.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.051+0000 7f8b677fe700 1 --2- 172.21.15.203:0/2486142588 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8b60056400 0x7f8b600588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:17.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.051+0000 7f8b677fe700 1 -- 172.21.15.203:0/2486142588 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f8b70070ef0 con 0x7f8b80061130 2021-01-06T22:16:17.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.051+0000 7f8b7effd700 1 --2- 172.21.15.203:0/2486142588 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8b60056400 0x7f8b600588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:17.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.051+0000 7f8b7effd700 1 --2- 172.21.15.203:0/2486142588 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8b60056400 0x7f8b600588c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f8b800c1380 tx=0x7f8b74007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:17.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.055+0000 7f8b677fe700 1 -- 172.21.15.203:0/2486142588 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f8b70046510 con 0x7f8b80061130 2021-01-06T22:16:17.293 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.291+0000 7f8b84981700 1 -- 172.21.15.203:0/2486142588 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b8005ca60 con 0x7f8b80061130 2021-01-06T22:16:17.293 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.291+0000 7f8b677fe700 1 -- 172.21.15.203:0/2486142588 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+3207 (secure 0 0 0) 0x7f8b7001ca40 con 0x7f8b80061130 2021-01-06T22:16:17.294 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:17.294 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:16.567219+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:replay","state_seq":1,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:17.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.295+0000 7f8b84981700 1 -- 172.21.15.203:0/2486142588 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8b60056400 msgr2=0x7f8b600588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:17.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.295+0000 7f8b84981700 1 --2- 172.21.15.203:0/2486142588 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8b60056400 0x7f8b600588c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f8b800c1380 tx=0x7f8b74007b80).stop 2021-01-06T22:16:17.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.295+0000 7f8b84981700 1 -- 172.21.15.203:0/2486142588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b80061130 msgr2=0x7f8b800b8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:17.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.295+0000 7f8b84981700 1 --2- 172.21.15.203:0/2486142588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b80061130 0x7f8b800b8280 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f8b80061ac0 tx=0x7f8b70016000).stop 2021-01-06T22:16:17.299 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.295+0000 7f8b84981700 1 -- 172.21.15.203:0/2486142588 shutdown_connections 2021-01-06T22:16:17.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.295+0000 7f8b84981700 1 --2- 172.21.15.203:0/2486142588 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8b60056400 0x7f8b600588c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:17.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.295+0000 7f8b84981700 1 --2- 172.21.15.203:0/2486142588 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b8014bb30 0x7f8b800c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:17.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.295+0000 7f8b84981700 1 --2- 172.21.15.203:0/2486142588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b80061130 0x7f8b800b8280 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:17.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.295+0000 7f8b84981700 1 --2- 172.21.15.203:0/2486142588 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b800bc700 0x7f8b80143b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:17.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.295+0000 7f8b84981700 1 -- 172.21.15.203:0/2486142588 >> 172.21.15.203:0/2486142588 conn(0x7f8b800667b0 msgr2=0x7f8b8014a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:17.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.295+0000 7f8b84981700 1 -- 172.21.15.203:0/2486142588 shutdown_connections 2021-01-06T22:16:17.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.295+0000 7f8b84981700 1 -- 172.21.15.203:0/2486142588 wait complete. 2021-01-06T22:16:17.302 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 57 2021-01-06T22:16:17.312 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2021-01-06T22:16:17.312 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2021-01-06T22:16:17.313 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:17.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.427+0000 7f78a48df700 1 Processor -- start 2021-01-06T22:16:17.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.427+0000 7f78a48df700 1 -- start start 2021-01-06T22:16:17.433 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f78a48df700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f78a0063010 0x7f78a0151160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:17.433 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f78a48df700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78a00633f0 0x7f78a014e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:17.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f78a48df700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f78a014e8b0 0x7f78a00be1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:17.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f78a48df700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a014b760 con 0x7f78a014e8b0 2021-01-06T22:16:17.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f78a48df700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a00bdb20 con 0x7f78a0063010 2021-01-06T22:16:17.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f78a48df700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a00602b0 con 0x7f78a00633f0 2021-01-06T22:16:17.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f789f7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f78a0063010 0x7f78a0151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:17.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f789f7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f78a0063010 0x7f78a0151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52032/0 (socket says 172.21.15.203:52032) 2021-01-06T22:16:17.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f789f7fe700 1 -- 172.21.15.203:0/813423544 learned_addr learned my addr 172.21.15.203:0/813423544 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:17.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f789effd700 1 --2- 172.21.15.203:0/813423544 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78a00633f0 0x7f78a014e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:17.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f789ffff700 1 --2- 172.21.15.203:0/813423544 >> v2:172.21.15.91:3300/0 conn(0x7f78a014e8b0 0x7f78a00be1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:17.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f789f7fe700 1 -- 172.21.15.203:0/813423544 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78a00633f0 msgr2=0x7f78a014e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:17.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f789f7fe700 1 --2- 172.21.15.203:0/813423544 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78a00633f0 0x7f78a014e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:17.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f789f7fe700 1 -- 172.21.15.203:0/813423544 >> v2:172.21.15.91:3300/0 conn(0x7f78a014e8b0 msgr2=0x7f78a00be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:17.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f789f7fe700 1 --2- 172.21.15.203:0/813423544 >> v2:172.21.15.91:3300/0 conn(0x7f78a014e8b0 0x7f78a00be1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:17.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f789f7fe700 1 -- 172.21.15.203:0/813423544 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a0151710 con 0x7f78a0063010 2021-01-06T22:16:17.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f789ffff700 1 --2- 172.21.15.203:0/813423544 >> v2:172.21.15.91:3300/0 conn(0x7f78a014e8b0 0x7f78a00be1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:16:17.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f789f7fe700 1 --2- 172.21.15.203:0/813423544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a0063010 0x7f78a0151160 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f78a00667e0 tx=0x7f7898009290).ready entity=mon.1 client_cookie=69ccf9f9792ca364 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:17.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f789e7fc700 1 -- 172.21.15.203:0/813423544 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789800bc50 con 0x7f78a0063010 2021-01-06T22:16:17.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f789e7fc700 1 -- 172.21.15.203:0/813423544 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789800bdb0 con 0x7f78a0063010 2021-01-06T22:16:17.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f789e7fc700 1 -- 172.21.15.203:0/813423544 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898011610 con 0x7f78a0063010 2021-01-06T22:16:17.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f78a48df700 1 -- 172.21.15.203:0/813423544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a0063010 msgr2=0x7f78a0151160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:17.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f78a48df700 1 --2- 172.21.15.203:0/813423544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a0063010 0x7f78a0151160 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f78a00667e0 tx=0x7f7898009290).stop 2021-01-06T22:16:17.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f78a48df700 1 -- 172.21.15.203:0/813423544 shutdown_connections 2021-01-06T22:16:17.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f78a48df700 1 --2- 172.21.15.203:0/813423544 >> v2:172.21.15.91:3300/0 conn(0x7f78a014e8b0 0x7f78a00be1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:17.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f78a48df700 1 --2- 172.21.15.203:0/813423544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a0063010 0x7f78a0151160 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:17.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f78a48df700 1 --2- 172.21.15.203:0/813423544 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78a00633f0 0x7f78a014e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:17.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f78a48df700 1 -- 172.21.15.203:0/813423544 >> 172.21.15.203:0/813423544 conn(0x7f78a0067e20 msgr2=0x7f78a0068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:17.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f78a48df700 1 -- 172.21.15.203:0/813423544 shutdown_connections 2021-01-06T22:16:17.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.431+0000 7f78a48df700 1 -- 172.21.15.203:0/813423544 wait complete. 2021-01-06T22:16:17.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f78a48df700 1 Processor -- start 2021-01-06T22:16:17.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f78a48df700 1 -- start start 2021-01-06T22:16:17.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f78a48df700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a0063010 0x7f78a015db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:17.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f78a48df700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a00633f0 0x7f78a0153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:17.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f78a48df700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78a014e8b0 0x7f78a00c0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:17.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f78a48df700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a015e050 con 0x7f78a0063010 2021-01-06T22:16:17.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f78a48df700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a00b86b0 con 0x7f78a00633f0 2021-01-06T22:16:17.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f78a48df700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a00b7e90 con 0x7f78a014e8b0 2021-01-06T22:16:17.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f789effd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a00633f0 0x7f78a0153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:17.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f789effd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a00633f0 0x7f78a0153140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52040/0 (socket says 172.21.15.203:52040) 2021-01-06T22:16:17.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f789effd700 1 -- 172.21.15.203:0/573634071 learned_addr learned my addr 172.21.15.203:0/573634071 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:17.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f789effd700 1 -- 172.21.15.203:0/573634071 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78a014e8b0 msgr2=0x7f78a00c0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:17.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f789effd700 1 --2- 172.21.15.203:0/573634071 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78a014e8b0 0x7f78a00c0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-01-06T22:16:17.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f789effd700 1 -- 172.21.15.203:0/573634071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a0063010 msgr2=0x7f78a015db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:17.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f789effd700 1 --2- 172.21.15.203:0/573634071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a0063010 0x7f78a015db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-01-06T22:16:17.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f789effd700 1 -- 172.21.15.203:0/573634071 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a00bfd00 con 0x7f78a00633f0 2021-01-06T22:16:17.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f789effd700 1 --2- 172.21.15.203:0/573634071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a00633f0 0x7f78a0153140 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f789000db90 tx=0x7f7890008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:17.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f78877fe700 1 -- 172.21.15.203:0/573634071 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890009790 con 0x7f78a00633f0 2021-01-06T22:16:17.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f78877fe700 1 -- 172.21.15.203:0/573634071 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78900098f0 con 0x7f78a00633f0 2021-01-06T22:16:17.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f78a48df700 1 -- 172.21.15.203:0/573634071 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a00bffa0 con 0x7f78a00633f0 2021-01-06T22:16:17.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f78877fe700 1 -- 172.21.15.203:0/573634071 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890009c70 con 0x7f78a00633f0 2021-01-06T22:16:17.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.435+0000 7f78a48df700 1 -- 172.21.15.203:0/573634071 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a00c07d0 con 0x7f78a00633f0 2021-01-06T22:16:17.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.439+0000 7f78877fe700 1 -- 172.21.15.203:0/573634071 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f7890013480 con 0x7f78a00633f0 2021-01-06T22:16:17.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.439+0000 7f78a48df700 1 -- 172.21.15.203:0/573634071 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a00450a0 con 0x7f78a00633f0 2021-01-06T22:16:17.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.439+0000 7f78877fe700 1 --2- 172.21.15.203:0/573634071 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f78800563d0 0x7f7880058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:17.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.439+0000 7f78877fe700 1 -- 172.21.15.203:0/573634071 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f789006fbc0 con 0x7f78a00633f0 2021-01-06T22:16:17.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.439+0000 7f789f7fe700 1 --2- 172.21.15.203:0/573634071 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f78800563d0 0x7f7880058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:17.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.439+0000 7f789f7fe700 1 --2- 172.21.15.203:0/573634071 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f78800563d0 0x7f7880058890 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f7898009fd0 tx=0x7f7898017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:17.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.443+0000 7f78877fe700 1 -- 172.21.15.203:0/573634071 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f7890045180 con 0x7f78a00633f0 2021-01-06T22:16:17.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.679+0000 7f78a48df700 1 -- 172.21.15.203:0/573634071 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78a0061e70 con 0x7f78a00633f0 2021-01-06T22:16:17.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.679+0000 7f78877fe700 1 -- 172.21.15.203:0/573634071 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+3209 (secure 0 0 0) 0x7f78900487a0 con 0x7f78a00633f0 2021-01-06T22:16:17.682 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:17.682 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:17.578702+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:resolve","state_seq":29,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:17.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.683+0000 7f78a48df700 1 -- 172.21.15.203:0/573634071 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f78800563d0 msgr2=0x7f7880058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:17.684 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.683+0000 7f78a48df700 1 --2- 172.21.15.203:0/573634071 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f78800563d0 0x7f7880058890 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f7898009fd0 tx=0x7f7898017000).stop 2021-01-06T22:16:17.684 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.683+0000 7f78a48df700 1 -- 172.21.15.203:0/573634071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a00633f0 msgr2=0x7f78a0153140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:17.684 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.683+0000 7f78a48df700 1 --2- 172.21.15.203:0/573634071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a00633f0 0x7f78a0153140 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f789000db90 tx=0x7f7890008040).stop 2021-01-06T22:16:17.684 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.683+0000 7f78a48df700 1 -- 172.21.15.203:0/573634071 shutdown_connections 2021-01-06T22:16:17.685 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.683+0000 7f78a48df700 1 --2- 172.21.15.203:0/573634071 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f78800563d0 0x7f7880058890 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0x7f7898009fd0 tx=0x7f7898017000).stop 2021-01-06T22:16:17.685 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.683+0000 7f78a48df700 1 --2- 172.21.15.203:0/573634071 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78a014e8b0 0x7f78a00c0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-01-06T22:16:17.685 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.683+0000 7f78a48df700 1 --2- 172.21.15.203:0/573634071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a0063010 0x7f78a015db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-01-06T22:16:17.685 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.683+0000 7f78a48df700 1 --2- 172.21.15.203:0/573634071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a00633f0 0x7f78a0153140 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:17.685 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.683+0000 7f78a48df700 1 -- 172.21.15.203:0/573634071 >> 172.21.15.203:0/573634071 conn(0x7f78a0067e20 msgr2=0x7f78a00bdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:17.686 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.683+0000 7f78a48df700 1 -- 172.21.15.203:0/573634071 shutdown_connections 2021-01-06T22:16:17.686 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:17.683+0000 7f78a48df700 1 -- 172.21.15.203:0/573634071 wait complete. 2021-01-06T22:16:17.686 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 58 2021-01-06T22:16:17.697 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 58, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 58, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5350": { "addr": "172.21.15.203:6837/4277691165", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6836", "nonce": 4277691165, "type": "v2" }, { "addr": "172.21.15.203:6837", "nonce": 4277691165, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5350, "incarnation": 51, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 37 }, "gid_5364": { "addr": "172.21.15.91:6837/617463325", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6835", "nonce": 617463325, "type": "v2" }, { "addr": "172.21.15.91:6837", "nonce": 617463325, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5364, "incarnation": 57, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:resolve", "state_seq": 29 } }, "last_failure": 0, "last_failure_osd_epoch": 30, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2021-01-06T22:16:17.578702+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 5350, "mds_1": 5364 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.91:6836/2428882934", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6834", "nonce": 2428882934, "type": "v2" }, { "addr": "172.21.15.91:6836", "nonce": 2428882934, "type": "v1" } ] }, "epoch": 26, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4724, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2021-01-06T22:16:17.739 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:16:17.742 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:16:19.699 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:19.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.835+0000 7fa71e011700 1 Processor -- start 2021-01-06T22:16:20.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.835+0000 7fa71e011700 1 -- start start 2021-01-06T22:16:20.238 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71e011700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7180c0fa0 0x7fa7180641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:20.238 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71e011700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa7180ba9e0 0x7fa71814ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:20.238 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71e011700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7180bccb0 con 0x7fa71814df70 2021-01-06T22:16:20.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71e011700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa718067ac0 con 0x7fa7180ba9e0 2021-01-06T22:16:20.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71e011700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180bcee0 con 0x7fa7180c0fa0 2021-01-06T22:16:20.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71d810700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa7180ba9e0 0x7fa71814ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:20.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71d810700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa7180ba9e0 0x7fa71814ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52056/0 (socket says 172.21.15.203:52056) 2021-01-06T22:16:20.240 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71d810700 1 -- 172.21.15.203:0/480476420 learned_addr learned my addr 172.21.15.203:0/480476420 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:20.240 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71c80e700 1 --2- 172.21.15.203:0/480476420 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7180c0fa0 0x7fa7180641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:20.240 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71d810700 1 -- 172.21.15.203:0/480476420 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7180c0fa0 msgr2=0x7fa7180641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:20.240 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71d810700 1 --2- 172.21.15.203:0/480476420 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7180c0fa0 0x7fa7180641e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:20.240 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71d810700 1 -- 172.21.15.203:0/480476420 >> v1:172.21.15.91:6789/0 conn(0x7fa71814df70 legacy=0x7fa71814e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:20.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71d810700 1 -- 172.21.15.203:0/480476420 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7180bd180 con 0x7fa7180ba9e0 2021-01-06T22:16:20.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71d810700 1 --2- 172.21.15.203:0/480476420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7180ba9e0 0x7fa71814ad90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fa710009d50 tx=0x7fa710009290).ready entity=mon.1 client_cookie=8edc89eda656c53c server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:20.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa707fff700 1 -- 172.21.15.203:0/480476420 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71000bd10 con 0x7fa7180ba9e0 2021-01-06T22:16:20.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa707fff700 1 -- 172.21.15.203:0/480476420 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71000be70 con 0x7fa7180ba9e0 2021-01-06T22:16:20.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa707fff700 1 -- 172.21.15.203:0/480476420 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710011610 con 0x7fa7180ba9e0 2021-01-06T22:16:20.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71e011700 1 -- 172.21.15.203:0/480476420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7180ba9e0 msgr2=0x7fa71814ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:20.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71e011700 1 --2- 172.21.15.203:0/480476420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7180ba9e0 0x7fa71814ad90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fa710009d50 tx=0x7fa710009290).stop 2021-01-06T22:16:20.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71e011700 1 -- 172.21.15.203:0/480476420 shutdown_connections 2021-01-06T22:16:20.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71e011700 1 --2- 172.21.15.203:0/480476420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7180ba9e0 0x7fa71814ad90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:20.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71e011700 1 --2- 172.21.15.203:0/480476420 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7180c0fa0 0x7fa7180641e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:20.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71e011700 1 -- 172.21.15.203:0/480476420 >> 172.21.15.203:0/480476420 conn(0x7fa7180667b0 msgr2=0x7fa7180c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:20.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71e011700 1 -- 172.21.15.203:0/480476420 shutdown_connections 2021-01-06T22:16:20.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.839+0000 7fa71e011700 1 -- 172.21.15.203:0/480476420 wait complete. 2021-01-06T22:16:20.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71e011700 1 Processor -- start 2021-01-06T22:16:20.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71e011700 1 -- start start 2021-01-06T22:16:20.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71e011700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7180ba9e0 0x7fa718147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:20.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71e011700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7180c0fa0 0x7fa7180b9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:20.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71e011700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa71814df70 0x7fa7180ba220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:20.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71e011700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180b7750 con 0x7fa7180ba9e0 2021-01-06T22:16:20.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71e011700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa718160790 con 0x7fa7180c0fa0 2021-01-06T22:16:20.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71e011700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa718160a30 con 0x7fa71814df70 2021-01-06T22:16:20.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71c80e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7180c0fa0 0x7fa7180b9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:20.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71c80e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7180c0fa0 0x7fa7180b9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52060/0 (socket says 172.21.15.203:52060) 2021-01-06T22:16:20.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71c80e700 1 -- 172.21.15.203:0/1829610664 learned_addr learned my addr 172.21.15.203:0/1829610664 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:20.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71d00f700 1 --2- 172.21.15.203:0/1829610664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7180ba9e0 0x7fa718147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:20.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71d810700 1 --2- 172.21.15.203:0/1829610664 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa71814df70 0x7fa7180ba220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:20.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71c80e700 1 -- 172.21.15.203:0/1829610664 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa71814df70 msgr2=0x7fa7180ba220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:20.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71c80e700 1 --2- 172.21.15.203:0/1829610664 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa71814df70 0x7fa7180ba220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:20.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71c80e700 1 -- 172.21.15.203:0/1829610664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7180ba9e0 msgr2=0x7fa718147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:20.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71c80e700 1 --2- 172.21.15.203:0/1829610664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7180ba9e0 0x7fa718147d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:20.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71c80e700 1 -- 172.21.15.203:0/1829610664 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa718160cd0 con 0x7fa7180c0fa0 2021-01-06T22:16:20.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71c80e700 1 --2- 172.21.15.203:0/1829610664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7180c0fa0 0x7fa7180b9d30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fa70800db60 tx=0x7fa708008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:20.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa7057fa700 1 -- 172.21.15.203:0/1829610664 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080097f0 con 0x7fa7180c0fa0 2021-01-06T22:16:20.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa7057fa700 1 -- 172.21.15.203:0/1829610664 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa708009950 con 0x7fa7180c0fa0 2021-01-06T22:16:20.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71e011700 1 -- 172.21.15.203:0/1829610664 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa718160f70 con 0x7fa7180c0fa0 2021-01-06T22:16:20.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71e011700 1 -- 172.21.15.203:0/1829610664 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7181614b0 con 0x7fa7180c0fa0 2021-01-06T22:16:20.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa7057fa700 1 -- 172.21.15.203:0/1829610664 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708009cd0 con 0x7fa7180c0fa0 2021-01-06T22:16:20.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.843+0000 7fa71e011700 1 -- 172.21.15.203:0/1829610664 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f0004f60 con 0x7fa7180c0fa0 2021-01-06T22:16:20.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.847+0000 7fa7057fa700 1 -- 172.21.15.203:0/1829610664 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa708013690 con 0x7fa7180c0fa0 2021-01-06T22:16:20.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.847+0000 7fa7057fa700 1 --2- 172.21.15.203:0/1829610664 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa6f80563d0 0x7fa6f8058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:20.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.847+0000 7fa7057fa700 1 -- 172.21.15.203:0/1829610664 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6675+0+0 (secure 0 0 0) 0x7fa70806fb80 con 0x7fa7180c0fa0 2021-01-06T22:16:20.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.847+0000 7fa71d00f700 1 --2- 172.21.15.203:0/1829610664 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa6f80563d0 0x7fa6f8058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:20.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.847+0000 7fa71d00f700 1 --2- 172.21.15.203:0/1829610664 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa6f80563d0 0x7fa6f8058890 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fa714003430 tx=0x7fa714009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:20.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:19.851+0000 7fa7057fa700 1 -- 172.21.15.203:0/1829610664 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa708045140 con 0x7fa7180c0fa0 2021-01-06T22:16:20.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:20.075+0000 7fa71e011700 1 -- 172.21.15.203:0/1829610664 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6f0005790 con 0x7fa7180c0fa0 2021-01-06T22:16:20.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:20.079+0000 7fa7057fa700 1 -- 172.21.15.203:0/1829610664 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+3208 (secure 0 0 0) 0x7fa708048760 con 0x7fa7180c0fa0 2021-01-06T22:16:20.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:20.079+0000 7fa71e011700 1 -- 172.21.15.203:0/1829610664 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa6f80563d0 msgr2=0x7fa6f8058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:20.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:20.079+0000 7fa71e011700 1 --2- 172.21.15.203:0/1829610664 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa6f80563d0 0x7fa6f8058890 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fa714003430 tx=0x7fa714009040).stop 2021-01-06T22:16:20.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:20.079+0000 7fa71e011700 1 -- 172.21.15.203:0/1829610664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7180c0fa0 msgr2=0x7fa7180b9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:20.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:20.079+0000 7fa71e011700 1 --2- 172.21.15.203:0/1829610664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7180c0fa0 0x7fa7180b9d30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fa70800db60 tx=0x7fa708008040).stop 2021-01-06T22:16:20.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:20.079+0000 7fa71e011700 1 -- 172.21.15.203:0/1829610664 shutdown_connections 2021-01-06T22:16:20.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:20.079+0000 7fa71e011700 1 --2- 172.21.15.203:0/1829610664 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa6f80563d0 0x7fa6f8058890 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:20.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:20.079+0000 7fa71e011700 1 --2- 172.21.15.203:0/1829610664 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa71814df70 0x7fa7180ba220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:20.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:20.079+0000 7fa71e011700 1 --2- 172.21.15.203:0/1829610664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7180ba9e0 0x7fa718147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:20.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:20.079+0000 7fa71e011700 1 --2- 172.21.15.203:0/1829610664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7180c0fa0 0x7fa7180b9d30 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:20.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:20.079+0000 7fa71e011700 1 -- 172.21.15.203:0/1829610664 >> 172.21.15.203:0/1829610664 conn(0x7fa7180667b0 msgr2=0x7fa718060610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:20.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:20.079+0000 7fa71e011700 1 -- 172.21.15.203:0/1829610664 shutdown_connections 2021-01-06T22:16:20.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:20.079+0000 7fa71e011700 1 -- 172.21.15.203:0/1829610664 wait complete. 2021-01-06T22:16:20.253 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 60 2021-01-06T22:16:20.254 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:20.254 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:19.584583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:rejoin","state_seq":31,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:20.254 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:16:22.257 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:22.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd704c700 1 Processor -- start 2021-01-06T22:16:22.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd704c700 1 -- start start 2021-01-06T22:16:22.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd704c700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6fd00c44f0 0x7f6fd00c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:22.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd704c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6fd0060fc0 0x7f6fd0150e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:22.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd704c700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd00c0f50 con 0x7f6fd00c48d0 2021-01-06T22:16:22.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd704c700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd0066cf0 con 0x7f6fd00c44f0 2021-01-06T22:16:22.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd704c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd0060750 con 0x7f6fd0060fc0 2021-01-06T22:16:22.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd684b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6fd0060fc0 0x7f6fd0150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:22.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd684b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6fd0060fc0 0x7f6fd0150e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52074/0 (socket says 172.21.15.203:52074) 2021-01-06T22:16:22.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd684b700 1 -- 172.21.15.203:0/3413042191 learned_addr learned my addr 172.21.15.203:0/3413042191 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:22.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd604a700 1 --2- 172.21.15.203:0/3413042191 >> v2:172.21.15.91:3300/0 conn(0x7f6fd00c44f0 0x7f6fd00c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:22.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd684b700 1 -- 172.21.15.203:0/3413042191 >> v2:172.21.15.91:3300/0 conn(0x7f6fd00c44f0 msgr2=0x7f6fd00c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:22.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd684b700 1 --2- 172.21.15.203:0/3413042191 >> v2:172.21.15.91:3300/0 conn(0x7f6fd00c44f0 0x7f6fd00c1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:22.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd684b700 1 -- 172.21.15.203:0/3413042191 >> v1:172.21.15.91:6789/0 conn(0x7f6fd00c48d0 legacy=0x7f6fd0150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:22.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd684b700 1 -- 172.21.15.203:0/3413042191 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd00609f0 con 0x7f6fd0060fc0 2021-01-06T22:16:22.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd684b700 1 --2- 172.21.15.203:0/3413042191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fd0060fc0 0x7f6fd0150e40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f6fd00b8310 tx=0x7f6fc8009290).ready entity=mon.1 client_cookie=b174986075b0309c server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:22.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd5048700 1 -- 172.21.15.203:0/3413042191 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc800bbc0 con 0x7f6fd0060fc0 2021-01-06T22:16:22.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd5048700 1 -- 172.21.15.203:0/3413042191 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc800bd20 con 0x7f6fd0060fc0 2021-01-06T22:16:22.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.399+0000 7f6fd5048700 1 -- 172.21.15.203:0/3413042191 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8011610 con 0x7f6fd0060fc0 2021-01-06T22:16:22.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd704c700 1 -- 172.21.15.203:0/3413042191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fd0060fc0 msgr2=0x7f6fd0150e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:22.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd704c700 1 --2- 172.21.15.203:0/3413042191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fd0060fc0 0x7f6fd0150e40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f6fd00b8310 tx=0x7f6fc8009290).stop 2021-01-06T22:16:22.408 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd704c700 1 -- 172.21.15.203:0/3413042191 shutdown_connections 2021-01-06T22:16:22.408 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd704c700 1 --2- 172.21.15.203:0/3413042191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fd0060fc0 0x7f6fd0150e40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:22.408 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd704c700 1 --2- 172.21.15.203:0/3413042191 >> v2:172.21.15.91:3300/0 conn(0x7f6fd00c44f0 0x7f6fd00c1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:22.408 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd704c700 1 -- 172.21.15.203:0/3413042191 >> 172.21.15.203:0/3413042191 conn(0x7f6fd0056cd0 msgr2=0x7f6fd0061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:22.408 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd704c700 1 -- 172.21.15.203:0/3413042191 shutdown_connections 2021-01-06T22:16:22.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd704c700 1 -- 172.21.15.203:0/3413042191 wait complete. 2021-01-06T22:16:22.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd704c700 1 Processor -- start 2021-01-06T22:16:22.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd704c700 1 -- start start 2021-01-06T22:16:22.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd704c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6fd0060fc0 0x7f6fd015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:22.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd704c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fd00c44f0 0x7f6fd0152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:22.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd704c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fd00c48d0 0x7f6fd014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:22.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd704c700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd015d7e0 con 0x7f6fd00c44f0 2021-01-06T22:16:22.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd704c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00ba600 con 0x7f6fd00c48d0 2021-01-06T22:16:22.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd704c700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00ba8a0 con 0x7f6fd0060fc0 2021-01-06T22:16:22.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd684b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fd00c48d0 0x7f6fd014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:22.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd684b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fd00c48d0 0x7f6fd014cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52080/0 (socket says 172.21.15.203:52080) 2021-01-06T22:16:22.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd684b700 1 -- 172.21.15.203:0/705519828 learned_addr learned my addr 172.21.15.203:0/705519828 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:22.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd5849700 1 --2- 172.21.15.203:0/705519828 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fd00c44f0 0x7f6fd0152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:22.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd684b700 1 -- 172.21.15.203:0/705519828 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6fd0060fc0 msgr2=0x7f6fd015e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:22.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd604a700 1 --2- 172.21.15.203:0/705519828 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6fd0060fc0 0x7f6fd015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:22.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd684b700 1 --2- 172.21.15.203:0/705519828 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6fd0060fc0 0x7f6fd015e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:22.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd684b700 1 -- 172.21.15.203:0/705519828 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fd00c44f0 msgr2=0x7f6fd0152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:22.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd684b700 1 --2- 172.21.15.203:0/705519828 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fd00c44f0 0x7f6fd0152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:22.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd684b700 1 -- 172.21.15.203:0/705519828 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd015d9d0 con 0x7f6fd00c48d0 2021-01-06T22:16:22.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd5849700 1 --2- 172.21.15.203:0/705519828 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fd00c44f0 0x7f6fd0152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:16:22.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd604a700 1 --2- 172.21.15.203:0/705519828 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6fd0060fc0 0x7f6fd015e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:16:22.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.403+0000 7f6fd684b700 1 --2- 172.21.15.203:0/705519828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fd00c48d0 0x7f6fd014cca0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f6fc8006fd0 tx=0x7f6fc8017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:22.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.407+0000 7f6fbe7fc700 1 -- 172.21.15.203:0/705519828 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8011e70 con 0x7f6fd00c48d0 2021-01-06T22:16:22.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.407+0000 7f6fbe7fc700 1 -- 172.21.15.203:0/705519828 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc800be90 con 0x7f6fd00c48d0 2021-01-06T22:16:22.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.407+0000 7f6fd704c700 1 -- 172.21.15.203:0/705519828 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd015dc70 con 0x7f6fd00c48d0 2021-01-06T22:16:22.417 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.407+0000 7f6fbe7fc700 1 -- 172.21.15.203:0/705519828 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8010e10 con 0x7f6fd00c48d0 2021-01-06T22:16:22.417 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.407+0000 7f6fd704c700 1 -- 172.21.15.203:0/705519828 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd00b8d50 con 0x7f6fd00c48d0 2021-01-06T22:16:22.417 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.407+0000 7f6fd704c700 1 -- 172.21.15.203:0/705519828 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd00450a0 con 0x7f6fd00c48d0 2021-01-06T22:16:22.417 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.407+0000 7f6fbe7fc700 1 -- 172.21.15.203:0/705519828 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f6fc800f070 con 0x7f6fd00c48d0 2021-01-06T22:16:22.417 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.407+0000 7f6fbe7fc700 1 --2- 172.21.15.203:0/705519828 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6fb00563d0 0x7f6fb0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:22.418 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.407+0000 7f6fbe7fc700 1 -- 172.21.15.203:0/705519828 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f6fc8072100 con 0x7f6fd00c48d0 2021-01-06T22:16:22.418 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.407+0000 7f6fd604a700 1 --2- 172.21.15.203:0/705519828 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6fb00563d0 0x7f6fb0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:22.418 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.407+0000 7f6fd604a700 1 --2- 172.21.15.203:0/705519828 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6fb00563d0 0x7f6fb0058890 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f6fcc006fd0 tx=0x7f6fcc008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:22.419 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.411+0000 7f6fbe7fc700 1 -- 172.21.15.203:0/705519828 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f6fc80477a0 con 0x7f6fd00c48d0 2021-01-06T22:16:22.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.635+0000 7f6fd704c700 1 -- 172.21.15.203:0/705519828 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fd00574f0 con 0x7f6fd00c48d0 2021-01-06T22:16:22.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.635+0000 7f6fbe7fc700 1 -- 172.21.15.203:0/705519828 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+3208 (secure 0 0 0) 0x7f6fc804adc0 con 0x7f6fd00c48d0 2021-01-06T22:16:22.641 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:22.641 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:20.595529+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:22.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.639+0000 7f6fd704c700 1 -- 172.21.15.203:0/705519828 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6fb00563d0 msgr2=0x7f6fb0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:22.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.639+0000 7f6fd704c700 1 --2- 172.21.15.203:0/705519828 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6fb00563d0 0x7f6fb0058890 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f6fcc006fd0 tx=0x7f6fcc008040).stop 2021-01-06T22:16:22.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.639+0000 7f6fd704c700 1 -- 172.21.15.203:0/705519828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fd00c48d0 msgr2=0x7f6fd014cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:22.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.639+0000 7f6fd704c700 1 --2- 172.21.15.203:0/705519828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fd00c48d0 0x7f6fd014cca0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f6fc8006fd0 tx=0x7f6fc8017000).stop 2021-01-06T22:16:22.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.639+0000 7f6fd704c700 1 -- 172.21.15.203:0/705519828 shutdown_connections 2021-01-06T22:16:22.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.639+0000 7f6fd704c700 1 --2- 172.21.15.203:0/705519828 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6fb00563d0 0x7f6fb0058890 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:22.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.639+0000 7f6fd704c700 1 --2- 172.21.15.203:0/705519828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fd00c48d0 0x7f6fd014cca0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:22.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.639+0000 7f6fd704c700 1 --2- 172.21.15.203:0/705519828 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6fd0060fc0 0x7f6fd015e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:22.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.639+0000 7f6fd704c700 1 --2- 172.21.15.203:0/705519828 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fd00c44f0 0x7f6fd0152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:22.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.639+0000 7f6fd704c700 1 -- 172.21.15.203:0/705519828 >> 172.21.15.203:0/705519828 conn(0x7f6fd0056cd0 msgr2=0x7f6fd00bc510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:22.649 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.639+0000 7f6fd704c700 1 -- 172.21.15.203:0/705519828 shutdown_connections 2021-01-06T22:16:22.649 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:22.639+0000 7f6fd704c700 1 -- 172.21.15.203:0/705519828 wait complete. 2021-01-06T22:16:22.650 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 61 2021-01-06T22:16:22.657 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=1, replacing gid=5068 2021-01-06T22:16:22.657 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.c 2021-01-06T22:16:22.852 INFO:tasks.ceph.mds.a.smithi203.stderr:2021-01-06T22:16:22.851+0000 7f8e8c4a9700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2021-01-06T22:16:27.658 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2021-01-06T22:16:27.658 INFO:tasks.ceph.mds.c:Restarting daemon 2021-01-06T22:16:27.659 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-01-06T22:16:27.661 INFO:tasks.ceph.mds.c:Started 2021-01-06T22:16:27.661 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:27.737 INFO:tasks.ceph.mds.c.smithi203.stdout:starting mds.c at 2021-01-06T22:16:27.810 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.807+0000 7f1b88bee700 1 Processor -- start 2021-01-06T22:16:27.810 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.807+0000 7f1b88bee700 1 -- start start 2021-01-06T22:16:27.812 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b88bee700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1b841478c0 0x7f1b840bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:27.813 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b88bee700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b840c16b0 con 0x7f1b8414e310 2021-01-06T22:16:27.813 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b88bee700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b840bb2d0 con 0x7f1b84064380 2021-01-06T22:16:27.813 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b88bee700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8414a560 con 0x7f1b841478c0 2021-01-06T22:16:27.813 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b837fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f1b84064380 0x7f1b840c0fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50700/0 (socket says 172.21.15.203:50700) 2021-01-06T22:16:27.814 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b837fe700 1 -- 172.21.15.203:0/3024381664 learned_addr learned my addr 172.21.15.203:0/3024381664 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:16:27.814 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b83fff700 1 --2- 172.21.15.203:0/3024381664 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1b841478c0 0x7f1b840bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:27.814 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b827fc700 1 -- 172.21.15.203:0/3024381664 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720617362 0 0) 0x7f1b840bb2d0 con 0x7f1b84064380 2021-01-06T22:16:27.814 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b827fc700 1 -- 172.21.15.203:0/3024381664 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b70003230 con 0x7f1b84064380 2021-01-06T22:16:27.815 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b827fc700 1 -- 172.21.15.203:0/3024381664 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925738709 0 0) 0x7f1b780010d0 con 0x7f1b8414e310 2021-01-06T22:16:27.815 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b827fc700 1 -- 172.21.15.203:0/3024381664 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b840bb2d0 con 0x7f1b8414e310 2021-01-06T22:16:27.815 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b827fc700 1 -- 172.21.15.203:0/3024381664 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2576261014 0 0) 0x7f1b70003230 con 0x7f1b84064380 2021-01-06T22:16:27.816 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b83fff700 1 -- 172.21.15.203:0/3024381664 >> v1:172.21.15.203:6789/0 conn(0x7f1b84064380 legacy=0x7f1b840c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:27.816 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b83fff700 1 -- 172.21.15.203:0/3024381664 >> v1:172.21.15.91:6789/0 conn(0x7f1b8414e310 legacy=0x7f1b841471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:27.816 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b83fff700 1 -- 172.21.15.203:0/3024381664 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8414a800 con 0x7f1b841478c0 2021-01-06T22:16:27.816 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b83fff700 1 --2- 172.21.15.203:0/3024381664 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1b841478c0 0x7f1b840bb730 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f1b7c009c40 tx=0x7f1b7c0091d0).ready entity=mon.2 client_cookie=6fb9185d02fa5cba server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:27.819 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b827fc700 1 -- 172.21.15.203:0/3024381664 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b74001500 con 0x7f1b841478c0 2021-01-06T22:16:27.820 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b827fc700 1 -- 172.21.15.203:0/3024381664 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7c00b7c0 con 0x7f1b841478c0 2021-01-06T22:16:27.820 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b827fc700 1 -- 172.21.15.203:0/3024381664 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c00bb40 con 0x7f1b841478c0 2021-01-06T22:16:27.820 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b88bee700 1 -- 172.21.15.203:0/3024381664 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1b841478c0 msgr2=0x7f1b840bb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:27.821 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b88bee700 1 --2- 172.21.15.203:0/3024381664 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1b841478c0 0x7f1b840bb730 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f1b7c009c40 tx=0x7f1b7c0091d0).stop 2021-01-06T22:16:27.821 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b88bee700 1 -- 172.21.15.203:0/3024381664 shutdown_connections 2021-01-06T22:16:27.821 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b88bee700 1 --2- 172.21.15.203:0/3024381664 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1b841478c0 0x7f1b840bb730 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:27.821 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b88bee700 1 -- 172.21.15.203:0/3024381664 >> 172.21.15.203:0/3024381664 conn(0x7f1b840667b0 msgr2=0x7f1b840c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:27.822 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b88bee700 1 -- 172.21.15.203:0/3024381664 shutdown_connections 2021-01-06T22:16:27.822 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.811+0000 7f1b88bee700 1 -- 172.21.15.203:0/3024381664 wait complete. 2021-01-06T22:16:27.822 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b88bee700 1 Processor -- start 2021-01-06T22:16:27.822 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b88bee700 1 -- start start 2021-01-06T22:16:27.823 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b88bee700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1b84064380 0x7f1b840b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:27.823 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b88bee700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1b841478c0 0x7f1b8415b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:27.823 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b88bee700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b8414e310 0x7f1b8415be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:27.824 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b88bee700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8415c350 con 0x7f1b84064380 2021-01-06T22:16:27.824 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b88bee700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b840b9fa0 con 0x7f1b8414e310 2021-01-06T22:16:27.824 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b88bee700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b840b8ed0 con 0x7f1b841478c0 2021-01-06T22:16:27.824 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b83fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b8414e310 0x7f1b8415be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:27.824 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b83fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b8414e310 0x7f1b8415be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52116/0 (socket says 172.21.15.203:52116) 2021-01-06T22:16:27.825 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b83fff700 1 -- 172.21.15.203:0/715951500 learned_addr learned my addr 172.21.15.203:0/715951500 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:27.825 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b837fe700 1 --2- 172.21.15.203:0/715951500 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1b84064380 0x7f1b840b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:27.825 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b82ffd700 1 --2- 172.21.15.203:0/715951500 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1b841478c0 0x7f1b8415b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:27.825 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b83fff700 1 -- 172.21.15.203:0/715951500 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1b841478c0 msgr2=0x7f1b8415b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:27.826 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b83fff700 1 --2- 172.21.15.203:0/715951500 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1b841478c0 0x7f1b8415b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:27.826 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b83fff700 1 -- 172.21.15.203:0/715951500 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1b84064380 msgr2=0x7f1b840b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:27.826 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b83fff700 1 --2- 172.21.15.203:0/715951500 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1b84064380 0x7f1b840b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:27.826 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b83fff700 1 -- 172.21.15.203:0/715951500 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b840b9170 con 0x7f1b8414e310 2021-01-06T22:16:27.827 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b83fff700 1 --2- 172.21.15.203:0/715951500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b8414e310 0x7f1b8415be60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f1b84063d70 tx=0x7f1b7c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:27.827 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b6b7fe700 1 -- 172.21.15.203:0/715951500 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c01d430 con 0x7f1b8414e310 2021-01-06T22:16:27.827 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b6b7fe700 1 -- 172.21.15.203:0/715951500 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7c01d590 con 0x7f1b8414e310 2021-01-06T22:16:27.827 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b88bee700 1 -- 172.21.15.203:0/715951500 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b840b9410 con 0x7f1b8414e310 2021-01-06T22:16:27.827 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b6b7fe700 1 -- 172.21.15.203:0/715951500 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c01d930 con 0x7f1b8414e310 2021-01-06T22:16:27.828 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.815+0000 7f1b88bee700 1 -- 172.21.15.203:0/715951500 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b840b9950 con 0x7f1b8414e310 2021-01-06T22:16:27.828 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.819+0000 7f1b88bee700 1 -- 172.21.15.203:0/715951500 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b8415b6b0 con 0x7f1b8414e310 2021-01-06T22:16:27.828 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.819+0000 7f1b6b7fe700 1 -- 172.21.15.203:0/715951500 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f1b7c00f070 con 0x7f1b8414e310 2021-01-06T22:16:27.830 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.819+0000 7f1b6b7fe700 1 --2- 172.21.15.203:0/715951500 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1b640563d0 0x7f1b64058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:27.830 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.819+0000 7f1b6b7fe700 1 -- 172.21.15.203:0/715951500 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f1b7c071bb0 con 0x7f1b8414e310 2021-01-06T22:16:27.830 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.819+0000 7f1b837fe700 1 --2- 172.21.15.203:0/715951500 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1b640563d0 0x7f1b64058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:27.831 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.819+0000 7f1b837fe700 1 --2- 172.21.15.203:0/715951500 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1b640563d0 0x7f1b64058890 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f1b74000fd0 tx=0x7f1b74000b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:27.831 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:27.827+0000 7f1b6b7fe700 1 -- 172.21.15.203:0/715951500 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f1b8415b6b0 con 0x7f1b8414e310 2021-01-06T22:16:28.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:28.051+0000 7f1b88bee700 1 -- 172.21.15.203:0/715951500 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b840450a0 con 0x7f1b8414e310 2021-01-06T22:16:28.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:28.051+0000 7f1b6b7fe700 1 -- 172.21.15.203:0/715951500 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+3208 (secure 0 0 0) 0x7f1b7c047250 con 0x7f1b8414e310 2021-01-06T22:16:28.055 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:28.055 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:20.595529+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:28.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:28.055+0000 7f1b88bee700 1 -- 172.21.15.203:0/715951500 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1b640563d0 msgr2=0x7f1b64058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:28.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:28.055+0000 7f1b88bee700 1 --2- 172.21.15.203:0/715951500 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1b640563d0 0x7f1b64058890 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f1b74000fd0 tx=0x7f1b74000b80).stop 2021-01-06T22:16:28.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:28.055+0000 7f1b88bee700 1 -- 172.21.15.203:0/715951500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b8414e310 msgr2=0x7f1b8415be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:28.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:28.055+0000 7f1b88bee700 1 --2- 172.21.15.203:0/715951500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b8414e310 0x7f1b8415be60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f1b84063d70 tx=0x7f1b7c017000).stop 2021-01-06T22:16:28.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:28.055+0000 7f1b88bee700 1 -- 172.21.15.203:0/715951500 shutdown_connections 2021-01-06T22:16:28.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:28.055+0000 7f1b88bee700 1 --2- 172.21.15.203:0/715951500 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1b640563d0 0x7f1b64058890 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:28.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:28.055+0000 7f1b88bee700 1 --2- 172.21.15.203:0/715951500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b8414e310 0x7f1b8415be60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:28.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:28.055+0000 7f1b88bee700 1 --2- 172.21.15.203:0/715951500 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1b84064380 0x7f1b840b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:28.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:28.055+0000 7f1b88bee700 1 --2- 172.21.15.203:0/715951500 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1b841478c0 0x7f1b8415b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:28.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:28.055+0000 7f1b88bee700 1 -- 172.21.15.203:0/715951500 >> 172.21.15.203:0/715951500 conn(0x7f1b840667b0 msgr2=0x7f1b840bff20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:28.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:28.055+0000 7f1b88bee700 1 -- 172.21.15.203:0/715951500 shutdown_connections 2021-01-06T22:16:28.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:28.055+0000 7f1b88bee700 1 -- 172.21.15.203:0/715951500 wait complete. 2021-01-06T22:16:28.060 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 61 2021-01-06T22:16:28.070 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2021-01-06T22:16:30.072 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:30.202 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.199+0000 7fe8c66b6700 1 Processor -- start 2021-01-06T22:16:30.203 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.199+0000 7fe8c66b6700 1 -- start start 2021-01-06T22:16:30.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c66b6700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe8c00c44f0 0x7fe8c00c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:30.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c66b6700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe8c00c48d0 0x7fe8c00c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:30.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c66b6700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe8c0060fc0 0x7fe8c0150730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:30.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c66b6700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c00c1d60 con 0x7fe8c00c48d0 2021-01-06T22:16:30.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c66b6700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c00c0f50 con 0x7fe8c00c44f0 2021-01-06T22:16:30.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c66b6700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c0066cf0 con 0x7fe8c0060fc0 2021-01-06T22:16:30.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c56b4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe8c00c44f0 0x7fe8c00c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:30.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c56b4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe8c00c44f0 0x7fe8c00c1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52124/0 (socket says 172.21.15.203:52124) 2021-01-06T22:16:30.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c56b4700 1 -- 172.21.15.203:0/2751617247 learned_addr learned my addr 172.21.15.203:0/2751617247 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:30.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c4eb3700 1 --2- 172.21.15.203:0/2751617247 >> v2:172.21.15.91:3300/0 conn(0x7fe8c00c48d0 0x7fe8c00c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:30.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c5eb5700 1 --2- 172.21.15.203:0/2751617247 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe8c0060fc0 0x7fe8c0150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:30.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c56b4700 1 -- 172.21.15.203:0/2751617247 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe8c0060fc0 msgr2=0x7fe8c0150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:30.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c56b4700 1 --2- 172.21.15.203:0/2751617247 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe8c0060fc0 0x7fe8c0150730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:30.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c56b4700 1 -- 172.21.15.203:0/2751617247 >> v2:172.21.15.91:3300/0 conn(0x7fe8c00c48d0 msgr2=0x7fe8c00c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:30.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c56b4700 1 --2- 172.21.15.203:0/2751617247 >> v2:172.21.15.91:3300/0 conn(0x7fe8c00c48d0 0x7fe8c00c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:30.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c56b4700 1 -- 172.21.15.203:0/2751617247 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c0060750 con 0x7fe8c00c44f0 2021-01-06T22:16:30.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c56b4700 1 --2- 172.21.15.203:0/2751617247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8c00c44f0 0x7fe8c00c1380 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7fe8c005c9f0 tx=0x7fe8bc006ea0).ready entity=mon.1 client_cookie=77d84e8ad4b9985b server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:30.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8affff700 1 -- 172.21.15.203:0/2751617247 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0098d0 con 0x7fe8c00c44f0 2021-01-06T22:16:30.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8affff700 1 -- 172.21.15.203:0/2751617247 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8bc009a30 con 0x7fe8c00c44f0 2021-01-06T22:16:30.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8affff700 1 -- 172.21.15.203:0/2751617247 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc009db0 con 0x7fe8c00c44f0 2021-01-06T22:16:30.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c66b6700 1 -- 172.21.15.203:0/2751617247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8c00c44f0 msgr2=0x7fe8c00c1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:30.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c66b6700 1 --2- 172.21.15.203:0/2751617247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8c00c44f0 0x7fe8c00c1380 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7fe8c005c9f0 tx=0x7fe8bc006ea0).stop 2021-01-06T22:16:30.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c66b6700 1 -- 172.21.15.203:0/2751617247 shutdown_connections 2021-01-06T22:16:30.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c66b6700 1 --2- 172.21.15.203:0/2751617247 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe8c0060fc0 0x7fe8c0150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:30.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c66b6700 1 --2- 172.21.15.203:0/2751617247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8c00c44f0 0x7fe8c00c1380 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:30.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c66b6700 1 --2- 172.21.15.203:0/2751617247 >> v2:172.21.15.91:3300/0 conn(0x7fe8c00c48d0 0x7fe8c00c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:30.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c66b6700 1 -- 172.21.15.203:0/2751617247 >> 172.21.15.203:0/2751617247 conn(0x7fe8c0056cd0 msgr2=0x7fe8c0061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:30.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c66b6700 1 -- 172.21.15.203:0/2751617247 shutdown_connections 2021-01-06T22:16:30.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.203+0000 7fe8c66b6700 1 -- 172.21.15.203:0/2751617247 wait complete. 2021-01-06T22:16:30.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c66b6700 1 Processor -- start 2021-01-06T22:16:30.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c66b6700 1 -- start start 2021-01-06T22:16:30.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c66b6700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8c0060fc0 0x7fe8c015ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:30.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c66b6700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe8c00c44f0 0x7fe8c00ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:30.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c66b6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8c00c48d0 0x7fe8c01616d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:30.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c66b6700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00c09f0 con 0x7fe8c0060fc0 2021-01-06T22:16:30.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c66b6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00b8100 con 0x7fe8c00c48d0 2021-01-06T22:16:30.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c66b6700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00b83a0 con 0x7fe8c00c44f0 2021-01-06T22:16:30.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c5eb5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8c00c48d0 0x7fe8c01616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:30.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c5eb5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8c00c48d0 0x7fe8c01616d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52134/0 (socket says 172.21.15.203:52134) 2021-01-06T22:16:30.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c5eb5700 1 -- 172.21.15.203:0/4076720297 learned_addr learned my addr 172.21.15.203:0/4076720297 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:30.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c56b4700 1 --2- 172.21.15.203:0/4076720297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8c0060fc0 0x7fe8c015ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:30.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c4eb3700 1 --2- 172.21.15.203:0/4076720297 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe8c00c44f0 0x7fe8c00ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:30.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c5eb5700 1 -- 172.21.15.203:0/4076720297 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe8c00c44f0 msgr2=0x7fe8c00ba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:30.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c5eb5700 1 --2- 172.21.15.203:0/4076720297 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe8c00c44f0 0x7fe8c00ba3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:30.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c5eb5700 1 -- 172.21.15.203:0/4076720297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8c0060fc0 msgr2=0x7fe8c015ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:30.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c5eb5700 1 --2- 172.21.15.203:0/4076720297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8c0060fc0 0x7fe8c015ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:30.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c5eb5700 1 -- 172.21.15.203:0/4076720297 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c0150680 con 0x7fe8c00c48d0 2021-01-06T22:16:30.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c5eb5700 1 --2- 172.21.15.203:0/4076720297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8c00c48d0 0x7fe8c01616d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fe8b800cb00 tx=0x7fe8b8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:30.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8ad7fa700 1 -- 172.21.15.203:0/4076720297 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8007750 con 0x7fe8c00c48d0 2021-01-06T22:16:30.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c66b6700 1 -- 172.21.15.203:0/4076720297 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c0150920 con 0x7fe8c00c48d0 2021-01-06T22:16:30.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8ad7fa700 1 -- 172.21.15.203:0/4076720297 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b80078b0 con 0x7fe8c00c48d0 2021-01-06T22:16:30.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8ad7fa700 1 -- 172.21.15.203:0/4076720297 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8007c30 con 0x7fe8c00c48d0 2021-01-06T22:16:30.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c66b6700 1 -- 172.21.15.203:0/4076720297 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c01510e0 con 0x7fe8c00c48d0 2021-01-06T22:16:30.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.207+0000 7fe8c66b6700 1 -- 172.21.15.203:0/4076720297 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c0150b20 con 0x7fe8c00c48d0 2021-01-06T22:16:30.222 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.211+0000 7fe8ad7fa700 1 -- 172.21.15.203:0/4076720297 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fe8b8011620 con 0x7fe8c00c48d0 2021-01-06T22:16:30.222 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.211+0000 7fe8ad7fa700 1 --2- 172.21.15.203:0/4076720297 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe8a00563d0 0x7fe8a0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:30.222 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.211+0000 7fe8ad7fa700 1 -- 172.21.15.203:0/4076720297 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6675+0+0 (secure 0 0 0) 0x7fe8b8070350 con 0x7fe8c00c48d0 2021-01-06T22:16:30.222 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.211+0000 7fe8c56b4700 1 --2- 172.21.15.203:0/4076720297 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe8a00563d0 0x7fe8a0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:30.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.211+0000 7fe8c56b4700 1 --2- 172.21.15.203:0/4076720297 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe8a00563d0 0x7fe8a0058890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fe8bc00b190 tx=0x7fe8bc016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:30.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.215+0000 7fe8ad7fa700 1 -- 172.21.15.203:0/4076720297 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fe8b80459f0 con 0x7fe8c00c48d0 2021-01-06T22:16:30.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.447+0000 7fe8c66b6700 1 -- 172.21.15.203:0/4076720297 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8c00450a0 con 0x7fe8c00c48d0 2021-01-06T22:16:30.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.451+0000 7fe8ad7fa700 1 -- 172.21.15.203:0/4076720297 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3564 (secure 0 0 0) 0x7fe8b8028090 con 0x7fe8c00c48d0 2021-01-06T22:16:30.452 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:30.452 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5539,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":62}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:20.595529+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:30.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.451+0000 7fe8c66b6700 1 -- 172.21.15.203:0/4076720297 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe8a00563d0 msgr2=0x7fe8a0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:30.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.451+0000 7fe8c66b6700 1 --2- 172.21.15.203:0/4076720297 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe8a00563d0 0x7fe8a0058890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fe8bc00b190 tx=0x7fe8bc016000).stop 2021-01-06T22:16:30.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.451+0000 7fe8c66b6700 1 -- 172.21.15.203:0/4076720297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8c00c48d0 msgr2=0x7fe8c01616d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:30.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.451+0000 7fe8c66b6700 1 --2- 172.21.15.203:0/4076720297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8c00c48d0 0x7fe8c01616d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fe8b800cb00 tx=0x7fe8b8008040).stop 2021-01-06T22:16:30.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.451+0000 7fe8c66b6700 1 -- 172.21.15.203:0/4076720297 shutdown_connections 2021-01-06T22:16:30.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.451+0000 7fe8c66b6700 1 --2- 172.21.15.203:0/4076720297 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe8a00563d0 0x7fe8a0058890 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:30.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.451+0000 7fe8c66b6700 1 --2- 172.21.15.203:0/4076720297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8c00c48d0 0x7fe8c01616d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:30.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.451+0000 7fe8c66b6700 1 --2- 172.21.15.203:0/4076720297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8c0060fc0 0x7fe8c015ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:30.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.451+0000 7fe8c66b6700 1 --2- 172.21.15.203:0/4076720297 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe8c00c44f0 0x7fe8c00ba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:30.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.451+0000 7fe8c66b6700 1 -- 172.21.15.203:0/4076720297 >> 172.21.15.203:0/4076720297 conn(0x7fe8c0056cd0 msgr2=0x7fe8c014fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:30.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.451+0000 7fe8c66b6700 1 -- 172.21.15.203:0/4076720297 shutdown_connections 2021-01-06T22:16:30.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:30.451+0000 7fe8c66b6700 1 -- 172.21.15.203:0/4076720297 wait complete. 2021-01-06T22:16:30.457 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 62 2021-01-06T22:16:30.469 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2021-01-06T22:16:30.469 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before thrashing 2021-01-06T22:16:35.469 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:35.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.595+0000 7f28c6028700 1 Processor -- start 2021-01-06T22:16:35.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.595+0000 7f28c6028700 1 -- start start 2021-01-06T22:16:35.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c6028700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f28c01513f0 0x7f28c00c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:35.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c6028700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28c014ad90 0x7f28c014b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:35.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c6028700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c00bacb0 con 0x7f28c00bccb0 2021-01-06T22:16:35.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c6028700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c00bd160 con 0x7f28c01513f0 2021-01-06T22:16:35.601 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c6028700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c014b6d0 con 0x7f28c014ad90 2021-01-06T22:16:35.602 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c5026700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f28c01513f0 0x7f28c00c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:35.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c5827700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28c014ad90 0x7f28c014b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:35.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c5026700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f28c01513f0 0x7f28c00c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:59986/0 (socket says 172.21.15.203:59986) 2021-01-06T22:16:35.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c5026700 1 -- 172.21.15.203:0/211278150 learned_addr learned my addr 172.21.15.203:0/211278150 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:35.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c5827700 1 -- 172.21.15.203:0/211278150 >> v2:172.21.15.91:3300/0 conn(0x7f28c01513f0 msgr2=0x7f28c00c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:35.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c5827700 1 --2- 172.21.15.203:0/211278150 >> v2:172.21.15.91:3300/0 conn(0x7f28c01513f0 0x7f28c00c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:35.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c5827700 1 -- 172.21.15.203:0/211278150 >> v1:172.21.15.91:6789/0 conn(0x7f28c00bccb0 legacy=0x7f28c01517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:35.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c5827700 1 -- 172.21.15.203:0/211278150 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c0152000 con 0x7f28c014ad90 2021-01-06T22:16:35.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c5827700 1 --2- 172.21.15.203:0/211278150 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28c014ad90 0x7f28c014b1e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f28b8009c90 tx=0x7f28b80091d0).ready entity=mon.2 client_cookie=bdbe6354c5e00564 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:35.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28affff700 1 -- 172.21.15.203:0/211278150 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b800ba70 con 0x7f28c014ad90 2021-01-06T22:16:35.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28affff700 1 -- 172.21.15.203:0/211278150 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b800bbd0 con 0x7f28c014ad90 2021-01-06T22:16:35.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28affff700 1 -- 172.21.15.203:0/211278150 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8011400 con 0x7f28c014ad90 2021-01-06T22:16:35.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c6028700 1 -- 172.21.15.203:0/211278150 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28c014ad90 msgr2=0x7f28c014b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:35.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c6028700 1 --2- 172.21.15.203:0/211278150 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28c014ad90 0x7f28c014b1e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f28b8009c90 tx=0x7f28b80091d0).stop 2021-01-06T22:16:35.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c6028700 1 -- 172.21.15.203:0/211278150 shutdown_connections 2021-01-06T22:16:35.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c6028700 1 --2- 172.21.15.203:0/211278150 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28c014ad90 0x7f28c014b1e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:35.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c6028700 1 --2- 172.21.15.203:0/211278150 >> v2:172.21.15.91:3300/0 conn(0x7f28c01513f0 0x7f28c00c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:35.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c6028700 1 -- 172.21.15.203:0/211278150 >> 172.21.15.203:0/211278150 conn(0x7f28c00667b0 msgr2=0x7f28c00c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:35.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c6028700 1 -- 172.21.15.203:0/211278150 shutdown_connections 2021-01-06T22:16:35.607 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.599+0000 7f28c6028700 1 -- 172.21.15.203:0/211278150 wait complete. 2021-01-06T22:16:35.608 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c6028700 1 Processor -- start 2021-01-06T22:16:35.609 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c6028700 1 -- start start 2021-01-06T22:16:35.609 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c6028700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28c00bccb0 0x7f28c0147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:35.609 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c6028700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f28c014ad90 0x7f28c01487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:35.609 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c6028700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28c01513f0 0x7f28c0144610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:35.609 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c6028700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c015d950 con 0x7f28c014ad90 2021-01-06T22:16:35.610 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c6028700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c015dbf0 con 0x7f28c00bccb0 2021-01-06T22:16:35.610 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c6028700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c015de90 con 0x7f28c01513f0 2021-01-06T22:16:35.610 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c5026700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28c00bccb0 0x7f28c0147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:35.610 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c5026700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28c00bccb0 0x7f28c0147ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52144/0 (socket says 172.21.15.203:52144) 2021-01-06T22:16:35.611 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c5026700 1 -- 172.21.15.203:0/2345717695 learned_addr learned my addr 172.21.15.203:0/2345717695 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:35.611 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c4825700 1 --2- 172.21.15.203:0/2345717695 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f28c014ad90 0x7f28c01487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:35.611 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c5827700 1 --2- 172.21.15.203:0/2345717695 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28c01513f0 0x7f28c0144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:35.611 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c5026700 1 -- 172.21.15.203:0/2345717695 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28c01513f0 msgr2=0x7f28c0144610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:35.611 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c5026700 1 --2- 172.21.15.203:0/2345717695 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28c01513f0 0x7f28c0144610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:35.612 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c5026700 1 -- 172.21.15.203:0/2345717695 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f28c014ad90 msgr2=0x7f28c01487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:35.612 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c5026700 1 --2- 172.21.15.203:0/2345717695 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f28c014ad90 0x7f28c01487a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:35.612 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.603+0000 7f28c5026700 1 -- 172.21.15.203:0/2345717695 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c015e130 con 0x7f28c00bccb0 2021-01-06T22:16:35.612 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.607+0000 7f28c5026700 1 --2- 172.21.15.203:0/2345717695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28c00bccb0 0x7f28c0147ea0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f28bc00eba0 tx=0x7f28bc006ea0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:35.612 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.607+0000 7f28ad7fa700 1 -- 172.21.15.203:0/2345717695 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc00b8a0 con 0x7f28c00bccb0 2021-01-06T22:16:35.613 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.607+0000 7f28ad7fa700 1 -- 172.21.15.203:0/2345717695 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc00ba00 con 0x7f28c00bccb0 2021-01-06T22:16:35.613 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.607+0000 7f28ad7fa700 1 -- 172.21.15.203:0/2345717695 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc00bd80 con 0x7f28c00bccb0 2021-01-06T22:16:35.613 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.607+0000 7f28c6028700 1 -- 172.21.15.203:0/2345717695 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c015e3d0 con 0x7f28c00bccb0 2021-01-06T22:16:35.613 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.607+0000 7f28c6028700 1 -- 172.21.15.203:0/2345717695 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c015d640 con 0x7f28c00bccb0 2021-01-06T22:16:35.614 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.607+0000 7f28c6028700 1 -- 172.21.15.203:0/2345717695 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c00450a0 con 0x7f28c00bccb0 2021-01-06T22:16:35.614 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.607+0000 7f28ad7fa700 1 -- 172.21.15.203:0/2345717695 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f28bc022070 con 0x7f28c00bccb0 2021-01-06T22:16:35.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.607+0000 7f28ad7fa700 1 --2- 172.21.15.203:0/2345717695 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f28a0056400 0x7f28a00588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:35.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.607+0000 7f28ad7fa700 1 -- 172.21.15.203:0/2345717695 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f28bc06fa50 con 0x7f28c00bccb0 2021-01-06T22:16:35.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.607+0000 7f28c4825700 1 --2- 172.21.15.203:0/2345717695 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f28a0056400 0x7f28a00588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:35.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.607+0000 7f28c4825700 1 --2- 172.21.15.203:0/2345717695 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f28a0056400 0x7f28a00588c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f28c00672b0 tx=0x7f28b0007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:35.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.615+0000 7f28ad7fa700 1 -- 172.21.15.203:0/2345717695 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f28bc045070 con 0x7f28c00bccb0 2021-01-06T22:16:35.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.843+0000 7f28c6028700 1 -- 172.21.15.203:0/2345717695 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28c005ca60 con 0x7f28c00bccb0 2021-01-06T22:16:35.848 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.847+0000 7f28ad7fa700 1 -- 172.21.15.203:0/2345717695 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3564 (secure 0 0 0) 0x7f28bc048690 con 0x7f28c00bccb0 2021-01-06T22:16:35.848 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:35.848 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5539,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":62}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:20.595529+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:35.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.847+0000 7f28c6028700 1 -- 172.21.15.203:0/2345717695 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f28a0056400 msgr2=0x7f28a00588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:35.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.847+0000 7f28c6028700 1 --2- 172.21.15.203:0/2345717695 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f28a0056400 0x7f28a00588c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f28c00672b0 tx=0x7f28b0007b80).stop 2021-01-06T22:16:35.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.847+0000 7f28c6028700 1 -- 172.21.15.203:0/2345717695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28c00bccb0 msgr2=0x7f28c0147ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:35.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.847+0000 7f28c6028700 1 --2- 172.21.15.203:0/2345717695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28c00bccb0 0x7f28c0147ea0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f28bc00eba0 tx=0x7f28bc006ea0).stop 2021-01-06T22:16:35.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.847+0000 7f28c6028700 1 -- 172.21.15.203:0/2345717695 shutdown_connections 2021-01-06T22:16:35.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.847+0000 7f28c6028700 1 --2- 172.21.15.203:0/2345717695 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f28a0056400 0x7f28a00588c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:35.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.847+0000 7f28c6028700 1 --2- 172.21.15.203:0/2345717695 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28c01513f0 0x7f28c0144610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:35.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.847+0000 7f28c6028700 1 --2- 172.21.15.203:0/2345717695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28c00bccb0 0x7f28c0147ea0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:35.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.847+0000 7f28c6028700 1 --2- 172.21.15.203:0/2345717695 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f28c014ad90 0x7f28c01487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:35.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.847+0000 7f28c6028700 1 -- 172.21.15.203:0/2345717695 >> 172.21.15.203:0/2345717695 conn(0x7f28c00667b0 msgr2=0x7f28c00613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:35.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.847+0000 7f28c6028700 1 -- 172.21.15.203:0/2345717695 shutdown_connections 2021-01-06T22:16:35.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:35.847+0000 7f28c6028700 1 -- 172.21.15.203:0/2345717695 wait complete. 2021-01-06T22:16:35.854 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 62 2021-01-06T22:16:35.862 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=0) 2021-01-06T22:16:35.863 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-01-06T22:16:35.863 INFO:teuthology.orchestra.run:waiting for 300 2021-01-06T22:16:35.873 INFO:tasks.ceph.mds.a:Stopped 2021-01-06T22:16:35.873 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:36.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.051+0000 7f0532f01700 1 Processor -- start 2021-01-06T22:16:36.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.051+0000 7f0532f01700 1 -- start start 2021-01-06T22:16:36.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f0532f01700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f052c061130 0x7f052c0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:36.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f0532f01700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f052c0bc700 0x7f052c14bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:36.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f0532f01700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052c14c1c0 con 0x7f052c14f880 2021-01-06T22:16:36.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f0532f01700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052c14c460 con 0x7f052c061130 2021-01-06T22:16:36.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f0532f01700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052c14c700 con 0x7f052c0bc700 2021-01-06T22:16:36.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f05316fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f052c0bc700 0x7f052c14bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:36.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f0532700700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f052c14f880 0x7f052c148d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50752/0 (socket says 172.21.15.203:50752) 2021-01-06T22:16:36.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f0532700700 1 -- 172.21.15.203:0/4032126579 learned_addr learned my addr 172.21.15.203:0/4032126579 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:16:36.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f0531eff700 1 --2- 172.21.15.203:0/4032126579 >> v2:172.21.15.91:3300/0 conn(0x7f052c061130 0x7f052c0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:36.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f0530efd700 1 -- 172.21.15.203:0/4032126579 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199272165 0 0) 0x7f052c14c1c0 con 0x7f052c14f880 2021-01-06T22:16:36.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f0530efd700 1 -- 172.21.15.203:0/4032126579 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0514003230 con 0x7f052c14f880 2021-01-06T22:16:36.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f05316fe700 1 -- 172.21.15.203:0/4032126579 >> v2:172.21.15.91:3300/0 conn(0x7f052c061130 msgr2=0x7f052c0c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:36.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f0530efd700 1 -- 172.21.15.203:0/4032126579 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2621711247 0 0) 0x7f0514003230 con 0x7f052c14f880 2021-01-06T22:16:36.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f05316fe700 1 --2- 172.21.15.203:0/4032126579 >> v2:172.21.15.91:3300/0 conn(0x7f052c061130 0x7f052c0c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:36.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f05316fe700 1 -- 172.21.15.203:0/4032126579 >> v1:172.21.15.203:6789/0 conn(0x7f052c14f880 legacy=0x7f052c148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:36.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f05316fe700 1 -- 172.21.15.203:0/4032126579 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c14c9a0 con 0x7f052c0bc700 2021-01-06T22:16:36.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f05316fe700 1 --2- 172.21.15.203:0/4032126579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c0bc700 0x7f052c14bb30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f052800b980 tx=0x7f0528006de0).ready entity=mon.1 client_cookie=5bd0bff65c940588 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:36.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f0530efd700 1 -- 172.21.15.203:0/4032126579 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524001860 con 0x7f052c0bc700 2021-01-06T22:16:36.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f0530efd700 1 -- 172.21.15.203:0/4032126579 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0528011400 con 0x7f052c0bc700 2021-01-06T22:16:36.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.055+0000 7f0530efd700 1 -- 172.21.15.203:0/4032126579 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528011780 con 0x7f052c0bc700 2021-01-06T22:16:36.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 -- 172.21.15.203:0/4032126579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c0bc700 msgr2=0x7f052c14bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:36.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 --2- 172.21.15.203:0/4032126579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c0bc700 0x7f052c14bb30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f052800b980 tx=0x7f0528006de0).stop 2021-01-06T22:16:36.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 -- 172.21.15.203:0/4032126579 shutdown_connections 2021-01-06T22:16:36.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 --2- 172.21.15.203:0/4032126579 >> v2:172.21.15.91:3300/0 conn(0x7f052c061130 0x7f052c0c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:36.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 --2- 172.21.15.203:0/4032126579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c0bc700 0x7f052c14bb30 secure :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0x7f052800b980 tx=0x7f0528006de0).stop 2021-01-06T22:16:36.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 -- 172.21.15.203:0/4032126579 >> 172.21.15.203:0/4032126579 conn(0x7f052c0667b0 msgr2=0x7f052c0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:36.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 -- 172.21.15.203:0/4032126579 shutdown_connections 2021-01-06T22:16:36.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 -- 172.21.15.203:0/4032126579 wait complete. 2021-01-06T22:16:36.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 Processor -- start 2021-01-06T22:16:36.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 -- start start 2021-01-06T22:16:36.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f052c061130 0x7f052c14b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:36.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f052c14f880 0x7f052c15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:36.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c0b4680 0x7f052c15efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:36.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c14fcd0 con 0x7f052c061130 2021-01-06T22:16:36.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c1516e0 con 0x7f052c0b4680 2021-01-06T22:16:36.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c14ff00 con 0x7f052c14f880 2021-01-06T22:16:36.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532700700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c0b4680 0x7f052c15efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:36.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532700700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c0b4680 0x7f052c15efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52164/0 (socket says 172.21.15.203:52164) 2021-01-06T22:16:36.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532700700 1 -- 172.21.15.203:0/1694768830 learned_addr learned my addr 172.21.15.203:0/1694768830 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:36.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0531eff700 1 --2- 172.21.15.203:0/1694768830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f052c061130 0x7f052c14b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:36.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532700700 1 -- 172.21.15.203:0/1694768830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f052c14f880 msgr2=0x7f052c15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:36.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532700700 1 --2- 172.21.15.203:0/1694768830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f052c14f880 0x7f052c15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-01-06T22:16:36.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532700700 1 -- 172.21.15.203:0/1694768830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f052c061130 msgr2=0x7f052c14b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:36.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532700700 1 --2- 172.21.15.203:0/1694768830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f052c061130 0x7f052c14b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:36.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532700700 1 -- 172.21.15.203:0/1694768830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c1501a0 con 0x7f052c0b4680 2021-01-06T22:16:36.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0531eff700 1 --2- 172.21.15.203:0/1694768830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f052c061130 0x7f052c14b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:16:36.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532700700 1 --2- 172.21.15.203:0/1694768830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c0b4680 0x7f052c15efb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f052400cbb0 tx=0x7f0524001d60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:36.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0519ffb700 1 -- 172.21.15.203:0/1694768830 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524004a80 con 0x7f052c0b4680 2021-01-06T22:16:36.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0519ffb700 1 -- 172.21.15.203:0/1694768830 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0524004be0 con 0x7f052c0b4680 2021-01-06T22:16:36.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 -- 172.21.15.203:0/1694768830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052c151940 con 0x7f052c0b4680 2021-01-06T22:16:36.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0519ffb700 1 -- 172.21.15.203:0/1694768830 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524007170 con 0x7f052c0b4680 2021-01-06T22:16:36.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.059+0000 7f0532f01700 1 -- 172.21.15.203:0/1694768830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c14ba40 con 0x7f052c0b4680 2021-01-06T22:16:36.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.063+0000 7f0532f01700 1 -- 172.21.15.203:0/1694768830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052c151ed0 con 0x7f052c0b4680 2021-01-06T22:16:36.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.063+0000 7f0519ffb700 1 -- 172.21.15.203:0/1694768830 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f0524010070 con 0x7f052c0b4680 2021-01-06T22:16:36.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.063+0000 7f0519ffb700 1 --2- 172.21.15.203:0/1694768830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f050c0563d0 0x7f050c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:36.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.063+0000 7f0519ffb700 1 -- 172.21.15.203:0/1694768830 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f052406f650 con 0x7f052c0b4680 2021-01-06T22:16:36.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.067+0000 7f0531eff700 1 --2- 172.21.15.203:0/1694768830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f050c0563d0 0x7f050c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:36.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.067+0000 7f0531eff700 1 --2- 172.21.15.203:0/1694768830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f050c0563d0 0x7f050c058890 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f051c00bb20 tx=0x7f051c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:36.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.071+0000 7f0519ffb700 1 -- 172.21.15.203:0/1694768830 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f0524044cc0 con 0x7f052c0b4680 2021-01-06T22:16:36.296 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.295+0000 7f0532f01700 1 -- 172.21.15.203:0/1694768830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f052c0450a0 con 0x7f052c0b4680 2021-01-06T22:16:36.297 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.295+0000 7f0519ffb700 1 -- 172.21.15.203:0/1694768830 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3564 (secure 0 0 0) 0x7f05240482e0 con 0x7f052c0b4680 2021-01-06T22:16:36.297 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:36.298 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5539,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":62}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:20.595529+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:36.300 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.299+0000 7f0532f01700 1 -- 172.21.15.203:0/1694768830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f050c0563d0 msgr2=0x7f050c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:36.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.299+0000 7f0532f01700 1 --2- 172.21.15.203:0/1694768830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f050c0563d0 0x7f050c058890 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f051c00bb20 tx=0x7f051c008040).stop 2021-01-06T22:16:36.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.299+0000 7f0532f01700 1 -- 172.21.15.203:0/1694768830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c0b4680 msgr2=0x7f052c15efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:36.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.299+0000 7f0532f01700 1 --2- 172.21.15.203:0/1694768830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c0b4680 0x7f052c15efb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f052400cbb0 tx=0x7f0524001d60).stop 2021-01-06T22:16:36.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.299+0000 7f0532f01700 1 -- 172.21.15.203:0/1694768830 shutdown_connections 2021-01-06T22:16:36.301 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.299+0000 7f0532f01700 1 --2- 172.21.15.203:0/1694768830 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f050c0563d0 0x7f050c058890 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:36.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.299+0000 7f0532f01700 1 --2- 172.21.15.203:0/1694768830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c0b4680 0x7f052c15efb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:36.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.299+0000 7f0532f01700 1 --2- 172.21.15.203:0/1694768830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f052c061130 0x7f052c14b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:36.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.299+0000 7f0532f01700 1 --2- 172.21.15.203:0/1694768830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f052c14f880 0x7f052c15eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-01-06T22:16:36.302 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.299+0000 7f0532f01700 1 -- 172.21.15.203:0/1694768830 >> 172.21.15.203:0/1694768830 conn(0x7f052c0667b0 msgr2=0x7f052c0c0870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:36.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.299+0000 7f0532f01700 1 -- 172.21.15.203:0/1694768830 shutdown_connections 2021-01-06T22:16:36.303 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:36.299+0000 7f0532f01700 1 -- 172.21.15.203:0/1694768830 wait complete. 2021-01-06T22:16:36.303 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 62 2021-01-06T22:16:36.313 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:16:38.315 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:38.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f424304a700 1 Processor -- start 2021-01-06T22:16:38.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f424304a700 1 -- start start 2021-01-06T22:16:38.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f424304a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f423c0a5e50 0x7f423c0ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:38.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f424304a700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f423c0bb7d0 0x7f423c14eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:38.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f424304a700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f423c0646b0 con 0x7f423c0bb3f0 2021-01-06T22:16:38.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f424304a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423c0670c0 con 0x7f423c0a5e50 2021-01-06T22:16:38.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f424304a700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c064450 con 0x7f423c0bb7d0 2021-01-06T22:16:38.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f4241847700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f423c0a5e50 0x7f423c0ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:38.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f4241847700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f423c0a5e50 0x7f423c0ba9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52174/0 (socket says 172.21.15.203:52174) 2021-01-06T22:16:38.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f4241847700 1 -- 172.21.15.203:0/3843859794 learned_addr learned my addr 172.21.15.203:0/3843859794 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:38.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f4242849700 1 --2- 172.21.15.203:0/3843859794 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f423c0bb7d0 0x7f423c14eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:38.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f4241847700 1 -- 172.21.15.203:0/3843859794 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f423c0bb7d0 msgr2=0x7f423c14eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:38.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f4241847700 1 --2- 172.21.15.203:0/3843859794 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f423c0bb7d0 0x7f423c14eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:38.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f4241847700 1 -- 172.21.15.203:0/3843859794 >> v1:172.21.15.91:6789/0 conn(0x7f423c0bb3f0 legacy=0x7f423c0669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:38.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f4241847700 1 -- 172.21.15.203:0/3843859794 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c152cc0 con 0x7f423c0a5e50 2021-01-06T22:16:38.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f4241847700 1 --2- 172.21.15.203:0/3843859794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0a5e50 0x7f423c0ba9d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f422400b9e0 tx=0x7f4224006ea0).ready entity=mon.1 client_cookie=b61e4abda5035b46 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:38.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f4241046700 1 -- 172.21.15.203:0/3843859794 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4224009890 con 0x7f423c0a5e50 2021-01-06T22:16:38.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f4241046700 1 -- 172.21.15.203:0/3843859794 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42240099f0 con 0x7f423c0a5e50 2021-01-06T22:16:38.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.443+0000 7f4241046700 1 -- 172.21.15.203:0/3843859794 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4224009d70 con 0x7f423c0a5e50 2021-01-06T22:16:38.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.447+0000 7f424304a700 1 -- 172.21.15.203:0/3843859794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0a5e50 msgr2=0x7f423c0ba9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:38.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.447+0000 7f424304a700 1 --2- 172.21.15.203:0/3843859794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0a5e50 0x7f423c0ba9d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f422400b9e0 tx=0x7f4224006ea0).stop 2021-01-06T22:16:38.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.447+0000 7f424304a700 1 -- 172.21.15.203:0/3843859794 shutdown_connections 2021-01-06T22:16:38.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.447+0000 7f424304a700 1 --2- 172.21.15.203:0/3843859794 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f423c0bb7d0 0x7f423c14eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:38.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.447+0000 7f424304a700 1 --2- 172.21.15.203:0/3843859794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0a5e50 0x7f423c0ba9d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:38.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.447+0000 7f424304a700 1 -- 172.21.15.203:0/3843859794 >> 172.21.15.203:0/3843859794 conn(0x7f423c061530 msgr2=0x7f423c063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:38.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.447+0000 7f424304a700 1 -- 172.21.15.203:0/3843859794 shutdown_connections 2021-01-06T22:16:38.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.447+0000 7f424304a700 1 -- 172.21.15.203:0/3843859794 wait complete. 2021-01-06T22:16:38.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.447+0000 7f424304a700 1 Processor -- start 2021-01-06T22:16:38.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.447+0000 7f424304a700 1 -- start start 2021-01-06T22:16:38.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f424304a700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f423c0a5e50 0x7f423c09e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:38.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f424304a700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f423c0bb3f0 0x7f423c165aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:38.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f424304a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0bb7d0 0x7f423c162430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:38.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f424304a700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c161b00 con 0x7f423c0bb3f0 2021-01-06T22:16:38.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f424304a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c165f90 con 0x7f423c0bb7d0 2021-01-06T22:16:38.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f424304a700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0a12b0 con 0x7f423c0a5e50 2021-01-06T22:16:38.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f4242849700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0bb7d0 0x7f423c162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:38.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f4242849700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0bb7d0 0x7f423c162430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52184/0 (socket says 172.21.15.203:52184) 2021-01-06T22:16:38.568 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f4242849700 1 -- 172.21.15.203:0/3974537318 learned_addr learned my addr 172.21.15.203:0/3974537318 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:38.568 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f4242048700 1 --2- 172.21.15.203:0/3974537318 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f423c0a5e50 0x7f423c09e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:38.569 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f4241847700 1 --2- 172.21.15.203:0/3974537318 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f423c0bb3f0 0x7f423c165aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:38.569 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f4242849700 1 -- 172.21.15.203:0/3974537318 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f423c0a5e50 msgr2=0x7f423c09e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:38.569 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f4242849700 1 --2- 172.21.15.203:0/3974537318 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f423c0a5e50 0x7f423c09e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:38.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f4242849700 1 -- 172.21.15.203:0/3974537318 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f423c0bb3f0 msgr2=0x7f423c165aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:38.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f4242849700 1 --2- 172.21.15.203:0/3974537318 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f423c0bb3f0 0x7f423c165aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:38.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f4242849700 1 -- 172.21.15.203:0/3974537318 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c0a1550 con 0x7f423c0bb7d0 2021-01-06T22:16:38.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f4242849700 1 --2- 172.21.15.203:0/3974537318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0bb7d0 0x7f423c162430 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f423400cad0 tx=0x7f4234008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:38.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f423a7fc700 1 -- 172.21.15.203:0/3974537318 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42340076b0 con 0x7f423c0bb7d0 2021-01-06T22:16:38.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f423a7fc700 1 -- 172.21.15.203:0/3974537318 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4234007810 con 0x7f423c0bb7d0 2021-01-06T22:16:38.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f424304a700 1 -- 172.21.15.203:0/3974537318 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423c0a17f0 con 0x7f423c0bb7d0 2021-01-06T22:16:38.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f423a7fc700 1 -- 172.21.15.203:0/3974537318 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234007bb0 con 0x7f423c0bb7d0 2021-01-06T22:16:38.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.451+0000 7f424304a700 1 -- 172.21.15.203:0/3974537318 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423c161f90 con 0x7f423c0bb7d0 2021-01-06T22:16:38.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.455+0000 7f424304a700 1 -- 172.21.15.203:0/3974537318 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423c0450a0 con 0x7f423c0bb7d0 2021-01-06T22:16:38.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.455+0000 7f423a7fc700 1 -- 172.21.15.203:0/3974537318 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f423401b460 con 0x7f423c0bb7d0 2021-01-06T22:16:38.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.455+0000 7f423a7fc700 1 --2- 172.21.15.203:0/3974537318 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4220056400 0x7f42200588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:38.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.455+0000 7f4242048700 1 --2- 172.21.15.203:0/3974537318 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4220056400 0x7f42200588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:38.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.455+0000 7f423a7fc700 1 -- 172.21.15.203:0/3974537318 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f423406fa20 con 0x7f423c0bb7d0 2021-01-06T22:16:38.576 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.455+0000 7f4242048700 1 --2- 172.21.15.203:0/3974537318 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4220056400 0x7f42200588c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f422c0036c0 tx=0x7f422c007b60).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:38.576 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.463+0000 7f423a7fc700 1 -- 172.21.15.203:0/3974537318 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f4234044fe0 con 0x7f423c0bb7d0 2021-01-06T22:16:38.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.667+0000 7f424304a700 1 -- 172.21.15.203:0/3974537318 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f423c05ca60 con 0x7f423c0bb7d0 2021-01-06T22:16:38.669 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.667+0000 7f423a7fc700 1 -- 172.21.15.203:0/3974537318 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3564 (secure 0 0 0) 0x7f4234048600 con 0x7f423c0bb7d0 2021-01-06T22:16:38.669 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:38.670 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5539,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":62}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:20.595529+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:38.672 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.667+0000 7f424304a700 1 -- 172.21.15.203:0/3974537318 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4220056400 msgr2=0x7f42200588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:38.672 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.667+0000 7f424304a700 1 --2- 172.21.15.203:0/3974537318 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4220056400 0x7f42200588c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f422c0036c0 tx=0x7f422c007b60).stop 2021-01-06T22:16:38.672 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.667+0000 7f424304a700 1 -- 172.21.15.203:0/3974537318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0bb7d0 msgr2=0x7f423c162430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:38.673 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.667+0000 7f424304a700 1 --2- 172.21.15.203:0/3974537318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0bb7d0 0x7f423c162430 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f423400cad0 tx=0x7f4234008040).stop 2021-01-06T22:16:38.673 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.667+0000 7f424304a700 1 -- 172.21.15.203:0/3974537318 shutdown_connections 2021-01-06T22:16:38.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.667+0000 7f424304a700 1 --2- 172.21.15.203:0/3974537318 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f4220056400 0x7f42200588c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:38.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.667+0000 7f424304a700 1 --2- 172.21.15.203:0/3974537318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0bb7d0 0x7f423c162430 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:38.674 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.667+0000 7f424304a700 1 --2- 172.21.15.203:0/3974537318 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f423c0a5e50 0x7f423c09e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:38.675 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.667+0000 7f424304a700 1 --2- 172.21.15.203:0/3974537318 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f423c0bb3f0 0x7f423c165aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:38.675 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.667+0000 7f424304a700 1 -- 172.21.15.203:0/3974537318 >> 172.21.15.203:0/3974537318 conn(0x7f423c061530 msgr2=0x7f423c0a55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:38.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.667+0000 7f424304a700 1 -- 172.21.15.203:0/3974537318 shutdown_connections 2021-01-06T22:16:38.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:38.667+0000 7f424304a700 1 -- 172.21.15.203:0/3974537318 wait complete. 2021-01-06T22:16:38.677 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 62 2021-01-06T22:16:38.687 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:16:40.688 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:40.827 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.823+0000 7f4160f9b700 1 Processor -- start 2021-01-06T22:16:40.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.823+0000 7f4160f9b700 1 -- start start 2021-01-06T22:16:40.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f4160f9b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f415c0bcd50 0x7f415c0bd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:40.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f4160f9b700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f415c14e2e0 0x7f415c1438a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:40.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f4160f9b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415c0b8260 con 0x7f415c0b7e00 2021-01-06T22:16:40.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f4160f9b700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f415c14e770 con 0x7f415c14e2e0 2021-01-06T22:16:40.909 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f4160f9b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f415c14b250 con 0x7f415c0bcd50 2021-01-06T22:16:40.909 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f415a59c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f415c0bcd50 0x7f415c0bd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:40.909 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f4159d9b700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f415c0b7e00 0x7f415c0638b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50786/0 (socket says 172.21.15.203:50786) 2021-01-06T22:16:40.909 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f4159d9b700 1 -- 172.21.15.203:0/1588125299 learned_addr learned my addr 172.21.15.203:0/1588125299 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:16:40.910 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f415ad9d700 1 --2- 172.21.15.203:0/1588125299 >> v2:172.21.15.91:3300/0 conn(0x7f415c14e2e0 0x7f415c1438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:40.910 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f415a59c700 1 -- 172.21.15.203:0/1588125299 >> v2:172.21.15.91:3300/0 conn(0x7f415c14e2e0 msgr2=0x7f415c1438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:40.910 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f415a59c700 1 --2- 172.21.15.203:0/1588125299 >> v2:172.21.15.91:3300/0 conn(0x7f415c14e2e0 0x7f415c1438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:40.910 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f415a59c700 1 -- 172.21.15.203:0/1588125299 >> v1:172.21.15.203:6789/0 conn(0x7f415c0b7e00 legacy=0x7f415c0638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:40.911 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f415a59c700 1 -- 172.21.15.203:0/1588125299 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c0b8490 con 0x7f415c0bcd50 2021-01-06T22:16:40.911 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f415ad9d700 1 --2- 172.21.15.203:0/1588125299 >> v2:172.21.15.91:3300/0 conn(0x7f415c14e2e0 0x7f415c1438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:16:40.911 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f415a59c700 1 --2- 172.21.15.203:0/1588125299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c0bcd50 0x7f415c0bd130 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f415c061730 tx=0x7f4144006de0).ready entity=mon.1 client_cookie=d44fb294a76f06b1 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:40.911 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f415959a700 1 -- 172.21.15.203:0/1588125299 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144009840 con 0x7f415c0bcd50 2021-01-06T22:16:40.911 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f415959a700 1 -- 172.21.15.203:0/1588125299 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41440099a0 con 0x7f415c0bcd50 2021-01-06T22:16:40.912 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f415959a700 1 -- 172.21.15.203:0/1588125299 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144009d20 con 0x7f415c0bcd50 2021-01-06T22:16:40.912 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f4160f9b700 1 -- 172.21.15.203:0/1588125299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c0bcd50 msgr2=0x7f415c0bd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:40.912 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f4160f9b700 1 --2- 172.21.15.203:0/1588125299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c0bcd50 0x7f415c0bd130 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f415c061730 tx=0x7f4144006de0).stop 2021-01-06T22:16:40.912 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f4160f9b700 1 -- 172.21.15.203:0/1588125299 shutdown_connections 2021-01-06T22:16:40.912 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f4160f9b700 1 --2- 172.21.15.203:0/1588125299 >> v2:172.21.15.91:3300/0 conn(0x7f415c14e2e0 0x7f415c1438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:40.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f4160f9b700 1 --2- 172.21.15.203:0/1588125299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c0bcd50 0x7f415c0bd130 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:40.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f4160f9b700 1 -- 172.21.15.203:0/1588125299 >> 172.21.15.203:0/1588125299 conn(0x7f415c056cd0 msgr2=0x7f415c0626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:40.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f4160f9b700 1 -- 172.21.15.203:0/1588125299 shutdown_connections 2021-01-06T22:16:40.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.827+0000 7f4160f9b700 1 -- 172.21.15.203:0/1588125299 wait complete. 2021-01-06T22:16:40.913 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.831+0000 7f4160f9b700 1 Processor -- start 2021-01-06T22:16:40.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.831+0000 7f4160f9b700 1 -- start start 2021-01-06T22:16:40.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f4160f9b700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c0b7e00 0x7f415c14b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:40.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f4160f9b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415c0bcd50 0x7f415c066e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:40.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f4160f9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c14e2e0 0x7f415c0bf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:40.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f4160f9b700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c15c930 con 0x7f415c0b7e00 2021-01-06T22:16:40.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f4160f9b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c141520 con 0x7f415c14e2e0 2021-01-06T22:16:40.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f4160f9b700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c1417c0 con 0x7f415c0bcd50 2021-01-06T22:16:40.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f415ad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c14e2e0 0x7f415c0bf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:40.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f415ad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c14e2e0 0x7f415c0bf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52198/0 (socket says 172.21.15.203:52198) 2021-01-06T22:16:40.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f415ad9d700 1 -- 172.21.15.203:0/3393863877 learned_addr learned my addr 172.21.15.203:0/3393863877 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:40.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f415a59c700 1 --2- 172.21.15.203:0/3393863877 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c0b7e00 0x7f415c14b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:40.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f4159d9b700 1 --2- 172.21.15.203:0/3393863877 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415c0bcd50 0x7f415c066e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:40.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f415ad9d700 1 -- 172.21.15.203:0/3393863877 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415c0bcd50 msgr2=0x7f415c066e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:40.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f415ad9d700 1 --2- 172.21.15.203:0/3393863877 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415c0bcd50 0x7f415c066e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:40.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f415ad9d700 1 -- 172.21.15.203:0/3393863877 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c0b7e00 msgr2=0x7f415c14b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:40.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f415ad9d700 1 --2- 172.21.15.203:0/3393863877 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c0b7e00 0x7f415c14b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:40.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f415ad9d700 1 -- 172.21.15.203:0/3393863877 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c15cb90 con 0x7f415c14e2e0 2021-01-06T22:16:40.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f415ad9d700 1 --2- 172.21.15.203:0/3393863877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c14e2e0 0x7f415c0bf650 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f415400e380 tx=0x7f415400b630).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:40.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f41527fc700 1 -- 172.21.15.203:0/3393863877 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415401a470 con 0x7f415c14e2e0 2021-01-06T22:16:40.918 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f41527fc700 1 -- 172.21.15.203:0/3393863877 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415401a5d0 con 0x7f415c14e2e0 2021-01-06T22:16:40.918 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f4160f9b700 1 -- 172.21.15.203:0/3393863877 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415c15ce30 con 0x7f415c14e2e0 2021-01-06T22:16:40.918 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f41527fc700 1 -- 172.21.15.203:0/3393863877 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415401a970 con 0x7f415c14e2e0 2021-01-06T22:16:40.918 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f4160f9b700 1 -- 172.21.15.203:0/3393863877 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415c14bc20 con 0x7f415c14e2e0 2021-01-06T22:16:40.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.835+0000 7f4160f9b700 1 -- 172.21.15.203:0/3393863877 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f415c05cfe0 con 0x7f415c14e2e0 2021-01-06T22:16:40.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.839+0000 7f41527fc700 1 -- 172.21.15.203:0/3393863877 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f4154021060 con 0x7f415c14e2e0 2021-01-06T22:16:40.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.839+0000 7f41527fc700 1 --2- 172.21.15.203:0/3393863877 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41400563d0 0x7f4140058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:40.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.839+0000 7f41527fc700 1 -- 172.21.15.203:0/3393863877 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f4154072580 con 0x7f415c14e2e0 2021-01-06T22:16:40.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.839+0000 7f415a59c700 1 --2- 172.21.15.203:0/3393863877 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41400563d0 0x7f4140058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:40.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.839+0000 7f415a59c700 1 --2- 172.21.15.203:0/3393863877 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41400563d0 0x7f4140058890 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f414400b130 tx=0x7f4144016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:40.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:40.843+0000 7f41527fc700 1 -- 172.21.15.203:0/3393863877 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f4154047ba0 con 0x7f415c14e2e0 2021-01-06T22:16:41.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:41.039+0000 7f4160f9b700 1 -- 172.21.15.203:0/3393863877 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f415c0576d0 con 0x7f415c14e2e0 2021-01-06T22:16:41.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:41.043+0000 7f41527fc700 1 -- 172.21.15.203:0/3393863877 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3564 (secure 0 0 0) 0x7f415401f020 con 0x7f415c14e2e0 2021-01-06T22:16:41.044 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:41.044 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5539,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":62}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:20.595529+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:41.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:41.043+0000 7f4160f9b700 1 -- 172.21.15.203:0/3393863877 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41400563d0 msgr2=0x7f4140058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:41.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:41.043+0000 7f4160f9b700 1 --2- 172.21.15.203:0/3393863877 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41400563d0 0x7f4140058890 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f414400b130 tx=0x7f4144016000).stop 2021-01-06T22:16:41.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:41.043+0000 7f4160f9b700 1 -- 172.21.15.203:0/3393863877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c14e2e0 msgr2=0x7f415c0bf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:41.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:41.043+0000 7f4160f9b700 1 --2- 172.21.15.203:0/3393863877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c14e2e0 0x7f415c0bf650 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f415400e380 tx=0x7f415400b630).stop 2021-01-06T22:16:41.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:41.043+0000 7f4160f9b700 1 -- 172.21.15.203:0/3393863877 shutdown_connections 2021-01-06T22:16:41.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:41.043+0000 7f4160f9b700 1 --2- 172.21.15.203:0/3393863877 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f41400563d0 0x7f4140058890 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:41.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:41.043+0000 7f4160f9b700 1 --2- 172.21.15.203:0/3393863877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c14e2e0 0x7f415c0bf650 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:41.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:41.043+0000 7f4160f9b700 1 --2- 172.21.15.203:0/3393863877 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c0b7e00 0x7f415c14b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:41.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:41.043+0000 7f4160f9b700 1 --2- 172.21.15.203:0/3393863877 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415c0bcd50 0x7f415c066e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:41.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:41.043+0000 7f4160f9b700 1 -- 172.21.15.203:0/3393863877 >> 172.21.15.203:0/3393863877 conn(0x7f415c056cd0 msgr2=0x7f415c143890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:41.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:41.043+0000 7f4160f9b700 1 -- 172.21.15.203:0/3393863877 shutdown_connections 2021-01-06T22:16:41.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:41.043+0000 7f4160f9b700 1 -- 172.21.15.203:0/3393863877 wait complete. 2021-01-06T22:16:41.049 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 62 2021-01-06T22:16:41.058 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:16:43.059 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:43.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.207+0000 7fa2b88fe700 1 Processor -- start 2021-01-06T22:16:43.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.207+0000 7fa2b88fe700 1 -- start start 2021-01-06T22:16:43.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b88fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2b40bb3f0 0x7fa2b40ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:43.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b88fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa2b40a5e50 0x7fa2b40669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:43.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b88fe700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b40646b0 con 0x7fa2b40bb7d0 2021-01-06T22:16:43.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b88fe700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b4064450 con 0x7fa2b40a5e50 2021-01-06T22:16:43.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b88fe700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b4068530 con 0x7fa2b40bb3f0 2021-01-06T22:16:43.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b2ffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa2b40a5e50 0x7fa2b40669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:43.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b3fff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fa2b40bb7d0 0x7fa2b40602b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50802/0 (socket says 172.21.15.203:50802) 2021-01-06T22:16:43.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b3fff700 1 -- 172.21.15.203:0/3854290496 learned_addr learned my addr 172.21.15.203:0/3854290496 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:16:43.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b37fe700 1 --2- 172.21.15.203:0/3854290496 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2b40bb3f0 0x7fa2b40ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:43.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b27fc700 1 -- 172.21.15.203:0/3854290496 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402467899 0 0) 0x7fa2b40646b0 con 0x7fa2b40bb7d0 2021-01-06T22:16:43.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b27fc700 1 -- 172.21.15.203:0/3854290496 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2a0003230 con 0x7fa2b40bb7d0 2021-01-06T22:16:43.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b2ffd700 1 -- 172.21.15.203:0/3854290496 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2b40bb3f0 msgr2=0x7fa2b40ba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:43.223 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b2ffd700 1 --2- 172.21.15.203:0/3854290496 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2b40bb3f0 0x7fa2b40ba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:43.224 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b2ffd700 1 -- 172.21.15.203:0/3854290496 >> v1:172.21.15.203:6789/0 conn(0x7fa2b40bb7d0 legacy=0x7fa2b40602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:43.224 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b2ffd700 1 -- 172.21.15.203:0/3854290496 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b4152cc0 con 0x7fa2b40a5e50 2021-01-06T22:16:43.224 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b2ffd700 1 --2- 172.21.15.203:0/3854290496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2b40a5e50 0x7fa2b40669b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fa2a800b980 tx=0x7fa2a8006de0).ready entity=mon.1 client_cookie=423df42859da561b server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:43.224 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b27fc700 1 -- 172.21.15.203:0/3854290496 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80097b0 con 0x7fa2b40a5e50 2021-01-06T22:16:43.225 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b27fc700 1 -- 172.21.15.203:0/3854290496 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a8009910 con 0x7fa2b40a5e50 2021-01-06T22:16:43.225 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b27fc700 1 -- 172.21.15.203:0/3854290496 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8009c90 con 0x7fa2b40a5e50 2021-01-06T22:16:43.225 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b88fe700 1 -- 172.21.15.203:0/3854290496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2b40a5e50 msgr2=0x7fa2b40669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:43.225 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b88fe700 1 --2- 172.21.15.203:0/3854290496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2b40a5e50 0x7fa2b40669b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fa2a800b980 tx=0x7fa2a8006de0).stop 2021-01-06T22:16:43.226 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b88fe700 1 -- 172.21.15.203:0/3854290496 shutdown_connections 2021-01-06T22:16:43.226 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b88fe700 1 --2- 172.21.15.203:0/3854290496 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2b40bb3f0 0x7fa2b40ba9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:43.226 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b88fe700 1 --2- 172.21.15.203:0/3854290496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2b40a5e50 0x7fa2b40669b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:43.226 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b88fe700 1 -- 172.21.15.203:0/3854290496 >> 172.21.15.203:0/3854290496 conn(0x7fa2b4061530 msgr2=0x7fa2b4063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:43.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b88fe700 1 -- 172.21.15.203:0/3854290496 shutdown_connections 2021-01-06T22:16:43.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b88fe700 1 -- 172.21.15.203:0/3854290496 wait complete. 2021-01-06T22:16:43.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b88fe700 1 Processor -- start 2021-01-06T22:16:43.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.211+0000 7fa2b88fe700 1 -- start start 2021-01-06T22:16:43.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b88fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2b40a5e50 0x7fa2b4162fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:43.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b88fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2b40bb3f0 0x7fa2b41634b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:43.229 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b88fe700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2b40bb7d0 0x7fa2b414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:43.229 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b88fe700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b41639a0 con 0x7fa2b40bb7d0 2021-01-06T22:16:43.229 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b88fe700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b4153380 con 0x7fa2b40bb3f0 2021-01-06T22:16:43.229 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b88fe700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b40b9490 con 0x7fa2b40a5e50 2021-01-06T22:16:43.230 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b2ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2b40bb3f0 0x7fa2b41634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:43.230 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b2ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2b40bb3f0 0x7fa2b41634b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52214/0 (socket says 172.21.15.203:52214) 2021-01-06T22:16:43.230 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b2ffd700 1 -- 172.21.15.203:0/39710952 learned_addr learned my addr 172.21.15.203:0/39710952 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:43.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b37fe700 1 --2- 172.21.15.203:0/39710952 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2b40a5e50 0x7fa2b4162fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:43.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b3fff700 1 --2- 172.21.15.203:0/39710952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2b40bb7d0 0x7fa2b414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:43.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b2ffd700 1 -- 172.21.15.203:0/39710952 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2b40a5e50 msgr2=0x7fa2b4162fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:43.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b2ffd700 1 --2- 172.21.15.203:0/39710952 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2b40a5e50 0x7fa2b4162fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:43.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b2ffd700 1 -- 172.21.15.203:0/39710952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2b40bb7d0 msgr2=0x7fa2b414e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:43.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b2ffd700 1 --2- 172.21.15.203:0/39710952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2b40bb7d0 0x7fa2b414e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:43.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b2ffd700 1 -- 172.21.15.203:0/39710952 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b40b96f0 con 0x7fa2b40bb3f0 2021-01-06T22:16:43.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b3fff700 1 --2- 172.21.15.203:0/39710952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2b40bb7d0 0x7fa2b414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:16:43.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b37fe700 1 --2- 172.21.15.203:0/39710952 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2b40a5e50 0x7fa2b4162fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:16:43.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b2ffd700 1 --2- 172.21.15.203:0/39710952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2b40bb3f0 0x7fa2b41634b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fa2a8006f80 tx=0x7fa2a8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:43.234 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa29b7fe700 1 -- 172.21.15.203:0/39710952 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8011b40 con 0x7fa2b40bb3f0 2021-01-06T22:16:43.234 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa29b7fe700 1 -- 172.21.15.203:0/39710952 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a8011cc0 con 0x7fa2b40bb3f0 2021-01-06T22:16:43.234 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa29b7fe700 1 -- 172.21.15.203:0/39710952 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a801d650 con 0x7fa2b40bb3f0 2021-01-06T22:16:43.234 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b88fe700 1 -- 172.21.15.203:0/39710952 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b40b9990 con 0x7fa2b40bb3f0 2021-01-06T22:16:43.235 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b88fe700 1 -- 172.21.15.203:0/39710952 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b40b9ed0 con 0x7fa2b40bb3f0 2021-01-06T22:16:43.235 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.215+0000 7fa2b88fe700 1 -- 172.21.15.203:0/39710952 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b4162e30 con 0x7fa2b40bb3f0 2021-01-06T22:16:43.235 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.219+0000 7fa29b7fe700 1 -- 172.21.15.203:0/39710952 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa2a800f070 con 0x7fa2b40bb3f0 2021-01-06T22:16:43.236 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.219+0000 7fa29b7fe700 1 --2- 172.21.15.203:0/39710952 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa2940563d0 0x7fa294058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:43.236 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.219+0000 7fa29b7fe700 1 -- 172.21.15.203:0/39710952 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6675+0+0 (secure 0 0 0) 0x7fa2a8070df0 con 0x7fa2b40bb3f0 2021-01-06T22:16:43.236 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.219+0000 7fa2b37fe700 1 --2- 172.21.15.203:0/39710952 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa2940563d0 0x7fa294058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:43.236 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.219+0000 7fa2b37fe700 1 --2- 172.21.15.203:0/39710952 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa2940563d0 0x7fa294058890 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fa2a400de70 tx=0x7fa2a4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:43.237 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.227+0000 7fa29b7fe700 1 -- 172.21.15.203:0/39710952 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa2a8046410 con 0x7fa2b40bb3f0 2021-01-06T22:16:43.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.427+0000 7fa2b88fe700 1 -- 172.21.15.203:0/39710952 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2b40450a0 con 0x7fa2b40bb3f0 2021-01-06T22:16:43.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.427+0000 7fa29b7fe700 1 -- 172.21.15.203:0/39710952 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3564 (secure 0 0 0) 0x7fa2a8021070 con 0x7fa2b40bb3f0 2021-01-06T22:16:43.431 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:43.432 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5539,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":62}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:20.595529+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:43.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.431+0000 7fa2b88fe700 1 -- 172.21.15.203:0/39710952 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa2940563d0 msgr2=0x7fa294058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:43.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.431+0000 7fa2b88fe700 1 --2- 172.21.15.203:0/39710952 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa2940563d0 0x7fa294058890 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fa2a400de70 tx=0x7fa2a4008040).stop 2021-01-06T22:16:43.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.431+0000 7fa2b88fe700 1 -- 172.21.15.203:0/39710952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2b40bb3f0 msgr2=0x7fa2b41634b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:43.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.431+0000 7fa2b88fe700 1 --2- 172.21.15.203:0/39710952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2b40bb3f0 0x7fa2b41634b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fa2a8006f80 tx=0x7fa2a8016000).stop 2021-01-06T22:16:43.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.431+0000 7fa2b88fe700 1 -- 172.21.15.203:0/39710952 shutdown_connections 2021-01-06T22:16:43.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.431+0000 7fa2b88fe700 1 --2- 172.21.15.203:0/39710952 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa2940563d0 0x7fa294058890 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:43.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.431+0000 7fa2b88fe700 1 --2- 172.21.15.203:0/39710952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2b40bb7d0 0x7fa2b414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:43.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.431+0000 7fa2b88fe700 1 --2- 172.21.15.203:0/39710952 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2b40a5e50 0x7fa2b4162fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:43.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.431+0000 7fa2b88fe700 1 --2- 172.21.15.203:0/39710952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2b40bb3f0 0x7fa2b41634b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:43.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.431+0000 7fa2b88fe700 1 -- 172.21.15.203:0/39710952 >> 172.21.15.203:0/39710952 conn(0x7fa2b4061530 msgr2=0x7fa2b414f780 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:43.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.431+0000 7fa2b88fe700 1 -- 172.21.15.203:0/39710952 shutdown_connections 2021-01-06T22:16:43.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:43.431+0000 7fa2b88fe700 1 -- 172.21.15.203:0/39710952 wait complete. 2021-01-06T22:16:43.437 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 62 2021-01-06T22:16:43.454 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:16:45.454 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:45.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.583+0000 7fa9f735f700 1 Processor -- start 2021-01-06T22:16:45.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.583+0000 7fa9f735f700 1 -- start start 2021-01-06T22:16:45.593 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.583+0000 7fa9f735f700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa9f00bb3f0 0x7fa9f00ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:45.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.583+0000 7fa9f735f700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9f00a5e50 0x7fa9f00669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:45.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.583+0000 7fa9f735f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa9f00bb7d0 0x7fa9f00602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:45.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.583+0000 7fa9f735f700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f00646b0 con 0x7fa9f00bb3f0 2021-01-06T22:16:45.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.583+0000 7fa9f735f700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f0152cc0 con 0x7fa9f00bb7d0 2021-01-06T22:16:45.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.583+0000 7fa9f735f700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f0064450 con 0x7fa9f00a5e50 2021-01-06T22:16:45.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.583+0000 7fa9f6b5e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa9f00bb7d0 0x7fa9f00602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:45.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.583+0000 7fa9f6b5e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa9f00bb7d0 0x7fa9f00602b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52218/0 (socket says 172.21.15.203:52218) 2021-01-06T22:16:45.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.583+0000 7fa9f6b5e700 1 -- 172.21.15.203:0/1086616770 learned_addr learned my addr 172.21.15.203:0/1086616770 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:45.596 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f5b5c700 1 --2- 172.21.15.203:0/1086616770 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9f00a5e50 0x7fa9f00669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:45.596 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f635d700 1 --2- 172.21.15.203:0/1086616770 >> v2:172.21.15.91:3300/0 conn(0x7fa9f00bb3f0 0x7fa9f00ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:45.596 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f6b5e700 1 -- 172.21.15.203:0/1086616770 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9f00a5e50 msgr2=0x7fa9f00669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:45.596 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f6b5e700 1 --2- 172.21.15.203:0/1086616770 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9f00a5e50 0x7fa9f00669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:45.596 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f6b5e700 1 -- 172.21.15.203:0/1086616770 >> v2:172.21.15.91:3300/0 conn(0x7fa9f00bb3f0 msgr2=0x7fa9f00ba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:45.597 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f6b5e700 1 --2- 172.21.15.203:0/1086616770 >> v2:172.21.15.91:3300/0 conn(0x7fa9f00bb3f0 0x7fa9f00ba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:45.597 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f6b5e700 1 -- 172.21.15.203:0/1086616770 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f0068530 con 0x7fa9f00bb7d0 2021-01-06T22:16:45.597 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f5b5c700 1 --2- 172.21.15.203:0/1086616770 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9f00a5e50 0x7fa9f00669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:16:45.597 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f6b5e700 1 --2- 172.21.15.203:0/1086616770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9f00bb7d0 0x7fa9f00602b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fa9f00a62d0 tx=0x7fa9e8009290).ready entity=mon.1 client_cookie=25ff987346e59484 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:45.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f535b700 1 -- 172.21.15.203:0/1086616770 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e800bbc0 con 0x7fa9f00bb7d0 2021-01-06T22:16:45.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f535b700 1 -- 172.21.15.203:0/1086616770 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e800bd20 con 0x7fa9f00bb7d0 2021-01-06T22:16:45.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f535b700 1 -- 172.21.15.203:0/1086616770 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8011610 con 0x7fa9f00bb7d0 2021-01-06T22:16:45.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f735f700 1 -- 172.21.15.203:0/1086616770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9f00bb7d0 msgr2=0x7fa9f00602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:45.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f735f700 1 --2- 172.21.15.203:0/1086616770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9f00bb7d0 0x7fa9f00602b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fa9f00a62d0 tx=0x7fa9e8009290).stop 2021-01-06T22:16:45.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f735f700 1 -- 172.21.15.203:0/1086616770 shutdown_connections 2021-01-06T22:16:45.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f735f700 1 --2- 172.21.15.203:0/1086616770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9f00bb7d0 0x7fa9f00602b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:45.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f735f700 1 --2- 172.21.15.203:0/1086616770 >> v2:172.21.15.91:3300/0 conn(0x7fa9f00bb3f0 0x7fa9f00ba9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:45.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f735f700 1 --2- 172.21.15.203:0/1086616770 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9f00a5e50 0x7fa9f00669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:45.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f735f700 1 -- 172.21.15.203:0/1086616770 >> 172.21.15.203:0/1086616770 conn(0x7fa9f0061530 msgr2=0x7fa9f0063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:45.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f735f700 1 -- 172.21.15.203:0/1086616770 shutdown_connections 2021-01-06T22:16:45.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f735f700 1 -- 172.21.15.203:0/1086616770 wait complete. 2021-01-06T22:16:45.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f735f700 1 Processor -- start 2021-01-06T22:16:45.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.587+0000 7fa9f735f700 1 -- start start 2021-01-06T22:16:45.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f735f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9f00a5e50 0x7fa9f0158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:45.601 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f735f700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9f00bb3f0 0x7fa9f014e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:45.601 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f735f700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa9f00bb7d0 0x7fa9f014e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:45.601 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f735f700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f0161780 con 0x7fa9f00bb7d0 2021-01-06T22:16:45.601 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f735f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f0164ef0 con 0x7fa9f00a5e50 2021-01-06T22:16:45.602 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f735f700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f01619e0 con 0x7fa9f00bb3f0 2021-01-06T22:16:45.602 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f635d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9f00a5e50 0x7fa9f0158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:45.602 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f635d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9f00a5e50 0x7fa9f0158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52224/0 (socket says 172.21.15.203:52224) 2021-01-06T22:16:45.602 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f635d700 1 -- 172.21.15.203:0/2190222461 learned_addr learned my addr 172.21.15.203:0/2190222461 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:45.602 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f5b5c700 1 --2- 172.21.15.203:0/2190222461 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9f00bb3f0 0x7fa9f014e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:45.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f6b5e700 1 --2- 172.21.15.203:0/2190222461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa9f00bb7d0 0x7fa9f014e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:45.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f635d700 1 -- 172.21.15.203:0/2190222461 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9f00bb3f0 msgr2=0x7fa9f014e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:45.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f635d700 1 --2- 172.21.15.203:0/2190222461 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9f00bb3f0 0x7fa9f014e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:45.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f635d700 1 -- 172.21.15.203:0/2190222461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa9f00bb7d0 msgr2=0x7fa9f014e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:45.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f635d700 1 --2- 172.21.15.203:0/2190222461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa9f00bb7d0 0x7fa9f014e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:45.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f635d700 1 -- 172.21.15.203:0/2190222461 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f0161c80 con 0x7fa9f00a5e50 2021-01-06T22:16:45.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f6b5e700 1 --2- 172.21.15.203:0/2190222461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa9f00bb7d0 0x7fa9f014e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:16:45.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f635d700 1 --2- 172.21.15.203:0/2190222461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9f00a5e50 0x7fa9f0158dc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fa9e000d8e0 tx=0x7fa9e0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:45.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9e67fc700 1 -- 172.21.15.203:0/2190222461 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0009650 con 0x7fa9f00a5e50 2021-01-06T22:16:45.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9e67fc700 1 -- 172.21.15.203:0/2190222461 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e00097b0 con 0x7fa9f00a5e50 2021-01-06T22:16:45.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f735f700 1 -- 172.21.15.203:0/2190222461 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f0161f20 con 0x7fa9f00a5e50 2021-01-06T22:16:45.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9e67fc700 1 -- 172.21.15.203:0/2190222461 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0009b50 con 0x7fa9f00a5e50 2021-01-06T22:16:45.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f735f700 1 -- 172.21.15.203:0/2190222461 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f0164bb0 con 0x7fa9f00a5e50 2021-01-06T22:16:45.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f735f700 1 -- 172.21.15.203:0/2190222461 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f00450a0 con 0x7fa9f00a5e50 2021-01-06T22:16:45.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9e67fc700 1 -- 172.21.15.203:0/2190222461 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa9e0013480 con 0x7fa9f00a5e50 2021-01-06T22:16:45.607 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9e67fc700 1 --2- 172.21.15.203:0/2190222461 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa9d00563d0 0x7fa9d0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:45.607 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9e67fc700 1 -- 172.21.15.203:0/2190222461 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6675+0+0 (secure 0 0 0) 0x7fa9e006fc70 con 0x7fa9f00a5e50 2021-01-06T22:16:45.607 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.591+0000 7fa9f5b5c700 1 --2- 172.21.15.203:0/2190222461 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa9d00563d0 0x7fa9d0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:45.607 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.595+0000 7fa9f5b5c700 1 --2- 172.21.15.203:0/2190222461 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa9d00563d0 0x7fa9d0058890 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fa9f0153dc0 tx=0x7fa9ec008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:45.608 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.599+0000 7fa9e67fc700 1 -- 172.21.15.203:0/2190222461 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa9e0045230 con 0x7fa9f00a5e50 2021-01-06T22:16:45.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.799+0000 7fa9f735f700 1 -- 172.21.15.203:0/2190222461 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9f0057980 con 0x7fa9f00a5e50 2021-01-06T22:16:45.804 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.803+0000 7fa9e67fc700 1 -- 172.21.15.203:0/2190222461 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3564 (secure 0 0 0) 0x7fa9e0048850 con 0x7fa9f00a5e50 2021-01-06T22:16:45.804 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:45.805 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5539,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":62}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:20.595529+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:45.807 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.803+0000 7fa9f735f700 1 -- 172.21.15.203:0/2190222461 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa9d00563d0 msgr2=0x7fa9d0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:45.807 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.803+0000 7fa9f735f700 1 --2- 172.21.15.203:0/2190222461 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa9d00563d0 0x7fa9d0058890 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fa9f0153dc0 tx=0x7fa9ec008040).stop 2021-01-06T22:16:45.807 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.803+0000 7fa9f735f700 1 -- 172.21.15.203:0/2190222461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9f00a5e50 msgr2=0x7fa9f0158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:45.807 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.803+0000 7fa9f735f700 1 --2- 172.21.15.203:0/2190222461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9f00a5e50 0x7fa9f0158dc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fa9e000d8e0 tx=0x7fa9e0008040).stop 2021-01-06T22:16:45.808 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.803+0000 7fa9f735f700 1 -- 172.21.15.203:0/2190222461 shutdown_connections 2021-01-06T22:16:45.808 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.803+0000 7fa9f735f700 1 --2- 172.21.15.203:0/2190222461 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa9d00563d0 0x7fa9d0058890 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:45.808 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.803+0000 7fa9f735f700 1 --2- 172.21.15.203:0/2190222461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa9f00bb7d0 0x7fa9f014e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:45.808 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.803+0000 7fa9f735f700 1 --2- 172.21.15.203:0/2190222461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9f00a5e50 0x7fa9f0158dc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:45.808 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.803+0000 7fa9f735f700 1 --2- 172.21.15.203:0/2190222461 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa9f00bb3f0 0x7fa9f014e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:45.809 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.803+0000 7fa9f735f700 1 -- 172.21.15.203:0/2190222461 >> 172.21.15.203:0/2190222461 conn(0x7fa9f0061530 msgr2=0x7fa9f00bf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:45.809 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.803+0000 7fa9f735f700 1 -- 172.21.15.203:0/2190222461 shutdown_connections 2021-01-06T22:16:45.809 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:45.803+0000 7fa9f735f700 1 -- 172.21.15.203:0/2190222461 wait complete. 2021-01-06T22:16:45.810 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 62 2021-01-06T22:16:45.820 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:16:47.822 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:47.831 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:16:47.834 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:16:47.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.955+0000 7f38549f8700 1 Processor -- start 2021-01-06T22:16:47.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.955+0000 7f38549f8700 1 -- start start 2021-01-06T22:16:47.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.955+0000 7f38549f8700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38500be300 0x7f385014ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:47.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.955+0000 7f38549f8700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38500bc3f0 con 0x7f3850147f40 2021-01-06T22:16:47.969 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.955+0000 7f38549f8700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38500bc690 con 0x7f3850147b60 2021-01-06T22:16:47.969 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.955+0000 7f38549f8700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38500bc1c0 con 0x7f38500be300 2021-01-06T22:16:47.969 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.955+0000 7f384ed9d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f3850147b60 0x7f38500c1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50830/0 (socket says 172.21.15.203:50830) 2021-01-06T22:16:47.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.955+0000 7f384ed9d700 1 -- 172.21.15.203:0/2399605708 learned_addr learned my addr 172.21.15.203:0/2399605708 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:16:47.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.955+0000 7f384f59e700 1 --2- 172.21.15.203:0/2399605708 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38500be300 0x7f385014ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:47.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.955+0000 7f384dd9b700 1 -- 172.21.15.203:0/2399605708 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4011213151 0 0) 0x7f38500bc690 con 0x7f3850147b60 2021-01-06T22:16:47.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.955+0000 7f384dd9b700 1 -- 172.21.15.203:0/2399605708 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3834003230 con 0x7f3850147b60 2021-01-06T22:16:47.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.955+0000 7f384dd9b700 1 -- 172.21.15.203:0/2399605708 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3493799519 0 0) 0x7f38500bc3f0 con 0x7f3850147f40 2021-01-06T22:16:47.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.955+0000 7f384dd9b700 1 -- 172.21.15.203:0/2399605708 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38500bc690 con 0x7f3850147f40 2021-01-06T22:16:47.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.955+0000 7f384f59e700 1 -- 172.21.15.203:0/2399605708 >> v1:172.21.15.203:6789/0 conn(0x7f3850147b60 legacy=0x7f38500c1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:47.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.955+0000 7f384f59e700 1 -- 172.21.15.203:0/2399605708 >> v1:172.21.15.91:6789/0 conn(0x7f3850147f40 legacy=0x7f38500bdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:47.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.955+0000 7f384f59e700 1 -- 172.21.15.203:0/2399605708 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38500bc930 con 0x7f38500be300 2021-01-06T22:16:47.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.959+0000 7f384f59e700 1 --2- 172.21.15.203:0/2399605708 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38500be300 0x7f385014ddb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f3840009cc0 tx=0x7f38400091d0).ready entity=mon.2 client_cookie=e255dadbdbd3503f server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:47.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.959+0000 7f384dd9b700 1 -- 172.21.15.203:0/2399605708 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384000bab0 con 0x7f38500be300 2021-01-06T22:16:47.975 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.959+0000 7f384dd9b700 1 -- 172.21.15.203:0/2399605708 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384000bc10 con 0x7f38500be300 2021-01-06T22:16:47.975 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.959+0000 7f384dd9b700 1 -- 172.21.15.203:0/2399605708 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840011400 con 0x7f38500be300 2021-01-06T22:16:47.975 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.959+0000 7f38549f8700 1 -- 172.21.15.203:0/2399605708 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38500be300 msgr2=0x7f385014ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:47.975 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.959+0000 7f38549f8700 1 --2- 172.21.15.203:0/2399605708 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38500be300 0x7f385014ddb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f3840009cc0 tx=0x7f38400091d0).stop 2021-01-06T22:16:47.976 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.959+0000 7f38549f8700 1 -- 172.21.15.203:0/2399605708 shutdown_connections 2021-01-06T22:16:47.976 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.959+0000 7f38549f8700 1 --2- 172.21.15.203:0/2399605708 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38500be300 0x7f385014ddb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:47.977 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.959+0000 7f38549f8700 1 -- 172.21.15.203:0/2399605708 >> 172.21.15.203:0/2399605708 conn(0x7f3850056cd0 msgr2=0x7f38500570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:47.977 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.959+0000 7f38549f8700 1 -- 172.21.15.203:0/2399605708 shutdown_connections 2021-01-06T22:16:47.977 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.959+0000 7f38549f8700 1 -- 172.21.15.203:0/2399605708 wait complete. 2021-01-06T22:16:47.978 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.959+0000 7f38549f8700 1 Processor -- start 2021-01-06T22:16:47.978 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.959+0000 7f38549f8700 1 -- start start 2021-01-06T22:16:47.978 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f38549f8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38500be300 0x7f385015dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:47.978 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f38549f8700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3850147b60 0x7f3850145210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:47.979 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f38549f8700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3850147f40 0x7f3850161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:47.979 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f38549f8700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3850150ab0 con 0x7f3850147b60 2021-01-06T22:16:47.979 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f38549f8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38500602b0 con 0x7f38500be300 2021-01-06T22:16:47.979 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f38549f8700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3850060550 con 0x7f3850147f40 2021-01-06T22:16:47.980 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f384ed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38500be300 0x7f385015dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:47.980 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f384ed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38500be300 0x7f385015dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52240/0 (socket says 172.21.15.203:52240) 2021-01-06T22:16:47.980 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f384ed9d700 1 -- 172.21.15.203:0/1290098415 learned_addr learned my addr 172.21.15.203:0/1290098415 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:47.980 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f384f59e700 1 --2- 172.21.15.203:0/1290098415 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3850147f40 0x7f3850161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:47.981 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f384e59c700 1 --2- 172.21.15.203:0/1290098415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3850147b60 0x7f3850145210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:47.981 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f384ed9d700 1 -- 172.21.15.203:0/1290098415 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3850147f40 msgr2=0x7f3850161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:47.982 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f384ed9d700 1 --2- 172.21.15.203:0/1290098415 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3850147f40 0x7f3850161010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:47.982 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f384ed9d700 1 -- 172.21.15.203:0/1290098415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3850147b60 msgr2=0x7f3850145210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:47.982 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f384ed9d700 1 --2- 172.21.15.203:0/1290098415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3850147b60 0x7f3850145210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:47.982 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f384ed9d700 1 -- 172.21.15.203:0/1290098415 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38500607f0 con 0x7f38500be300 2021-01-06T22:16:47.983 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f384ed9d700 1 --2- 172.21.15.203:0/1290098415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38500be300 0x7f385015dee0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f384800c760 tx=0x7f3848007b60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:47.983 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f3846ffd700 1 -- 172.21.15.203:0/1290098415 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848007320 con 0x7f38500be300 2021-01-06T22:16:47.983 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f3846ffd700 1 -- 172.21.15.203:0/1290098415 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3848007480 con 0x7f38500be300 2021-01-06T22:16:47.983 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f38549f8700 1 -- 172.21.15.203:0/1290098415 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3850060a90 con 0x7f38500be300 2021-01-06T22:16:47.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f3846ffd700 1 -- 172.21.15.203:0/1290098415 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38480057f0 con 0x7f38500be300 2021-01-06T22:16:47.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f38549f8700 1 -- 172.21.15.203:0/1290098415 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3850160b70 con 0x7f38500be300 2021-01-06T22:16:47.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f38549f8700 1 -- 172.21.15.203:0/1290098415 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3850060f30 con 0x7f38500be300 2021-01-06T22:16:47.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f3846ffd700 1 -- 172.21.15.203:0/1290098415 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f3848010030 con 0x7f38500be300 2021-01-06T22:16:47.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.963+0000 7f3846ffd700 1 --2- 172.21.15.203:0/1290098415 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f38280563d0 0x7f3828058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:47.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.967+0000 7f3846ffd700 1 -- 172.21.15.203:0/1290098415 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f384806f540 con 0x7f38500be300 2021-01-06T22:16:47.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.967+0000 7f384e59c700 1 --2- 172.21.15.203:0/1290098415 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f38280563d0 0x7f3828058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:47.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.967+0000 7f384e59c700 1 --2- 172.21.15.203:0/1290098415 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f38280563d0 0x7f3828058890 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f3850142c40 tx=0x7f3838009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:47.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:47.971+0000 7f3846ffd700 1 -- 172.21.15.203:0/1290098415 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f3848044b60 con 0x7f38500be300 2021-01-06T22:16:48.175 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:48.171+0000 7f38549f8700 1 -- 172.21.15.203:0/1290098415 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3850160980 con 0x7f38500be300 2021-01-06T22:16:48.176 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:48.171+0000 7f3846ffd700 1 -- 172.21.15.203:0/1290098415 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3564 (secure 0 0 0) 0x7f3848048180 con 0x7f38500be300 2021-01-06T22:16:48.176 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:48.177 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5539,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":62}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:20.595529+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:48.179 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:48.175+0000 7f38549f8700 1 -- 172.21.15.203:0/1290098415 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f38280563d0 msgr2=0x7f3828058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:48.180 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:48.175+0000 7f38549f8700 1 --2- 172.21.15.203:0/1290098415 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f38280563d0 0x7f3828058890 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f3850142c40 tx=0x7f3838009040).stop 2021-01-06T22:16:48.180 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:48.175+0000 7f38549f8700 1 -- 172.21.15.203:0/1290098415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38500be300 msgr2=0x7f385015dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:48.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:48.175+0000 7f38549f8700 1 --2- 172.21.15.203:0/1290098415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38500be300 0x7f385015dee0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f384800c760 tx=0x7f3848007b60).stop 2021-01-06T22:16:48.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:48.175+0000 7f38549f8700 1 -- 172.21.15.203:0/1290098415 shutdown_connections 2021-01-06T22:16:48.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:48.175+0000 7f38549f8700 1 --2- 172.21.15.203:0/1290098415 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f38280563d0 0x7f3828058890 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:48.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:48.175+0000 7f38549f8700 1 --2- 172.21.15.203:0/1290098415 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3850147f40 0x7f3850161010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:48.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:48.175+0000 7f38549f8700 1 --2- 172.21.15.203:0/1290098415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38500be300 0x7f385015dee0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:48.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:48.175+0000 7f38549f8700 1 --2- 172.21.15.203:0/1290098415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3850147b60 0x7f3850145210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:48.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:48.175+0000 7f38549f8700 1 -- 172.21.15.203:0/1290098415 >> 172.21.15.203:0/1290098415 conn(0x7f3850056cd0 msgr2=0x7f38500c10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:48.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:48.175+0000 7f38549f8700 1 -- 172.21.15.203:0/1290098415 shutdown_connections 2021-01-06T22:16:48.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:48.175+0000 7f38549f8700 1 -- 172.21.15.203:0/1290098415 wait complete. 2021-01-06T22:16:48.185 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 62 2021-01-06T22:16:48.190 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:16:50.191 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:50.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd069182700 1 Processor -- start 2021-01-06T22:16:50.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd069182700 1 -- start start 2021-01-06T22:16:50.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd069182700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0640c04c0 0x7fd0640680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:50.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd069182700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd0640c08a0 0x7fd064063ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:50.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd069182700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0641531f0 con 0x7fd064155b60 2021-01-06T22:16:50.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd069182700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0641522e0 con 0x7fd0640c08a0 2021-01-06T22:16:50.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd069182700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd064063a10 con 0x7fd0640c04c0 2021-01-06T22:16:50.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd06359e700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fd064155b60 0x7fd06414f310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50848/0 (socket says 172.21.15.203:50848) 2021-01-06T22:16:50.326 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd06359e700 1 -- 172.21.15.203:0/2615115063 learned_addr learned my addr 172.21.15.203:0/2615115063 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:16:50.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd06259c700 1 --2- 172.21.15.203:0/2615115063 >> v2:172.21.15.91:3300/0 conn(0x7fd0640c08a0 0x7fd064063ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:50.327 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd062d9d700 1 --2- 172.21.15.203:0/2615115063 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0640c04c0 0x7fd0640680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:50.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd061d9b700 1 -- 172.21.15.203:0/2615115063 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198724412 0 0) 0x7fd0641531f0 con 0x7fd064155b60 2021-01-06T22:16:50.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd061d9b700 1 -- 172.21.15.203:0/2615115063 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd044003230 con 0x7fd064155b60 2021-01-06T22:16:50.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd062d9d700 1 -- 172.21.15.203:0/2615115063 >> v2:172.21.15.91:3300/0 conn(0x7fd0640c08a0 msgr2=0x7fd064063ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:50.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd062d9d700 1 --2- 172.21.15.203:0/2615115063 >> v2:172.21.15.91:3300/0 conn(0x7fd0640c08a0 0x7fd064063ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:50.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd062d9d700 1 -- 172.21.15.203:0/2615115063 >> v1:172.21.15.203:6789/0 conn(0x7fd064155b60 legacy=0x7fd06414f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:50.330 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd062d9d700 1 -- 172.21.15.203:0/2615115063 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0641524d0 con 0x7fd0640c04c0 2021-01-06T22:16:50.330 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd06259c700 1 --2- 172.21.15.203:0/2615115063 >> v2:172.21.15.91:3300/0 conn(0x7fd0640c08a0 0x7fd064063ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:16:50.330 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd062d9d700 1 --2- 172.21.15.203:0/2615115063 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0640c04c0 0x7fd0640680b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fd04c00b980 tx=0x7fd04c006de0).ready entity=mon.2 client_cookie=28752390230fa46c server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:50.330 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd061d9b700 1 -- 172.21.15.203:0/2615115063 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c0096d0 con 0x7fd0640c04c0 2021-01-06T22:16:50.331 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd061d9b700 1 -- 172.21.15.203:0/2615115063 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c009830 con 0x7fd0640c04c0 2021-01-06T22:16:50.331 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd061d9b700 1 -- 172.21.15.203:0/2615115063 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c009bb0 con 0x7fd0640c04c0 2021-01-06T22:16:50.331 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd069182700 1 -- 172.21.15.203:0/2615115063 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0640c04c0 msgr2=0x7fd0640680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:50.331 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd069182700 1 --2- 172.21.15.203:0/2615115063 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0640c04c0 0x7fd0640680b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fd04c00b980 tx=0x7fd04c006de0).stop 2021-01-06T22:16:50.331 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd069182700 1 -- 172.21.15.203:0/2615115063 shutdown_connections 2021-01-06T22:16:50.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd069182700 1 --2- 172.21.15.203:0/2615115063 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0640c04c0 0x7fd0640680b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:50.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd069182700 1 --2- 172.21.15.203:0/2615115063 >> v2:172.21.15.91:3300/0 conn(0x7fd0640c08a0 0x7fd064063ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:50.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd069182700 1 -- 172.21.15.203:0/2615115063 >> 172.21.15.203:0/2615115063 conn(0x7fd064056cd0 msgr2=0x7fd0640626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:50.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd069182700 1 -- 172.21.15.203:0/2615115063 shutdown_connections 2021-01-06T22:16:50.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.323+0000 7fd069182700 1 -- 172.21.15.203:0/2615115063 wait complete. 2021-01-06T22:16:50.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd069182700 1 Processor -- start 2021-01-06T22:16:50.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd069182700 1 -- start start 2021-01-06T22:16:50.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd069182700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0640c04c0 0x7fd064063e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:50.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd069182700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0640c08a0 0x7fd0640ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:50.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd069182700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd064155b60 0x7fd0640be300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:50.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd069182700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd064165360 con 0x7fd064155b60 2021-01-06T22:16:50.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd069182700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd064165600 con 0x7fd0640c04c0 2021-01-06T22:16:50.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd069182700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0641658a0 con 0x7fd0640c08a0 2021-01-06T22:16:50.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd062d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0640c04c0 0x7fd064063e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:50.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd062d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0640c04c0 0x7fd064063e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52258/0 (socket says 172.21.15.203:52258) 2021-01-06T22:16:50.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd062d9d700 1 -- 172.21.15.203:0/2214397439 learned_addr learned my addr 172.21.15.203:0/2214397439 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:50.337 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd06259c700 1 --2- 172.21.15.203:0/2214397439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0640c08a0 0x7fd0640ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:50.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd06359e700 1 --2- 172.21.15.203:0/2214397439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd064155b60 0x7fd0640be300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:50.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd062d9d700 1 -- 172.21.15.203:0/2214397439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0640c08a0 msgr2=0x7fd0640ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:50.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd062d9d700 1 --2- 172.21.15.203:0/2214397439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0640c08a0 0x7fd0640ae420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:50.338 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd062d9d700 1 -- 172.21.15.203:0/2214397439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd064155b60 msgr2=0x7fd0640be300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:50.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd062d9d700 1 --2- 172.21.15.203:0/2214397439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd064155b60 0x7fd0640be300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:50.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd062d9d700 1 -- 172.21.15.203:0/2214397439 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd064165b40 con 0x7fd0640c04c0 2021-01-06T22:16:50.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd062d9d700 1 --2- 172.21.15.203:0/2214397439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0640c04c0 0x7fd064063e10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fd0640620c0 tx=0x7fd04c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:50.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd05affd700 1 -- 172.21.15.203:0/2214397439 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c020070 con 0x7fd0640c04c0 2021-01-06T22:16:50.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd05affd700 1 -- 172.21.15.203:0/2214397439 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c011df0 con 0x7fd0640c04c0 2021-01-06T22:16:50.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd069182700 1 -- 172.21.15.203:0/2214397439 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd064165de0 con 0x7fd0640c04c0 2021-01-06T22:16:50.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd05affd700 1 -- 172.21.15.203:0/2214397439 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c01c670 con 0x7fd0640c04c0 2021-01-06T22:16:50.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.327+0000 7fd069182700 1 -- 172.21.15.203:0/2214397439 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0640acf20 con 0x7fd0640c04c0 2021-01-06T22:16:50.345 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.331+0000 7fd069182700 1 -- 172.21.15.203:0/2214397439 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06405cfe0 con 0x7fd0640c04c0 2021-01-06T22:16:50.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.331+0000 7fd05affd700 1 -- 172.21.15.203:0/2214397439 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fd04c00f070 con 0x7fd0640c04c0 2021-01-06T22:16:50.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.331+0000 7fd05affd700 1 --2- 172.21.15.203:0/2214397439 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd048056400 0x7fd0480588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:50.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.331+0000 7fd05affd700 1 -- 172.21.15.203:0/2214397439 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6675+0+0 (secure 0 0 0) 0x7fd04c070990 con 0x7fd0640c04c0 2021-01-06T22:16:50.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.331+0000 7fd06259c700 1 --2- 172.21.15.203:0/2214397439 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd048056400 0x7fd0480588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:50.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.331+0000 7fd06259c700 1 --2- 172.21.15.203:0/2214397439 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd048056400 0x7fd0480588c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fd064153630 tx=0x7fd0540091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:50.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.335+0000 7fd05affd700 1 -- 172.21.15.203:0/2214397439 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fd04c045ea0 con 0x7fd0640c04c0 2021-01-06T22:16:50.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.535+0000 7fd069182700 1 -- 172.21.15.203:0/2214397439 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0640447f0 con 0x7fd0640c04c0 2021-01-06T22:16:50.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.535+0000 7fd05affd700 1 -- 172.21.15.203:0/2214397439 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3564 (secure 0 0 0) 0x7fd04c006970 con 0x7fd0640c04c0 2021-01-06T22:16:50.538 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:50.539 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5539,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":62}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:20.595529+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5350,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5350":{"gid":5350,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":37,"addr":"172.21.15.203:6837/4277691165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4277691165},{"type":"v1","addr":"172.21.15.203:6837","nonce":4277691165}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:50.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.539+0000 7fd069182700 1 -- 172.21.15.203:0/2214397439 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd048056400 msgr2=0x7fd0480588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:50.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.539+0000 7fd069182700 1 --2- 172.21.15.203:0/2214397439 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd048056400 0x7fd0480588c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fd064153630 tx=0x7fd0540091d0).stop 2021-01-06T22:16:50.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.539+0000 7fd069182700 1 -- 172.21.15.203:0/2214397439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0640c04c0 msgr2=0x7fd064063e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:50.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.539+0000 7fd069182700 1 --2- 172.21.15.203:0/2214397439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0640c04c0 0x7fd064063e10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fd0640620c0 tx=0x7fd04c016000).stop 2021-01-06T22:16:50.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.539+0000 7fd069182700 1 -- 172.21.15.203:0/2214397439 shutdown_connections 2021-01-06T22:16:50.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.539+0000 7fd069182700 1 --2- 172.21.15.203:0/2214397439 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd048056400 0x7fd0480588c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:50.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.539+0000 7fd069182700 1 --2- 172.21.15.203:0/2214397439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd064155b60 0x7fd0640be300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:50.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.539+0000 7fd069182700 1 --2- 172.21.15.203:0/2214397439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0640c04c0 0x7fd064063e10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:50.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.539+0000 7fd069182700 1 --2- 172.21.15.203:0/2214397439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0640c08a0 0x7fd0640ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:50.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.539+0000 7fd069182700 1 -- 172.21.15.203:0/2214397439 >> 172.21.15.203:0/2214397439 conn(0x7fd064056cd0 msgr2=0x7fd064151010 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:50.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.539+0000 7fd069182700 1 -- 172.21.15.203:0/2214397439 shutdown_connections 2021-01-06T22:16:50.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:50.539+0000 7fd069182700 1 -- 172.21.15.203:0/2214397439 wait complete. 2021-01-06T22:16:50.543 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 62 2021-01-06T22:16:50.558 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:16:52.560 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:52.695 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.691+0000 7f1d6d84c700 1 Processor -- start 2021-01-06T22:16:52.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d6d84c700 1 -- start start 2021-01-06T22:16:52.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d6d84c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d680c44f0 0x7f1d680c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:52.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d6d84c700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1d680c48d0 0x7f1d680c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:52.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d6d84c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1d68060fc0 0x7f1d68150730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:52.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d6d84c700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d680c1d60 con 0x7f1d680c48d0 2021-01-06T22:16:52.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d6d84c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d680c0f50 con 0x7f1d68060fc0 2021-01-06T22:16:52.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d6d84c700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d68066cf0 con 0x7f1d680c44f0 2021-01-06T22:16:52.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d6d04b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1d68060fc0 0x7f1d68150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:52.704 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d6d04b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1d68060fc0 0x7f1d68150730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52278/0 (socket says 172.21.15.203:52278) 2021-01-06T22:16:52.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d6d04b700 1 -- 172.21.15.203:0/2140554296 learned_addr learned my addr 172.21.15.203:0/2140554296 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:52.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d6d04b700 1 -- 172.21.15.203:0/2140554296 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d680c44f0 msgr2=0x7f1d680c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:52.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d6d04b700 1 --2- 172.21.15.203:0/2140554296 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d680c44f0 0x7f1d680c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:16:52.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d6d04b700 1 -- 172.21.15.203:0/2140554296 >> v2:172.21.15.91:3300/0 conn(0x7f1d680c48d0 msgr2=0x7f1d680c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:52.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d6d04b700 1 --2- 172.21.15.203:0/2140554296 >> v2:172.21.15.91:3300/0 conn(0x7f1d680c48d0 0x7f1d680c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:16:52.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d6d04b700 1 -- 172.21.15.203:0/2140554296 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d68060750 con 0x7f1d68060fc0 2021-01-06T22:16:52.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d6d04b700 1 --2- 172.21.15.203:0/2140554296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d68060fc0 0x7f1d68150730 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f1d68063850 tx=0x7f1d60006ea0).ready entity=mon.1 client_cookie=1491656c58efb79 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:52.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d677fe700 1 -- 172.21.15.203:0/2140554296 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d600098d0 con 0x7f1d68060fc0 2021-01-06T22:16:52.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d677fe700 1 -- 172.21.15.203:0/2140554296 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d60009a30 con 0x7f1d68060fc0 2021-01-06T22:16:52.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.695+0000 7f1d677fe700 1 -- 172.21.15.203:0/2140554296 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60009db0 con 0x7f1d68060fc0 2021-01-06T22:16:52.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.699+0000 7f1d6d84c700 1 -- 172.21.15.203:0/2140554296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d68060fc0 msgr2=0x7f1d68150730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:52.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.699+0000 7f1d6d84c700 1 --2- 172.21.15.203:0/2140554296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d68060fc0 0x7f1d68150730 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f1d68063850 tx=0x7f1d60006ea0).stop 2021-01-06T22:16:52.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.699+0000 7f1d6d84c700 1 -- 172.21.15.203:0/2140554296 shutdown_connections 2021-01-06T22:16:52.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.699+0000 7f1d6d84c700 1 --2- 172.21.15.203:0/2140554296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d68060fc0 0x7f1d68150730 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:52.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.699+0000 7f1d6d84c700 1 --2- 172.21.15.203:0/2140554296 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d680c44f0 0x7f1d680c1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:16:52.713 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.699+0000 7f1d6d84c700 1 --2- 172.21.15.203:0/2140554296 >> v2:172.21.15.91:3300/0 conn(0x7f1d680c48d0 0x7f1d680c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:16:52.713 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.699+0000 7f1d6d84c700 1 -- 172.21.15.203:0/2140554296 >> 172.21.15.203:0/2140554296 conn(0x7f1d68056cd0 msgr2=0x7f1d68061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:52.713 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.699+0000 7f1d6d84c700 1 -- 172.21.15.203:0/2140554296 shutdown_connections 2021-01-06T22:16:52.713 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.699+0000 7f1d6d84c700 1 -- 172.21.15.203:0/2140554296 wait complete. 2021-01-06T22:16:52.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.699+0000 7f1d6d84c700 1 Processor -- start 2021-01-06T22:16:52.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.699+0000 7f1d6d84c700 1 -- start start 2021-01-06T22:16:52.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d6d84c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d68060fc0 0x7f1d6814de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:52.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d6d84c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d680c44f0 0x7f1d680be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:52.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d6d84c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d680c48d0 0x7f1d680baf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:52.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d6d84c700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680b84e0 con 0x7f1d680c48d0 2021-01-06T22:16:52.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d6d84c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6814a8a0 con 0x7f1d680c44f0 2021-01-06T22:16:52.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d6d84c700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6814ab40 con 0x7f1d68060fc0 2021-01-06T22:16:52.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d67fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d680c44f0 0x7f1d680be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:52.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d67fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d680c44f0 0x7f1d680be3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52282/0 (socket says 172.21.15.203:52282) 2021-01-06T22:16:52.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d67fff700 1 -- 172.21.15.203:0/2086088370 learned_addr learned my addr 172.21.15.203:0/2086088370 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:52.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d6d04b700 1 --2- 172.21.15.203:0/2086088370 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d680c48d0 0x7f1d680baf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:52.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d6c84a700 1 --2- 172.21.15.203:0/2086088370 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d68060fc0 0x7f1d6814de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:52.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d67fff700 1 -- 172.21.15.203:0/2086088370 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d68060fc0 msgr2=0x7f1d6814de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:52.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d67fff700 1 --2- 172.21.15.203:0/2086088370 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d68060fc0 0x7f1d6814de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:52.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d67fff700 1 -- 172.21.15.203:0/2086088370 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d680c48d0 msgr2=0x7f1d680baf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:52.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d67fff700 1 --2- 172.21.15.203:0/2086088370 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d680c48d0 0x7f1d680baf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:52.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d67fff700 1 -- 172.21.15.203:0/2086088370 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6814ade0 con 0x7f1d680c44f0 2021-01-06T22:16:52.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d67fff700 1 --2- 172.21.15.203:0/2086088370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d680c44f0 0x7f1d680be3e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f1d5c0096b0 tx=0x7f1d5c006d30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:52.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d64ff9700 1 -- 172.21.15.203:0/2086088370 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c00d490 con 0x7f1d680c44f0 2021-01-06T22:16:52.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d64ff9700 1 -- 172.21.15.203:0/2086088370 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5c000e90 con 0x7f1d680c44f0 2021-01-06T22:16:52.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d6d84c700 1 -- 172.21.15.203:0/2086088370 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d68160d30 con 0x7f1d680c44f0 2021-01-06T22:16:52.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d64ff9700 1 -- 172.21.15.203:0/2086088370 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c00d840 con 0x7f1d680c44f0 2021-01-06T22:16:52.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.703+0000 7f1d6d84c700 1 -- 172.21.15.203:0/2086088370 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d680bf970 con 0x7f1d680c44f0 2021-01-06T22:16:52.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.707+0000 7f1d6d84c700 1 -- 172.21.15.203:0/2086088370 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d680450a0 con 0x7f1d680c44f0 2021-01-06T22:16:52.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.707+0000 7f1d64ff9700 1 -- 172.21.15.203:0/2086088370 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f1d5c00f030 con 0x7f1d680c44f0 2021-01-06T22:16:52.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.707+0000 7f1d64ff9700 1 --2- 172.21.15.203:0/2086088370 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d480563d0 0x7f1d48058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:52.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.707+0000 7f1d64ff9700 1 -- 172.21.15.203:0/2086088370 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f1d5c06f330 con 0x7f1d680c44f0 2021-01-06T22:16:52.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.707+0000 7f1d6c84a700 1 --2- 172.21.15.203:0/2086088370 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d480563d0 0x7f1d48058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:52.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.707+0000 7f1d6c84a700 1 --2- 172.21.15.203:0/2086088370 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d480563d0 0x7f1d48058890 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f1d58009c20 tx=0x7f1d580091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:52.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.715+0000 7f1d64ff9700 1 -- 172.21.15.203:0/2086088370 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f1d5c0448a0 con 0x7f1d680c44f0 2021-01-06T22:16:52.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.915+0000 7f1d6d84c700 1 -- 172.21.15.203:0/2086088370 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d68057980 con 0x7f1d680c44f0 2021-01-06T22:16:52.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.915+0000 7f1d64ff9700 1 -- 172.21.15.203:0/2086088370 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+3208 (secure 0 0 0) 0x7f1d5c047ec0 con 0x7f1d680c44f0 2021-01-06T22:16:52.917 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:52.918 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:52.612287+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:resolve","state_seq":8,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:52.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.915+0000 7f1d6d84c700 1 -- 172.21.15.203:0/2086088370 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d480563d0 msgr2=0x7f1d48058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:52.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.915+0000 7f1d6d84c700 1 --2- 172.21.15.203:0/2086088370 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d480563d0 0x7f1d48058890 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f1d58009c20 tx=0x7f1d580091d0).stop 2021-01-06T22:16:52.919 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.915+0000 7f1d6d84c700 1 -- 172.21.15.203:0/2086088370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d680c44f0 msgr2=0x7f1d680be3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:52.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.915+0000 7f1d6d84c700 1 --2- 172.21.15.203:0/2086088370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d680c44f0 0x7f1d680be3e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f1d5c0096b0 tx=0x7f1d5c006d30).stop 2021-01-06T22:16:52.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.915+0000 7f1d6d84c700 1 -- 172.21.15.203:0/2086088370 shutdown_connections 2021-01-06T22:16:52.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.915+0000 7f1d6d84c700 1 --2- 172.21.15.203:0/2086088370 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d480563d0 0x7f1d48058890 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:52.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.915+0000 7f1d6d84c700 1 --2- 172.21.15.203:0/2086088370 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d680c48d0 0x7f1d680baf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:52.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.915+0000 7f1d6d84c700 1 --2- 172.21.15.203:0/2086088370 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d68060fc0 0x7f1d6814de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:52.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.915+0000 7f1d6d84c700 1 --2- 172.21.15.203:0/2086088370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d680c44f0 0x7f1d680be3e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:52.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.915+0000 7f1d6d84c700 1 -- 172.21.15.203:0/2086088370 >> 172.21.15.203:0/2086088370 conn(0x7f1d68056cd0 msgr2=0x7f1d680c0470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:52.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.915+0000 7f1d6d84c700 1 -- 172.21.15.203:0/2086088370 shutdown_connections 2021-01-06T22:16:52.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:52.919+0000 7f1d6d84c700 1 -- 172.21.15.203:0/2086088370 wait complete. 2021-01-06T22:16:52.923 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 64 2021-01-06T22:16:52.933 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2021-01-06T22:16:52.934 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2021-01-06T22:16:52.934 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:53.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc85a657700 1 Processor -- start 2021-01-06T22:16:53.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc85a657700 1 -- start start 2021-01-06T22:16:53.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc85a657700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc854064380 0x7fc8540c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:53.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc85a657700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8541478c0 0x7fc8540bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:53.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc85a657700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8540bb2d0 con 0x7fc85414e310 2021-01-06T22:16:53.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc85a657700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85414a560 con 0x7fc854064380 2021-01-06T22:16:53.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc85a657700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85414a800 con 0x7fc8541478c0 2021-01-06T22:16:53.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc859655700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc854064380 0x7fc8540c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:53.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc859e56700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8541478c0 0x7fc8540bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:53.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc859655700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc854064380 0x7fc8540c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:60136/0 (socket says 172.21.15.203:60136) 2021-01-06T22:16:53.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc859655700 1 -- 172.21.15.203:0/1722619626 learned_addr learned my addr 172.21.15.203:0/1722619626 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:53.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc859e56700 1 -- 172.21.15.203:0/1722619626 >> v2:172.21.15.91:3300/0 conn(0x7fc854064380 msgr2=0x7fc8540c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:53.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc859e56700 1 --2- 172.21.15.203:0/1722619626 >> v2:172.21.15.91:3300/0 conn(0x7fc854064380 0x7fc8540c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:53.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc859e56700 1 -- 172.21.15.203:0/1722619626 >> v1:172.21.15.91:6789/0 conn(0x7fc85414e310 legacy=0x7fc8541471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:53.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc859e56700 1 -- 172.21.15.203:0/1722619626 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85414aaa0 con 0x7fc8541478c0 2021-01-06T22:16:53.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc859655700 1 --2- 172.21.15.203:0/1722619626 >> v2:172.21.15.91:3300/0 conn(0x7fc854064380 0x7fc8540c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:16:53.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc859e56700 1 --2- 172.21.15.203:0/1722619626 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8541478c0 0x7fc8540bb730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fc85000b980 tx=0x7fc850006de0).ready entity=mon.2 client_cookie=b26e4ad5042f64c9 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:53.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.051+0000 7fc84bfff700 1 -- 172.21.15.203:0/1722619626 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850009840 con 0x7fc8541478c0 2021-01-06T22:16:53.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc84bfff700 1 -- 172.21.15.203:0/1722619626 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8500099a0 con 0x7fc8541478c0 2021-01-06T22:16:53.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc84bfff700 1 -- 172.21.15.203:0/1722619626 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850009d20 con 0x7fc8541478c0 2021-01-06T22:16:53.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc85a657700 1 -- 172.21.15.203:0/1722619626 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8541478c0 msgr2=0x7fc8540bb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:53.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc85a657700 1 --2- 172.21.15.203:0/1722619626 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8541478c0 0x7fc8540bb730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fc85000b980 tx=0x7fc850006de0).stop 2021-01-06T22:16:53.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc85a657700 1 -- 172.21.15.203:0/1722619626 shutdown_connections 2021-01-06T22:16:53.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc85a657700 1 --2- 172.21.15.203:0/1722619626 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8541478c0 0x7fc8540bb730 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:53.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc85a657700 1 --2- 172.21.15.203:0/1722619626 >> v2:172.21.15.91:3300/0 conn(0x7fc854064380 0x7fc8540c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:53.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc85a657700 1 -- 172.21.15.203:0/1722619626 >> 172.21.15.203:0/1722619626 conn(0x7fc8540667b0 msgr2=0x7fc8540c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:53.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc85a657700 1 -- 172.21.15.203:0/1722619626 shutdown_connections 2021-01-06T22:16:53.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc85a657700 1 -- 172.21.15.203:0/1722619626 wait complete. 2021-01-06T22:16:53.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc85a657700 1 Processor -- start 2021-01-06T22:16:53.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc85a657700 1 -- start start 2021-01-06T22:16:53.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc85a657700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc854064380 0x7fc854151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:53.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc85a657700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8541478c0 0x7fc8540bb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:53.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc85a657700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc85414e310 0x7fc85415e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:53.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc85a657700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85414ead0 con 0x7fc8541478c0 2021-01-06T22:16:53.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc85a657700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8541526c0 con 0x7fc85414e310 2021-01-06T22:16:53.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.055+0000 7fc85a657700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85414ed20 con 0x7fc854064380 2021-01-06T22:16:53.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc859e56700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc85414e310 0x7fc85415e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:53.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc859e56700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc85414e310 0x7fc85415e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52298/0 (socket says 172.21.15.203:52298) 2021-01-06T22:16:53.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc859e56700 1 -- 172.21.15.203:0/3747769201 learned_addr learned my addr 172.21.15.203:0/3747769201 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:53.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc859655700 1 --2- 172.21.15.203:0/3747769201 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc854064380 0x7fc854151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:53.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc858e54700 1 --2- 172.21.15.203:0/3747769201 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8541478c0 0x7fc8540bb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:53.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc859e56700 1 -- 172.21.15.203:0/3747769201 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc854064380 msgr2=0x7fc854151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:53.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc859e56700 1 --2- 172.21.15.203:0/3747769201 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc854064380 0x7fc854151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:53.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc859e56700 1 -- 172.21.15.203:0/3747769201 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8541478c0 msgr2=0x7fc8540bb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:53.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc859e56700 1 --2- 172.21.15.203:0/3747769201 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8541478c0 0x7fc8540bb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:53.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc859e56700 1 -- 172.21.15.203:0/3747769201 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85414efc0 con 0x7fc85414e310 2021-01-06T22:16:53.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc859e56700 1 --2- 172.21.15.203:0/3747769201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc85414e310 0x7fc85415e8b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fc850006f80 tx=0x7fc850016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:53.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc8497fa700 1 -- 172.21.15.203:0/3747769201 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850009340 con 0x7fc85414e310 2021-01-06T22:16:53.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc8497fa700 1 -- 172.21.15.203:0/3747769201 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85001d430 con 0x7fc85414e310 2021-01-06T22:16:53.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc85a657700 1 -- 172.21.15.203:0/3747769201 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85414f260 con 0x7fc85414e310 2021-01-06T22:16:53.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc8497fa700 1 -- 172.21.15.203:0/3747769201 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85001d7b0 con 0x7fc85414e310 2021-01-06T22:16:53.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc85a657700 1 -- 172.21.15.203:0/3747769201 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8540c03a0 con 0x7fc85414e310 2021-01-06T22:16:53.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc85a657700 1 -- 172.21.15.203:0/3747769201 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85414af20 con 0x7fc85414e310 2021-01-06T22:16:53.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc8497fa700 1 -- 172.21.15.203:0/3747769201 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fc85000f070 con 0x7fc85414e310 2021-01-06T22:16:53.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc8497fa700 1 --2- 172.21.15.203:0/3747769201 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc8340563d0 0x7fc834058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:53.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.059+0000 7fc8497fa700 1 -- 172.21.15.203:0/3747769201 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6761+0+0 (secure 0 0 0) 0x7fc8500710b0 con 0x7fc85414e310 2021-01-06T22:16:53.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.063+0000 7fc859655700 1 --2- 172.21.15.203:0/3747769201 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc8340563d0 0x7fc834058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:53.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.063+0000 7fc859655700 1 --2- 172.21.15.203:0/3747769201 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc8340563d0 0x7fc834058890 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fc84c006d60 tx=0x7fc84c009290).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:53.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.067+0000 7fc8497fa700 1 -- 172.21.15.203:0/3747769201 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fc850046680 con 0x7fc85414e310 2021-01-06T22:16:53.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.251+0000 7fc85a657700 1 -- 172.21.15.203:0/3747769201 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8540450a0 con 0x7fc85414e310 2021-01-06T22:16:53.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.255+0000 7fc8497fa700 1 -- 172.21.15.203:0/3747769201 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+3208 (secure 0 0 0) 0x7fc850049ca0 con 0x7fc85414e310 2021-01-06T22:16:53.256 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:53.256 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:52.612287+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:resolve","state_seq":8,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:53.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.255+0000 7fc85a657700 1 -- 172.21.15.203:0/3747769201 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc8340563d0 msgr2=0x7fc834058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:53.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.255+0000 7fc85a657700 1 --2- 172.21.15.203:0/3747769201 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc8340563d0 0x7fc834058890 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fc84c006d60 tx=0x7fc84c009290).stop 2021-01-06T22:16:53.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.255+0000 7fc85a657700 1 -- 172.21.15.203:0/3747769201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc85414e310 msgr2=0x7fc85415e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:53.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.255+0000 7fc85a657700 1 --2- 172.21.15.203:0/3747769201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc85414e310 0x7fc85415e8b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fc850006f80 tx=0x7fc850016000).stop 2021-01-06T22:16:53.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.255+0000 7fc85a657700 1 -- 172.21.15.203:0/3747769201 shutdown_connections 2021-01-06T22:16:53.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.255+0000 7fc85a657700 1 --2- 172.21.15.203:0/3747769201 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc8340563d0 0x7fc834058890 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:53.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.255+0000 7fc85a657700 1 --2- 172.21.15.203:0/3747769201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc85414e310 0x7fc85415e8b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:53.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.255+0000 7fc85a657700 1 --2- 172.21.15.203:0/3747769201 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc854064380 0x7fc854151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:53.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.255+0000 7fc85a657700 1 --2- 172.21.15.203:0/3747769201 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8541478c0 0x7fc8540bb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:53.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.255+0000 7fc85a657700 1 -- 172.21.15.203:0/3747769201 >> 172.21.15.203:0/3747769201 conn(0x7fc8540667b0 msgr2=0x7fc854061390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:53.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.255+0000 7fc85a657700 1 -- 172.21.15.203:0/3747769201 shutdown_connections 2021-01-06T22:16:53.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:53.255+0000 7fc85a657700 1 -- 172.21.15.203:0/3747769201 wait complete. 2021-01-06T22:16:53.260 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 64 2021-01-06T22:16:53.271 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 64, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 64, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5364": { "addr": "172.21.15.91:6837/617463325", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6835", "nonce": 617463325, "type": "v2" }, { "addr": "172.21.15.91:6837", "nonce": 617463325, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5364, "incarnation": 57, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 32 }, "gid_5539": { "addr": "172.21.15.203:6835/1801698156", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6834", "nonce": 1801698156, "type": "v2" }, { "addr": "172.21.15.203:6835", "nonce": 1801698156, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5539, "incarnation": 63, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:resolve", "state_seq": 8 } }, "last_failure": 0, "last_failure_osd_epoch": 31, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2021-01-06T22:16:52.612287+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 5539, "mds_1": 5364 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.91:6836/2428882934", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6834", "nonce": 2428882934, "type": "v2" }, { "addr": "172.21.15.91:6836", "nonce": 2428882934, "type": "v1" } ] }, "epoch": 26, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4724, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2021-01-06T22:16:55.272 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:55.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.402+0000 7f16affff700 1 Processor -- start 2021-01-06T22:16:55.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.402+0000 7f16affff700 1 -- start start 2021-01-06T22:16:55.408 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16affff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f16b00bcbd0 0x7f16b00baf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:55.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16affff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16b00bcfb0 0x7f16b00bb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:55.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16affff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b00a4610 con 0x7f16b00bc450 2021-01-06T22:16:55.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16affff700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b0064730 con 0x7f16b00bcbd0 2021-01-06T22:16:55.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16affff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b00649d0 con 0x7f16b00bcfb0 2021-01-06T22:16:55.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16aeffd700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f16b00bcbd0 0x7f16b00baf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:55.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16ae7fc700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16b00bcfb0 0x7f16b00bb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:55.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16af7fe700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f16b00bc450 0x7f16b014e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:45552/0 (socket says 172.21.15.203:45552) 2021-01-06T22:16:55.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16af7fe700 1 -- 172.21.15.203:0/3050211638 learned_addr learned my addr 172.21.15.203:0/3050211638 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:16:55.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16adffb700 1 -- 172.21.15.203:0/3050211638 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 589967396 0 0) 0x7f16b00a4610 con 0x7f16b00bc450 2021-01-06T22:16:55.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16adffb700 1 -- 172.21.15.203:0/3050211638 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1690003230 con 0x7f16b00bc450 2021-01-06T22:16:55.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16ae7fc700 1 -- 172.21.15.203:0/3050211638 >> v2:172.21.15.91:3300/0 conn(0x7f16b00bcbd0 msgr2=0x7f16b00baf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:55.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16ae7fc700 1 --2- 172.21.15.203:0/3050211638 >> v2:172.21.15.91:3300/0 conn(0x7f16b00bcbd0 0x7f16b00baf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:55.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16ae7fc700 1 -- 172.21.15.203:0/3050211638 >> v1:172.21.15.91:6789/0 conn(0x7f16b00bc450 legacy=0x7f16b014e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:55.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16ae7fc700 1 -- 172.21.15.203:0/3050211638 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b00a3e30 con 0x7f16b00bcfb0 2021-01-06T22:16:55.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16aeffd700 1 --2- 172.21.15.203:0/3050211638 >> v2:172.21.15.91:3300/0 conn(0x7f16b00bcbd0 0x7f16b00baf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:16:55.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16ae7fc700 1 --2- 172.21.15.203:0/3050211638 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16b00bcfb0 0x7f16b00bb470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f1698009c90 tx=0x7f16980091d0).ready entity=mon.2 client_cookie=3d5f905f24815c5c server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:55.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16adffb700 1 -- 172.21.15.203:0/3050211638 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800b9c0 con 0x7f16b00bcfb0 2021-01-06T22:16:55.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16adffb700 1 -- 172.21.15.203:0/3050211638 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169800bb20 con 0x7f16b00bcfb0 2021-01-06T22:16:55.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16adffb700 1 -- 172.21.15.203:0/3050211638 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800bea0 con 0x7f16b00bcfb0 2021-01-06T22:16:55.417 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16affff700 1 -- 172.21.15.203:0/3050211638 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16b00bcfb0 msgr2=0x7f16b00bb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:55.417 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16affff700 1 --2- 172.21.15.203:0/3050211638 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16b00bcfb0 0x7f16b00bb470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f1698009c90 tx=0x7f16980091d0).stop 2021-01-06T22:16:55.417 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16affff700 1 -- 172.21.15.203:0/3050211638 shutdown_connections 2021-01-06T22:16:55.417 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16affff700 1 --2- 172.21.15.203:0/3050211638 >> v2:172.21.15.91:3300/0 conn(0x7f16b00bcbd0 0x7f16b00baf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:55.417 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16affff700 1 --2- 172.21.15.203:0/3050211638 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16b00bcfb0 0x7f16b00bb470 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:55.418 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16affff700 1 -- 172.21.15.203:0/3050211638 >> 172.21.15.203:0/3050211638 conn(0x7f16b0061530 msgr2=0x7f16b0063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:55.418 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16affff700 1 -- 172.21.15.203:0/3050211638 shutdown_connections 2021-01-06T22:16:55.418 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.406+0000 7f16affff700 1 -- 172.21.15.203:0/3050211638 wait complete. 2021-01-06T22:16:55.418 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16affff700 1 Processor -- start 2021-01-06T22:16:55.419 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16affff700 1 -- start start 2021-01-06T22:16:55.419 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16affff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16b00bc450 0x7f16b00b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:55.419 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16affff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16b00bcbd0 0x7f16b014daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:55.419 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16affff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16b00bcfb0 0x7f16b00be3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:55.419 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16affff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b00b9d40 con 0x7f16b00bcbd0 2021-01-06T22:16:55.420 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16affff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b00b9fe0 con 0x7f16b00bcfb0 2021-01-06T22:16:55.420 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16affff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b014d1a0 con 0x7f16b00bc450 2021-01-06T22:16:55.420 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16af7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16b00bcfb0 0x7f16b00be3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:55.420 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16af7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16b00bcfb0 0x7f16b00be3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52314/0 (socket says 172.21.15.203:52314) 2021-01-06T22:16:55.421 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16af7fe700 1 -- 172.21.15.203:0/2513279356 learned_addr learned my addr 172.21.15.203:0/2513279356 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:55.421 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16ae7fc700 1 --2- 172.21.15.203:0/2513279356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16b00bcbd0 0x7f16b014daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:55.421 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16aeffd700 1 --2- 172.21.15.203:0/2513279356 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16b00bc450 0x7f16b00b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:55.421 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16af7fe700 1 -- 172.21.15.203:0/2513279356 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16b00bc450 msgr2=0x7f16b00b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:55.421 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16af7fe700 1 --2- 172.21.15.203:0/2513279356 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16b00bc450 0x7f16b00b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:55.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16af7fe700 1 -- 172.21.15.203:0/2513279356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16b00bcbd0 msgr2=0x7f16b014daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:55.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16af7fe700 1 --2- 172.21.15.203:0/2513279356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16b00bcbd0 0x7f16b014daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:55.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16af7fe700 1 -- 172.21.15.203:0/2513279356 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b014d440 con 0x7f16b00bcfb0 2021-01-06T22:16:55.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16aeffd700 1 --2- 172.21.15.203:0/2513279356 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16b00bc450 0x7f16b00b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:16:55.423 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16af7fe700 1 --2- 172.21.15.203:0/2513279356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16b00bcfb0 0x7f16b00be3c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f16a80010a0 tx=0x7f16a8007a90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:55.423 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16a6ffd700 1 -- 172.21.15.203:0/2513279356 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a80072e0 con 0x7f16b00bcfb0 2021-01-06T22:16:55.423 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16affff700 1 -- 172.21.15.203:0/2513279356 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b014d6e0 con 0x7f16b00bcfb0 2021-01-06T22:16:55.423 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16a6ffd700 1 -- 172.21.15.203:0/2513279356 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a8007440 con 0x7f16b00bcfb0 2021-01-06T22:16:55.424 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16a6ffd700 1 -- 172.21.15.203:0/2513279356 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a80057f0 con 0x7f16b00bcfb0 2021-01-06T22:16:55.424 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.410+0000 7f16affff700 1 -- 172.21.15.203:0/2513279356 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b00ba5b0 con 0x7f16b00bcfb0 2021-01-06T22:16:55.425 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.414+0000 7f16affff700 1 -- 172.21.15.203:0/2513279356 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b00450a0 con 0x7f16b00bcfb0 2021-01-06T22:16:55.425 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.414+0000 7f16a6ffd700 1 -- 172.21.15.203:0/2513279356 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f16a8010070 con 0x7f16b00bcfb0 2021-01-06T22:16:55.425 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.414+0000 7f16a6ffd700 1 --2- 172.21.15.203:0/2513279356 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1694056400 0x7f16940588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:55.425 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.414+0000 7f16a6ffd700 1 -- 172.21.15.203:0/2513279356 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f16a806eef0 con 0x7f16b00bcfb0 2021-01-06T22:16:55.426 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.414+0000 7f16aeffd700 1 --2- 172.21.15.203:0/2513279356 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1694056400 0x7f16940588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:55.426 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.414+0000 7f16aeffd700 1 --2- 172.21.15.203:0/2513279356 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1694056400 0x7f16940588c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f16b00c04f0 tx=0x7f16a0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:55.426 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.422+0000 7f16a6ffd700 1 -- 172.21.15.203:0/2513279356 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f16a80444c0 con 0x7f16b00bcfb0 2021-01-06T22:16:55.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.626+0000 7f16affff700 1 -- 172.21.15.203:0/2513279356 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16b014f2d0 con 0x7f16b00bcfb0 2021-01-06T22:16:55.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.630+0000 7f16a6ffd700 1 -- 172.21.15.203:0/2513279356 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+3208 (secure 0 0 0) 0x7f16a8007bd0 con 0x7f16b00bcfb0 2021-01-06T22:16:55.632 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:55.633 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:54.626935+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:55.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.630+0000 7f16affff700 1 -- 172.21.15.203:0/2513279356 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1694056400 msgr2=0x7f16940588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:55.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.630+0000 7f16affff700 1 --2- 172.21.15.203:0/2513279356 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1694056400 0x7f16940588c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f16b00c04f0 tx=0x7f16a0008040).stop 2021-01-06T22:16:55.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.630+0000 7f16affff700 1 -- 172.21.15.203:0/2513279356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16b00bcfb0 msgr2=0x7f16b00be3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:55.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.630+0000 7f16affff700 1 --2- 172.21.15.203:0/2513279356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16b00bcfb0 0x7f16b00be3c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f16a80010a0 tx=0x7f16a8007a90).stop 2021-01-06T22:16:55.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.630+0000 7f16affff700 1 -- 172.21.15.203:0/2513279356 shutdown_connections 2021-01-06T22:16:55.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.630+0000 7f16affff700 1 --2- 172.21.15.203:0/2513279356 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1694056400 0x7f16940588c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:55.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.630+0000 7f16affff700 1 --2- 172.21.15.203:0/2513279356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16b00bcfb0 0x7f16b00be3c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:55.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.630+0000 7f16affff700 1 --2- 172.21.15.203:0/2513279356 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16b00bc450 0x7f16b00b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:55.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.630+0000 7f16affff700 1 --2- 172.21.15.203:0/2513279356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16b00bcbd0 0x7f16b014daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:55.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.630+0000 7f16affff700 1 -- 172.21.15.203:0/2513279356 >> 172.21.15.203:0/2513279356 conn(0x7f16b0061530 msgr2=0x7f16b00602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:55.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.630+0000 7f16affff700 1 -- 172.21.15.203:0/2513279356 shutdown_connections 2021-01-06T22:16:55.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:55.630+0000 7f16affff700 1 -- 172.21.15.203:0/2513279356 wait complete. 2021-01-06T22:16:55.638 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 66 2021-01-06T22:16:55.655 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:16:57.655 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:16:57.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d2613700 1 Processor -- start 2021-01-06T22:16:57.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d2613700 1 -- start start 2021-01-06T22:16:57.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d2613700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa5cc1513f0 0x7fa5cc0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:57.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d2613700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc14ad90 0x7fa5cc14b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:57.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d2613700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc0bacb0 con 0x7fa5cc0bccb0 2021-01-06T22:16:57.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d2613700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5cc0bd160 con 0x7fa5cc1513f0 2021-01-06T22:16:57.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d2613700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc14b6d0 con 0x7fa5cc14ad90 2021-01-06T22:16:57.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d1611700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa5cc1513f0 0x7fa5cc0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:57.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d1611700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa5cc1513f0 0x7fa5cc0c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52330/0 (socket says 172.21.15.203:52330) 2021-01-06T22:16:57.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d1611700 1 -- 172.21.15.203:0/2149740486 learned_addr learned my addr 172.21.15.203:0/2149740486 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:57.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d0e10700 1 --1- 172.21.15.203:0/2149740486 >> v1:172.21.15.203:6789/0 conn(0x7fa5cc0bccb0 0x7fa5cc1517d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50926/0 (socket says 172.21.15.203:50926) 2021-01-06T22:16:57.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d1e12700 1 --2- 172.21.15.203:0/2149740486 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc14ad90 0x7fa5cc14b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:57.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d1611700 1 -- 172.21.15.203:0/2149740486 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc14ad90 msgr2=0x7fa5cc14b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:16:57.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d1611700 1 --2- 172.21.15.203:0/2149740486 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc14ad90 0x7fa5cc14b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:57.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d1611700 1 -- 172.21.15.203:0/2149740486 >> v1:172.21.15.203:6789/0 conn(0x7fa5cc0bccb0 legacy=0x7fa5cc1517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:57.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d1611700 1 -- 172.21.15.203:0/2149740486 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc152000 con 0x7fa5cc1513f0 2021-01-06T22:16:57.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5d1611700 1 --2- 172.21.15.203:0/2149740486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5cc1513f0 0x7fa5cc0c0fa0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fa5c800ba10 tx=0x7fa5c8006ea0).ready entity=mon.1 client_cookie=e1f8c6089f4351e2 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:57.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5bbfff700 1 -- 172.21.15.203:0/2149740486 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8009890 con 0x7fa5cc1513f0 2021-01-06T22:16:57.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5bbfff700 1 -- 172.21.15.203:0/2149740486 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c80099f0 con 0x7fa5cc1513f0 2021-01-06T22:16:57.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.786+0000 7fa5bbfff700 1 -- 172.21.15.203:0/2149740486 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8009d70 con 0x7fa5cc1513f0 2021-01-06T22:16:57.797 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d2613700 1 -- 172.21.15.203:0/2149740486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5cc1513f0 msgr2=0x7fa5cc0c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:57.797 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d2613700 1 --2- 172.21.15.203:0/2149740486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5cc1513f0 0x7fa5cc0c0fa0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fa5c800ba10 tx=0x7fa5c8006ea0).stop 2021-01-06T22:16:57.797 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d2613700 1 -- 172.21.15.203:0/2149740486 shutdown_connections 2021-01-06T22:16:57.798 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d2613700 1 --2- 172.21.15.203:0/2149740486 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc14ad90 0x7fa5cc14b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:57.798 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d2613700 1 --2- 172.21.15.203:0/2149740486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5cc1513f0 0x7fa5cc0c0fa0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:57.798 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d2613700 1 -- 172.21.15.203:0/2149740486 >> 172.21.15.203:0/2149740486 conn(0x7fa5cc0667b0 msgr2=0x7fa5cc0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:57.798 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d2613700 1 -- 172.21.15.203:0/2149740486 shutdown_connections 2021-01-06T22:16:57.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d2613700 1 -- 172.21.15.203:0/2149740486 wait complete. 2021-01-06T22:16:57.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d2613700 1 Processor -- start 2021-01-06T22:16:57.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d2613700 1 -- start start 2021-01-06T22:16:57.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d2613700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc0bccb0 0x7fa5cc1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:57.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d2613700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc14ad90 0x7fa5cc15dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:57.800 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d2613700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5cc1513f0 0x7fa5cc15e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:57.800 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d2613700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc15e6f0 con 0x7fa5cc14ad90 2021-01-06T22:16:57.800 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d2613700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc1618e0 con 0x7fa5cc1513f0 2021-01-06T22:16:57.800 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d2613700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc161b80 con 0x7fa5cc0bccb0 2021-01-06T22:16:57.801 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d1e12700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5cc1513f0 0x7fa5cc15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:57.801 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d1e12700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5cc1513f0 0x7fa5cc15e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52340/0 (socket says 172.21.15.203:52340) 2021-01-06T22:16:57.801 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d1e12700 1 -- 172.21.15.203:0/3613593542 learned_addr learned my addr 172.21.15.203:0/3613593542 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:16:57.801 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.790+0000 7fa5d1611700 1 --2- 172.21.15.203:0/3613593542 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc0bccb0 0x7fa5cc1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:57.801 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5d0e10700 1 --2- 172.21.15.203:0/3613593542 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc14ad90 0x7fa5cc15dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:57.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5d1e12700 1 -- 172.21.15.203:0/3613593542 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc0bccb0 msgr2=0x7fa5cc1524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:57.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5d1e12700 1 --2- 172.21.15.203:0/3613593542 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc0bccb0 0x7fa5cc1524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:57.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5d1e12700 1 -- 172.21.15.203:0/3613593542 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc14ad90 msgr2=0x7fa5cc15dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:57.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5d1e12700 1 --2- 172.21.15.203:0/3613593542 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc14ad90 0x7fa5cc15dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:57.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5d1e12700 1 -- 172.21.15.203:0/3613593542 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc1406b0 con 0x7fa5cc1513f0 2021-01-06T22:16:57.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5d1e12700 1 --2- 172.21.15.203:0/3613593542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5cc1513f0 0x7fa5cc15e200 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7fa5c400ebf0 tx=0x7fa5c4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:57.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5b97fa700 1 -- 172.21.15.203:0/3613593542 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c400c6e0 con 0x7fa5cc1513f0 2021-01-06T22:16:57.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5b97fa700 1 -- 172.21.15.203:0/3613593542 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c400c840 con 0x7fa5cc1513f0 2021-01-06T22:16:57.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5b97fa700 1 -- 172.21.15.203:0/3613593542 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c400cbe0 con 0x7fa5cc1513f0 2021-01-06T22:16:57.804 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5d2613700 1 -- 172.21.15.203:0/3613593542 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5cc140950 con 0x7fa5cc1513f0 2021-01-06T22:16:57.804 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5d2613700 1 -- 172.21.15.203:0/3613593542 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5cc153f70 con 0x7fa5cc1513f0 2021-01-06T22:16:57.805 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5d2613700 1 -- 172.21.15.203:0/3613593542 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5cc0450a0 con 0x7fa5cc1513f0 2021-01-06T22:16:57.805 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5b97fa700 1 -- 172.21.15.203:0/3613593542 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa5c401d460 con 0x7fa5cc1513f0 2021-01-06T22:16:57.805 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5b97fa700 1 --2- 172.21.15.203:0/3613593542 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa5ac0563d0 0x7fa5ac058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:16:57.805 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5b97fa700 1 -- 172.21.15.203:0/3613593542 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6761+0+0 (secure 0 0 0) 0x7fa5c406f9c0 con 0x7fa5cc1513f0 2021-01-06T22:16:57.806 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.794+0000 7fa5d1611700 1 --2- 172.21.15.203:0/3613593542 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa5ac0563d0 0x7fa5ac058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:16:57.806 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.798+0000 7fa5d1611700 1 --2- 172.21.15.203:0/3613593542 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa5ac0563d0 0x7fa5ac058890 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fa5c8006ee0 tx=0x7fa5c8016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:16:57.807 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:57.802+0000 7fa5b97fa700 1 -- 172.21.15.203:0/3613593542 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa5c4048e60 con 0x7fa5cc1513f0 2021-01-06T22:16:58.029 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:58.026+0000 7fa5d2613700 1 -- 172.21.15.203:0/3613593542 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5cc05ca60 con 0x7fa5cc1513f0 2021-01-06T22:16:58.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:58.026+0000 7fa5b97fa700 1 -- 172.21.15.203:0/3613593542 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+3208 (secure 0 0 0) 0x7fa5c4044f30 con 0x7fa5cc1513f0 2021-01-06T22:16:58.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:58.030+0000 7fa5d2613700 1 -- 172.21.15.203:0/3613593542 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa5ac0563d0 msgr2=0x7fa5ac058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:58.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:58.030+0000 7fa5d2613700 1 --2- 172.21.15.203:0/3613593542 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa5ac0563d0 0x7fa5ac058890 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fa5c8006ee0 tx=0x7fa5c8016000).stop 2021-01-06T22:16:58.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:58.030+0000 7fa5d2613700 1 -- 172.21.15.203:0/3613593542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5cc1513f0 msgr2=0x7fa5cc15e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:16:58.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:58.030+0000 7fa5d2613700 1 --2- 172.21.15.203:0/3613593542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5cc1513f0 0x7fa5cc15e200 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7fa5c400ebf0 tx=0x7fa5c4008040).stop 2021-01-06T22:16:58.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:58.030+0000 7fa5d2613700 1 -- 172.21.15.203:0/3613593542 shutdown_connections 2021-01-06T22:16:58.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:58.030+0000 7fa5d2613700 1 --2- 172.21.15.203:0/3613593542 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa5ac0563d0 0x7fa5ac058890 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:58.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:58.030+0000 7fa5d2613700 1 --2- 172.21.15.203:0/3613593542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5cc1513f0 0x7fa5cc15e200 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:58.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:58.030+0000 7fa5d2613700 1 --2- 172.21.15.203:0/3613593542 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc0bccb0 0x7fa5cc1524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:58.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:58.030+0000 7fa5d2613700 1 --2- 172.21.15.203:0/3613593542 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc14ad90 0x7fa5cc15dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:16:58.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:58.030+0000 7fa5d2613700 1 -- 172.21.15.203:0/3613593542 >> 172.21.15.203:0/3613593542 conn(0x7fa5cc0667b0 msgr2=0x7fa5cc0bfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:16:58.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:58.030+0000 7fa5d2613700 1 -- 172.21.15.203:0/3613593542 shutdown_connections 2021-01-06T22:16:58.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:16:58.030+0000 7fa5d2613700 1 -- 172.21.15.203:0/3613593542 wait complete. 2021-01-06T22:16:58.399 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 67 2021-01-06T22:16:58.400 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:16:58.400 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:55.638183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:16:58.400 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=0, replacing gid=5350 2021-01-06T22:16:58.401 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 3 secs before reviving mds.a 2021-01-06T22:16:58.601 INFO:tasks.ceph.mds.c.smithi203.stderr:2021-01-06T22:16:58.598+0000 7fd881215700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2021-01-06T22:16:58.601 INFO:tasks.ceph.mds.c.smithi203.stderr:2021-01-06T22:16:58.598+0000 7fd881215700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2021-01-06T22:17:01.401 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2021-01-06T22:17:01.402 INFO:tasks.ceph.mds.a:Restarting daemon 2021-01-06T22:17:01.402 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-01-06T22:17:01.405 INFO:tasks.ceph.mds.a:Started 2021-01-06T22:17:01.405 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:17:01.476 INFO:tasks.ceph.mds.a.smithi203.stdout:starting mds.a at 2021-01-06T22:17:01.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.530+0000 7f7424f86700 1 Processor -- start 2021-01-06T22:17:01.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.530+0000 7f7424f86700 1 -- start start 2021-01-06T22:17:01.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.530+0000 7f7424f86700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74201513f0 0x7f74200c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:01.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.530+0000 7f7424f86700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f74200bccb0 0x7f74201517d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:01.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.530+0000 7f7424f86700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74200bacb0 con 0x7f742014ad90 2021-01-06T22:17:01.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.530+0000 7f7424f86700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74200bd130 con 0x7f74200bccb0 2021-01-06T22:17:01.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.530+0000 7f7424f86700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7420152000 con 0x7f74201513f0 2021-01-06T22:17:01.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.530+0000 7f741effd700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f74200bccb0 0x7f74201517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:01.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.530+0000 7f741ffff700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f742014ad90 0x7f74200641e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:45610/0 (socket says 172.21.15.203:45610) 2021-01-06T22:17:01.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.530+0000 7f741ffff700 1 -- 172.21.15.203:0/2237697261 learned_addr learned my addr 172.21.15.203:0/2237697261 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:17:01.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.530+0000 7f741f7fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74201513f0 0x7f74200c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:01.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.530+0000 7f741e7fc700 1 -- 172.21.15.203:0/2237697261 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855383115 0 0) 0x7f7418001450 con 0x7f742014ad90 2021-01-06T22:17:01.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.530+0000 7f741effd700 1 -- 172.21.15.203:0/2237697261 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74201513f0 msgr2=0x7f74200c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:01.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.530+0000 7f741effd700 1 --2- 172.21.15.203:0/2237697261 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74201513f0 0x7f74200c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:01.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.530+0000 7f741effd700 1 -- 172.21.15.203:0/2237697261 >> v1:172.21.15.91:6789/0 conn(0x7f742014ad90 legacy=0x7f74200641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:01.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.530+0000 7f741effd700 1 -- 172.21.15.203:0/2237697261 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7420067e50 con 0x7f74200bccb0 2021-01-06T22:17:01.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f741effd700 1 --2- 172.21.15.203:0/2237697261 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74200bccb0 0x7f74201517d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f741400b980 tx=0x7f7414006de0).ready entity=mon.0 client_cookie=81ecb25b2db2701e server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:01.540 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f741e7fc700 1 -- 172.21.15.203:0/2237697261 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74140097b0 con 0x7f74200bccb0 2021-01-06T22:17:01.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f741e7fc700 1 -- 172.21.15.203:0/2237697261 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7414009910 con 0x7f74200bccb0 2021-01-06T22:17:01.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f741e7fc700 1 -- 172.21.15.203:0/2237697261 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7414009c90 con 0x7f74200bccb0 2021-01-06T22:17:01.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f7424f86700 1 -- 172.21.15.203:0/2237697261 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74200bccb0 msgr2=0x7f74201517d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:01.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f7424f86700 1 --2- 172.21.15.203:0/2237697261 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74200bccb0 0x7f74201517d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f741400b980 tx=0x7f7414006de0).stop 2021-01-06T22:17:01.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f7424f86700 1 -- 172.21.15.203:0/2237697261 shutdown_connections 2021-01-06T22:17:01.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f7424f86700 1 --2- 172.21.15.203:0/2237697261 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74201513f0 0x7f74200c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:01.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f7424f86700 1 --2- 172.21.15.203:0/2237697261 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74200bccb0 0x7f74201517d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:01.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f7424f86700 1 -- 172.21.15.203:0/2237697261 >> 172.21.15.203:0/2237697261 conn(0x7f74200667b0 msgr2=0x7f74200c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:01.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f7424f86700 1 -- 172.21.15.203:0/2237697261 shutdown_connections 2021-01-06T22:17:01.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f7424f86700 1 -- 172.21.15.203:0/2237697261 wait complete. 2021-01-06T22:17:01.545 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f7424f86700 1 Processor -- start 2021-01-06T22:17:01.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f7424f86700 1 -- start start 2021-01-06T22:17:01.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f7424f86700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74200bccb0 0x7f742015dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:01.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f7424f86700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f742014ad90 0x7f74200becb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:01.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f7424f86700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74201513f0 0x7f74201525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:01.549 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f7424f86700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74200c0540 con 0x7f742014ad90 2021-01-06T22:17:01.549 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f7424f86700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7420148580 con 0x7f74201513f0 2021-01-06T22:17:01.549 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f7424f86700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7420147860 con 0x7f74200bccb0 2021-01-06T22:17:01.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f741ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74201513f0 0x7f74201525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:01.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f741ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74201513f0 0x7f74201525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52370/0 (socket says 172.21.15.203:52370) 2021-01-06T22:17:01.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f741ffff700 1 -- 172.21.15.203:0/1301444658 learned_addr learned my addr 172.21.15.203:0/1301444658 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:01.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f741effd700 1 --2- 172.21.15.203:0/1301444658 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f742014ad90 0x7f74200becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:01.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f741f7fe700 1 --2- 172.21.15.203:0/1301444658 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74200bccb0 0x7f742015dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:01.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f741ffff700 1 -- 172.21.15.203:0/1301444658 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74200bccb0 msgr2=0x7f742015dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:01.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f741ffff700 1 --2- 172.21.15.203:0/1301444658 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74200bccb0 0x7f742015dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:01.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f741ffff700 1 -- 172.21.15.203:0/1301444658 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f742014ad90 msgr2=0x7f74200becb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:01.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f741ffff700 1 --2- 172.21.15.203:0/1301444658 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f742014ad90 0x7f74200becb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:01.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.534+0000 7f741ffff700 1 -- 172.21.15.203:0/1301444658 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7420147b00 con 0x7f74201513f0 2021-01-06T22:17:01.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.538+0000 7f741ffff700 1 --2- 172.21.15.203:0/1301444658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74201513f0 0x7f74201525e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f741800ca30 tx=0x7f7418007bd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:01.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.538+0000 7f74077fe700 1 -- 172.21.15.203:0/1301444658 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7418005b70 con 0x7f74201513f0 2021-01-06T22:17:01.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.538+0000 7f74077fe700 1 -- 172.21.15.203:0/1301444658 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7418005cd0 con 0x7f74201513f0 2021-01-06T22:17:01.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.538+0000 7f7424f86700 1 -- 172.21.15.203:0/1301444658 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7420147da0 con 0x7f74201513f0 2021-01-06T22:17:01.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.538+0000 7f74077fe700 1 -- 172.21.15.203:0/1301444658 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7418007320 con 0x7f74201513f0 2021-01-06T22:17:01.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.538+0000 7f7424f86700 1 -- 172.21.15.203:0/1301444658 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74201482e0 con 0x7f74201513f0 2021-01-06T22:17:01.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.538+0000 7f7424f86700 1 -- 172.21.15.203:0/1301444658 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742015e2d0 con 0x7f74201513f0 2021-01-06T22:17:01.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.538+0000 7f74077fe700 1 -- 172.21.15.203:0/1301444658 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f7418010030 con 0x7f74201513f0 2021-01-06T22:17:01.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.538+0000 7f74077fe700 1 --2- 172.21.15.203:0/1301444658 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f74000563d0 0x7f7400058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:01.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.538+0000 7f74077fe700 1 -- 172.21.15.203:0/1301444658 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f741806f440 con 0x7f74201513f0 2021-01-06T22:17:01.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.538+0000 7f741f7fe700 1 --2- 172.21.15.203:0/1301444658 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f74000563d0 0x7f7400058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:01.555 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.538+0000 7f741f7fe700 1 --2- 172.21.15.203:0/1301444658 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f74000563d0 0x7f7400058890 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f741000cac0 tx=0x7f7410006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:01.555 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.546+0000 7f74077fe700 1 -- 172.21.15.203:0/1301444658 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f7418044a10 con 0x7f74201513f0 2021-01-06T22:17:01.765 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.762+0000 7f7424f86700 1 -- 172.21.15.203:0/1301444658 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74200bacb0 con 0x7f74201513f0 2021-01-06T22:17:01.766 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.762+0000 7f74077fe700 1 -- 172.21.15.203:0/1301444658 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+3208 (secure 0 0 0) 0x7f74200bacb0 con 0x7f74201513f0 2021-01-06T22:17:01.766 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:17:01.767 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:55.638183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:17:01.769 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.766+0000 7f7424f86700 1 -- 172.21.15.203:0/1301444658 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f74000563d0 msgr2=0x7f7400058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:01.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.766+0000 7f7424f86700 1 --2- 172.21.15.203:0/1301444658 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f74000563d0 0x7f7400058890 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f741000cac0 tx=0x7f7410006de0).stop 2021-01-06T22:17:01.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.766+0000 7f7424f86700 1 -- 172.21.15.203:0/1301444658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74201513f0 msgr2=0x7f74201525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:01.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.766+0000 7f7424f86700 1 --2- 172.21.15.203:0/1301444658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74201513f0 0x7f74201525e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f741800ca30 tx=0x7f7418007bd0).stop 2021-01-06T22:17:01.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.766+0000 7f7424f86700 1 -- 172.21.15.203:0/1301444658 shutdown_connections 2021-01-06T22:17:01.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.766+0000 7f7424f86700 1 --2- 172.21.15.203:0/1301444658 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f74000563d0 0x7f7400058890 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:01.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.766+0000 7f7424f86700 1 --2- 172.21.15.203:0/1301444658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74201513f0 0x7f74201525e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:01.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.766+0000 7f7424f86700 1 --2- 172.21.15.203:0/1301444658 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74200bccb0 0x7f742015dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:01.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.766+0000 7f7424f86700 1 --2- 172.21.15.203:0/1301444658 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f742014ad90 0x7f74200becb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:01.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.766+0000 7f7424f86700 1 -- 172.21.15.203:0/1301444658 >> 172.21.15.203:0/1301444658 conn(0x7f74200667b0 msgr2=0x7f74201490f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:01.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.766+0000 7f7424f86700 1 -- 172.21.15.203:0/1301444658 shutdown_connections 2021-01-06T22:17:01.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:01.766+0000 7f7424f86700 1 -- 172.21.15.203:0/1301444658 wait complete. 2021-01-06T22:17:01.773 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 67 2021-01-06T22:17:01.785 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2021-01-06T22:17:03.786 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:17:03.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.926+0000 7f00f759e700 1 Processor -- start 2021-01-06T22:17:03.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.926+0000 7f00f759e700 1 -- start start 2021-01-06T22:17:03.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.926+0000 7f00f759e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f00f8061130 0x7f00f80c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:03.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.926+0000 7f00f759e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00f80bc700 0x7f00f814bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:03.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.926+0000 7f00f759e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f814c1c0 con 0x7f00f814f880 2021-01-06T22:17:03.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.926+0000 7f00f759e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00f814c460 con 0x7f00f8061130 2021-01-06T22:17:03.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.926+0000 7f00f759e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f814c700 con 0x7f00f80bc700 2021-01-06T22:17:03.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.926+0000 7f00f6d9d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f00f814f880 0x7f00f8148d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:50978/0 (socket says 172.21.15.203:50978) 2021-01-06T22:17:03.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.926+0000 7f00f6d9d700 1 -- 172.21.15.203:0/59874214 learned_addr learned my addr 172.21.15.203:0/59874214 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:17:03.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f659c700 1 --2- 172.21.15.203:0/59874214 >> v2:172.21.15.91:3300/0 conn(0x7f00f8061130 0x7f00f80c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:03.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00effff700 1 --2- 172.21.15.203:0/59874214 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00f80bc700 0x7f00f814bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:03.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 -- 172.21.15.203:0/59874214 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414032363 0 0) 0x7f00f814c1c0 con 0x7f00f814f880 2021-01-06T22:17:03.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 -- 172.21.15.203:0/59874214 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d8003230 con 0x7f00f814f880 2021-01-06T22:17:03.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 -- 172.21.15.203:0/59874214 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 267442172 0 0) 0x7f00d8003230 con 0x7f00f814f880 2021-01-06T22:17:03.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 -- 172.21.15.203:0/59874214 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00f80bc700 msgr2=0x7f00f814bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:03.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 --2- 172.21.15.203:0/59874214 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00f80bc700 0x7f00f814bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:03.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 -- 172.21.15.203:0/59874214 >> v2:172.21.15.91:3300/0 conn(0x7f00f8061130 msgr2=0x7f00f80c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:03.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 --2- 172.21.15.203:0/59874214 >> v2:172.21.15.91:3300/0 conn(0x7f00f8061130 0x7f00f80c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:03.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 -- 172.21.15.203:0/59874214 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f814c9a0 con 0x7f00f814f880 2021-01-06T22:17:03.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00effff700 1 --2- 172.21.15.203:0/59874214 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00f80bc700 0x7f00f814bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:17:03.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 -- 172.21.15.203:0/59874214 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f00f0001860 con 0x7f00f814f880 2021-01-06T22:17:03.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 -- 172.21.15.203:0/59874214 >> v1:172.21.15.203:6789/0 conn(0x7f00f814f880 legacy=0x7f00f8148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:03.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 --2- 172.21.15.203:0/59874214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00d8003720 0x7f00d8003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:03.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 --2- 172.21.15.203:0/59874214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d80044d0 0x7f00d8004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:03.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 --2- 172.21.15.203:0/59874214 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00f80bc700 0x7f00d8004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:03.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 -- 172.21.15.203:0/59874214 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f814c460 con 0x7f00d8003720 2021-01-06T22:17:03.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 -- 172.21.15.203:0/59874214 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f814c700 con 0x7f00d80044d0 2021-01-06T22:17:03.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 -- 172.21.15.203:0/59874214 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d8004220 con 0x7f00f80bc700 2021-01-06T22:17:03.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00effff700 1 --2- 172.21.15.203:0/59874214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d80044d0 0x7f00d8004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:03.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f659c700 1 --2- 172.21.15.203:0/59874214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00d8003720 0x7f00d8003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:03.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f6d9d700 1 --2- 172.21.15.203:0/59874214 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00f80bc700 0x7f00d8004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:03.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00effff700 1 -- 172.21.15.203:0/59874214 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00f80bc700 msgr2=0x7f00d8004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:03.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00effff700 1 --2- 172.21.15.203:0/59874214 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00f80bc700 0x7f00d8004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:03.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00effff700 1 -- 172.21.15.203:0/59874214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00d8003720 msgr2=0x7f00d8003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:03.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00effff700 1 --2- 172.21.15.203:0/59874214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00d8003720 0x7f00d8003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:03.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00effff700 1 -- 172.21.15.203:0/59874214 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d8003230 con 0x7f00d80044d0 2021-01-06T22:17:03.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00effff700 1 --2- 172.21.15.203:0/59874214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d80044d0 0x7f00d8004950 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f00e000be10 tx=0x7f00e0008040).ready entity=mon.1 client_cookie=6571c7970c07a6f3 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:03.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 -- 172.21.15.203:0/59874214 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e000b4f0 con 0x7f00d80044d0 2021-01-06T22:17:03.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 -- 172.21.15.203:0/59874214 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e000edc0 con 0x7f00d80044d0 2021-01-06T22:17:03.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f5d9b700 1 -- 172.21.15.203:0/59874214 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0012640 con 0x7f00d80044d0 2021-01-06T22:17:03.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f759e700 1 -- 172.21.15.203:0/59874214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d80044d0 msgr2=0x7f00d8004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:03.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f759e700 1 --2- 172.21.15.203:0/59874214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d80044d0 0x7f00d8004950 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f00e000be10 tx=0x7f00e0008040).stop 2021-01-06T22:17:03.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f6d9d700 1 -- 172.21.15.203:0/59874214 reap_dead start 2021-01-06T22:17:03.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f759e700 1 -- 172.21.15.203:0/59874214 shutdown_connections 2021-01-06T22:17:03.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f759e700 1 -- 172.21.15.203:0/59874214 >> 172.21.15.203:0/59874214 conn(0x7f00f80667b0 msgr2=0x7f00f80c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:03.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f759e700 1 -- 172.21.15.203:0/59874214 shutdown_connections 2021-01-06T22:17:03.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.930+0000 7f00f759e700 1 -- 172.21.15.203:0/59874214 wait complete. 2021-01-06T22:17:03.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f759e700 1 Processor -- start 2021-01-06T22:17:03.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f759e700 1 -- start start 2021-01-06T22:17:03.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f759e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d80044d0 0x7f00f8160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:03.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f759e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00f8141a60 0x7f00f813fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:03.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f759e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00f814e640 0x7f00f813ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:03.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f759e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f80c0610 con 0x7f00f814e640 2021-01-06T22:17:03.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f759e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f80bf7f0 con 0x7f00d80044d0 2021-01-06T22:17:03.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f759e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f80bf010 con 0x7f00f8141a60 2021-01-06T22:17:03.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d80044d0 0x7f00f8160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:03.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d80044d0 0x7f00f8160820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52392/0 (socket says 172.21.15.203:52392) 2021-01-06T22:17:03.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f659c700 1 -- 172.21.15.203:0/2062006226 learned_addr learned my addr 172.21.15.203:0/2062006226 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:03.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00effff700 1 --2- 172.21.15.203:0/2062006226 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00f8141a60 0x7f00f813fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:03.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f6d9d700 1 --2- 172.21.15.203:0/2062006226 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00f814e640 0x7f00f813ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:03.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f659c700 1 -- 172.21.15.203:0/2062006226 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00f8141a60 msgr2=0x7f00f813fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:03.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f659c700 1 --2- 172.21.15.203:0/2062006226 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00f8141a60 0x7f00f813fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:03.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f659c700 1 -- 172.21.15.203:0/2062006226 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00f814e640 msgr2=0x7f00f813ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:03.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f659c700 1 --2- 172.21.15.203:0/2062006226 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00f814e640 0x7f00f813ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:03.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f659c700 1 -- 172.21.15.203:0/2062006226 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f80bf2b0 con 0x7f00d80044d0 2021-01-06T22:17:03.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f6d9d700 1 --2- 172.21.15.203:0/2062006226 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00f814e640 0x7f00f813ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:17:03.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f659c700 1 --2- 172.21.15.203:0/2062006226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d80044d0 0x7f00f8160820 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f00e8010c50 tx=0x7f00e8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:03.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00ee7fc700 1 -- 172.21.15.203:0/2062006226 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e800eac0 con 0x7f00d80044d0 2021-01-06T22:17:03.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00ee7fc700 1 -- 172.21.15.203:0/2062006226 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e800ec20 con 0x7f00d80044d0 2021-01-06T22:17:03.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f759e700 1 -- 172.21.15.203:0/2062006226 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f80bf550 con 0x7f00d80044d0 2021-01-06T22:17:03.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00ee7fc700 1 -- 172.21.15.203:0/2062006226 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e8014470 con 0x7f00d80044d0 2021-01-06T22:17:03.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.934+0000 7f00f759e700 1 -- 172.21.15.203:0/2062006226 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f8140bb0 con 0x7f00d80044d0 2021-01-06T22:17:03.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.938+0000 7f00ee7fc700 1 -- 172.21.15.203:0/2062006226 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f00e8024070 con 0x7f00d80044d0 2021-01-06T22:17:03.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.938+0000 7f00ee7fc700 1 --2- 172.21.15.203:0/2062006226 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f00dc056400 0x7f00dc0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:03.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.938+0000 7f00ee7fc700 1 -- 172.21.15.203:0/2062006226 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f00e80700c0 con 0x7f00d80044d0 2021-01-06T22:17:03.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.938+0000 7f00effff700 1 --2- 172.21.15.203:0/2062006226 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f00dc056400 0x7f00dc0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:03.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.938+0000 7f00effff700 1 --2- 172.21.15.203:0/2062006226 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f00dc056400 0x7f00dc0588c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f00f8141ea0 tx=0x7f00e001a000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:03.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.938+0000 7f00f759e700 1 -- 172.21.15.203:0/2062006226 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f80450a0 con 0x7f00d80044d0 2021-01-06T22:17:03.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:03.946+0000 7f00ee7fc700 1 -- 172.21.15.203:0/2062006226 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f00e8018080 con 0x7f00d80044d0 2021-01-06T22:17:04.163 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:04.158+0000 7f00f759e700 1 -- 172.21.15.203:0/2062006226 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00f8061f70 con 0x7f00d80044d0 2021-01-06T22:17:04.165 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:04.162+0000 7f00ee7fc700 1 -- 172.21.15.203:0/2062006226 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+3564 (secure 0 0 0) 0x7f00e8045690 con 0x7f00d80044d0 2021-01-06T22:17:04.165 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:17:04.166 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5632,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:55.638183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:17:04.167 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:04.162+0000 7f00f759e700 1 -- 172.21.15.203:0/2062006226 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f00dc056400 msgr2=0x7f00dc0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:04.167 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:04.162+0000 7f00f759e700 1 --2- 172.21.15.203:0/2062006226 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f00dc056400 0x7f00dc0588c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f00f8141ea0 tx=0x7f00e001a000).stop 2021-01-06T22:17:04.168 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:04.162+0000 7f00f759e700 1 -- 172.21.15.203:0/2062006226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d80044d0 msgr2=0x7f00f8160820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:04.168 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:04.162+0000 7f00f759e700 1 --2- 172.21.15.203:0/2062006226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d80044d0 0x7f00f8160820 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f00e8010c50 tx=0x7f00e8008040).stop 2021-01-06T22:17:04.168 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:04.162+0000 7f00f759e700 1 -- 172.21.15.203:0/2062006226 shutdown_connections 2021-01-06T22:17:04.168 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:04.162+0000 7f00f759e700 1 --2- 172.21.15.203:0/2062006226 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f00dc056400 0x7f00dc0588c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:04.169 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:04.162+0000 7f00f759e700 1 --2- 172.21.15.203:0/2062006226 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00f814e640 0x7f00f813ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:04.169 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:04.162+0000 7f00f759e700 1 --2- 172.21.15.203:0/2062006226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d80044d0 0x7f00f8160820 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:04.169 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:04.162+0000 7f00f759e700 1 --2- 172.21.15.203:0/2062006226 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00f8141a60 0x7f00f813fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:04.169 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:04.162+0000 7f00f759e700 1 -- 172.21.15.203:0/2062006226 >> 172.21.15.203:0/2062006226 conn(0x7f00f80667b0 msgr2=0x7f00f80b9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:04.169 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:04.162+0000 7f00f759e700 1 -- 172.21.15.203:0/2062006226 shutdown_connections 2021-01-06T22:17:04.170 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:04.162+0000 7f00f759e700 1 -- 172.21.15.203:0/2062006226 wait complete. 2021-01-06T22:17:04.170 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 68 2021-01-06T22:17:04.179 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2021-01-06T22:17:04.180 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 26 secs before thrashing 2021-01-06T22:17:17.900 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:17:17.903 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:17:30.181 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:17:30.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.994+0000 7f9b3b59e700 1 Processor -- start 2021-01-06T22:17:30.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.994+0000 7f9b3b59e700 1 -- start start 2021-01-06T22:17:31.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3b59e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9b3c0c48d0 0x7f9b3c150e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:31.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3b59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3c0c0f50 con 0x7f9b3c0c1a90 2021-01-06T22:17:31.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3b59e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3c066cf0 con 0x7f9b3c0c44f0 2021-01-06T22:17:31.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3b59e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3c060750 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3a59c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f9b3c0c44f0 0x7f9b3c0c1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51010/0 (socket says 172.21.15.203:51010) 2021-01-06T22:17:31.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3a59c700 1 -- 172.21.15.203:0/2886412136 learned_addr learned my addr 172.21.15.203:0/2886412136 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:17:31.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3ad9d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9b3c0c48d0 0x7f9b3c150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:31.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3959a700 1 -- 172.21.15.203:0/2886412136 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846372202 0 0) 0x7f9b3c066cf0 con 0x7f9b3c0c44f0 2021-01-06T22:17:31.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3959a700 1 -- 172.21.15.203:0/2886412136 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b20003230 con 0x7f9b3c0c44f0 2021-01-06T22:17:31.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3ad9d700 1 -- 172.21.15.203:0/2886412136 >> v1:172.21.15.203:6789/0 conn(0x7f9b3c0c44f0 legacy=0x7f9b3c0c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:31.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3ad9d700 1 -- 172.21.15.203:0/2886412136 >> v1:172.21.15.91:6789/0 conn(0x7f9b3c0c1a90 legacy=0x7f9b3c150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:31.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3ad9d700 1 -- 172.21.15.203:0/2886412136 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c0609f0 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3ad9d700 1 --2- 172.21.15.203:0/2886412136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b3c0c48d0 0x7f9b3c150e40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f9b2c00b980 tx=0x7f9b2c006de0).ready entity=mon.1 client_cookie=5995e2981848aa6d server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:31.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3959a700 1 -- 172.21.15.203:0/2886412136 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c0097b0 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3959a700 1 -- 172.21.15.203:0/2886412136 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2c009910 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3959a700 1 -- 172.21.15.203:0/2886412136 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c009c90 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3b59e700 1 -- 172.21.15.203:0/2886412136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b3c0c48d0 msgr2=0x7f9b3c150e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:31.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3b59e700 1 --2- 172.21.15.203:0/2886412136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b3c0c48d0 0x7f9b3c150e40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f9b2c00b980 tx=0x7f9b2c006de0).stop 2021-01-06T22:17:31.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3b59e700 1 -- 172.21.15.203:0/2886412136 shutdown_connections 2021-01-06T22:17:31.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3b59e700 1 --2- 172.21.15.203:0/2886412136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b3c0c48d0 0x7f9b3c150e40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3b59e700 1 -- 172.21.15.203:0/2886412136 >> 172.21.15.203:0/2886412136 conn(0x7f9b3c056cd0 msgr2=0x7f9b3c061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:31.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3b59e700 1 -- 172.21.15.203:0/2886412136 shutdown_connections 2021-01-06T22:17:31.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3b59e700 1 -- 172.21.15.203:0/2886412136 wait complete. 2021-01-06T22:17:31.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3b59e700 1 Processor -- start 2021-01-06T22:17:31.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:30.998+0000 7f9b3b59e700 1 -- start start 2021-01-06T22:17:31.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3b59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9b3c0c1a90 0x7f9b3c14f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:31.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3b59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9b3c0c44f0 0x7f9b3c147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:31.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3b59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b3c0c48d0 0x7f9b3c151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:31.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3b59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0b86a0 con 0x7f9b3c0c44f0 2021-01-06T22:17:31.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3b59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0b88d0 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3b59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0b8b70 con 0x7f9b3c0c1a90 2021-01-06T22:17:31.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3ad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b3c0c48d0 0x7f9b3c151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:31.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3ad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b3c0c48d0 0x7f9b3c151680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52426/0 (socket says 172.21.15.203:52426) 2021-01-06T22:17:31.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3ad9d700 1 -- 172.21.15.203:0/3598869567 learned_addr learned my addr 172.21.15.203:0/3598869567 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:31.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3a59c700 1 --2- 172.21.15.203:0/3598869567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9b3c0c1a90 0x7f9b3c14f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:31.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b39d9b700 1 --2- 172.21.15.203:0/3598869567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9b3c0c44f0 0x7f9b3c147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:31.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3ad9d700 1 -- 172.21.15.203:0/3598869567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9b3c0c1a90 msgr2=0x7f9b3c14f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:31.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3ad9d700 1 --2- 172.21.15.203:0/3598869567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9b3c0c1a90 0x7f9b3c14f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3ad9d700 1 -- 172.21.15.203:0/3598869567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9b3c0c44f0 msgr2=0x7f9b3c147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:31.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3ad9d700 1 --2- 172.21.15.203:0/3598869567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9b3c0c44f0 0x7f9b3c147db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3ad9d700 1 -- 172.21.15.203:0/3598869567 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c0b8e10 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3ad9d700 1 --2- 172.21.15.203:0/3598869567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b3c0c48d0 0x7f9b3c151680 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f9b2c00be40 tx=0x7f9b2c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:31.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b2a7fc700 1 -- 172.21.15.203:0/3598869567 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c011ea0 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b2a7fc700 1 -- 172.21.15.203:0/3598869567 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2c01d400 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3b59e700 1 -- 172.21.15.203:0/3598869567 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3c0b90b0 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b2a7fc700 1 -- 172.21.15.203:0/3598869567 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c01d7a0 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3b59e700 1 -- 172.21.15.203:0/3598869567 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3c0b95f0 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.002+0000 7f9b3b59e700 1 -- 172.21.15.203:0/3598869567 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3c0450a0 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.006+0000 7f9b2a7fc700 1 -- 172.21.15.203:0/3598869567 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f9b2c00f070 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.006+0000 7f9b2a7fc700 1 --2- 172.21.15.203:0/3598869567 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9b180563d0 0x7f9b18058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:31.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.006+0000 7f9b2a7fc700 1 -- 172.21.15.203:0/3598869567 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f9b2c071180 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.006+0000 7f9b3a59c700 1 --2- 172.21.15.203:0/3598869567 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9b180563d0 0x7f9b18058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:31.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.006+0000 7f9b3a59c700 1 --2- 172.21.15.203:0/3598869567 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9b180563d0 0x7f9b18058890 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f9b34005590 tx=0x7f9b34009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:31.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.010+0000 7f9b2a7fc700 1 -- 172.21.15.203:0/3598869567 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f9b2c046750 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.230+0000 7f9b3b59e700 1 -- 172.21.15.203:0/3598869567 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b3c0b7da0 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.230+0000 7f9b2a7fc700 1 -- 172.21.15.203:0/3598869567 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+3564 (secure 0 0 0) 0x7f9b2c021090 con 0x7f9b3c0c48d0 2021-01-06T22:17:31.233 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:17:31.233 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5632,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:55.638183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:17:31.235 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.230+0000 7f9b3b59e700 1 -- 172.21.15.203:0/3598869567 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9b180563d0 msgr2=0x7f9b18058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:31.235 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.230+0000 7f9b3b59e700 1 --2- 172.21.15.203:0/3598869567 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9b180563d0 0x7f9b18058890 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f9b34005590 tx=0x7f9b34009040).stop 2021-01-06T22:17:31.235 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.230+0000 7f9b3b59e700 1 -- 172.21.15.203:0/3598869567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b3c0c48d0 msgr2=0x7f9b3c151680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:31.236 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.230+0000 7f9b3b59e700 1 --2- 172.21.15.203:0/3598869567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b3c0c48d0 0x7f9b3c151680 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f9b2c00be40 tx=0x7f9b2c016000).stop 2021-01-06T22:17:31.236 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.230+0000 7f9b3b59e700 1 -- 172.21.15.203:0/3598869567 shutdown_connections 2021-01-06T22:17:31.236 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.230+0000 7f9b3b59e700 1 --2- 172.21.15.203:0/3598869567 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9b180563d0 0x7f9b18058890 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.236 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.230+0000 7f9b3b59e700 1 --2- 172.21.15.203:0/3598869567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b3c0c48d0 0x7f9b3c151680 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.237 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.230+0000 7f9b3b59e700 1 --2- 172.21.15.203:0/3598869567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9b3c0c1a90 0x7f9b3c14f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.237 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.230+0000 7f9b3b59e700 1 --2- 172.21.15.203:0/3598869567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9b3c0c44f0 0x7f9b3c147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.237 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.230+0000 7f9b3b59e700 1 -- 172.21.15.203:0/3598869567 >> 172.21.15.203:0/3598869567 conn(0x7f9b3c056cd0 msgr2=0x7f9b3c0bd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:31.237 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.234+0000 7f9b3b59e700 1 -- 172.21.15.203:0/3598869567 shutdown_connections 2021-01-06T22:17:31.237 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.234+0000 7f9b3b59e700 1 -- 172.21.15.203:0/3598869567 wait complete. 2021-01-06T22:17:31.238 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 68 2021-01-06T22:17:31.248 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=1) 2021-01-06T22:17:31.248 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2021-01-06T22:17:31.249 INFO:teuthology.orchestra.run:waiting for 300 2021-01-06T22:17:31.262 INFO:tasks.ceph.mds.f:Stopped 2021-01-06T22:17:31.262 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:17:31.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.382+0000 7fe6a6ec3700 1 Processor -- start 2021-01-06T22:17:31.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a6ec3700 1 -- start start 2021-01-06T22:17:31.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a6ec3700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe6a0061130 0x7fe6a00c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:31.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a6ec3700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6a00bc700 0x7fe6a014bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:31.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a6ec3700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe6a014f880 0x7fe6a0148d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:31.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a6ec3700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a014c1c0 con 0x7fe6a014f880 2021-01-06T22:17:31.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a6ec3700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a014c460 con 0x7fe6a0061130 2021-01-06T22:17:31.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a6ec3700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a014c700 con 0x7fe6a00bc700 2021-01-06T22:17:31.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a5ec1700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe6a0061130 0x7fe6a00c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:31.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a5ec1700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe6a0061130 0x7fe6a00c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52434/0 (socket says 172.21.15.203:52434) 2021-01-06T22:17:31.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a5ec1700 1 -- 172.21.15.203:0/3809258347 learned_addr learned my addr 172.21.15.203:0/3809258347 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:31.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a56c0700 1 --2- 172.21.15.203:0/3809258347 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6a00bc700 0x7fe6a014bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:31.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a66c2700 1 --2- 172.21.15.203:0/3809258347 >> v2:172.21.15.91:3300/0 conn(0x7fe6a014f880 0x7fe6a0148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:31.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a5ec1700 1 -- 172.21.15.203:0/3809258347 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6a00bc700 msgr2=0x7fe6a014bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:31.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a5ec1700 1 --2- 172.21.15.203:0/3809258347 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6a00bc700 0x7fe6a014bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a5ec1700 1 -- 172.21.15.203:0/3809258347 >> v2:172.21.15.91:3300/0 conn(0x7fe6a014f880 msgr2=0x7fe6a0148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:31.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a5ec1700 1 --2- 172.21.15.203:0/3809258347 >> v2:172.21.15.91:3300/0 conn(0x7fe6a014f880 0x7fe6a0148d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a5ec1700 1 -- 172.21.15.203:0/3809258347 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a014c9a0 con 0x7fe6a0061130 2021-01-06T22:17:31.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a5ec1700 1 --2- 172.21.15.203:0/3809258347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a0061130 0x7fe6a00c0fa0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fe6a00c07d0 tx=0x7fe69c009290).ready entity=mon.1 client_cookie=990c2e6ef94aa19e server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:31.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a4ebf700 1 -- 172.21.15.203:0/3809258347 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c00bc50 con 0x7fe6a0061130 2021-01-06T22:17:31.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a4ebf700 1 -- 172.21.15.203:0/3809258347 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c00bdb0 con 0x7fe6a0061130 2021-01-06T22:17:31.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a4ebf700 1 -- 172.21.15.203:0/3809258347 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c011610 con 0x7fe6a0061130 2021-01-06T22:17:31.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a6ec3700 1 -- 172.21.15.203:0/3809258347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a0061130 msgr2=0x7fe6a00c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:31.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a6ec3700 1 --2- 172.21.15.203:0/3809258347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a0061130 0x7fe6a00c0fa0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fe6a00c07d0 tx=0x7fe69c009290).stop 2021-01-06T22:17:31.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a6ec3700 1 -- 172.21.15.203:0/3809258347 shutdown_connections 2021-01-06T22:17:31.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a6ec3700 1 --2- 172.21.15.203:0/3809258347 >> v2:172.21.15.91:3300/0 conn(0x7fe6a014f880 0x7fe6a0148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a6ec3700 1 --2- 172.21.15.203:0/3809258347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a0061130 0x7fe6a00c0fa0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a6ec3700 1 --2- 172.21.15.203:0/3809258347 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6a00bc700 0x7fe6a014bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a6ec3700 1 -- 172.21.15.203:0/3809258347 >> 172.21.15.203:0/3809258347 conn(0x7fe6a00667b0 msgr2=0x7fe6a00c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:31.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a6ec3700 1 -- 172.21.15.203:0/3809258347 shutdown_connections 2021-01-06T22:17:31.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.386+0000 7fe6a6ec3700 1 -- 172.21.15.203:0/3809258347 wait complete. 2021-01-06T22:17:31.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a6ec3700 1 Processor -- start 2021-01-06T22:17:31.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a6ec3700 1 -- start start 2021-01-06T22:17:31.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a6ec3700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6a0061130 0x7fe6a00bff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:31.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a6ec3700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6a00bc700 0x7fe6a015dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:31.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a6ec3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a014f880 0x7fe6a0145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:31.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a6ec3700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a015d4e0 con 0x7fe6a00bc700 2021-01-06T22:17:31.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a6ec3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a00b1840 con 0x7fe6a014f880 2021-01-06T22:17:31.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a6ec3700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a00b8090 con 0x7fe6a0061130 2021-01-06T22:17:31.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a66c2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a014f880 0x7fe6a0145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:31.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a66c2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a014f880 0x7fe6a0145d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52442/0 (socket says 172.21.15.203:52442) 2021-01-06T22:17:31.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a66c2700 1 -- 172.21.15.203:0/3268759114 learned_addr learned my addr 172.21.15.203:0/3268759114 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:31.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a56c0700 1 --2- 172.21.15.203:0/3268759114 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6a00bc700 0x7fe6a015dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:31.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a5ec1700 1 --2- 172.21.15.203:0/3268759114 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6a0061130 0x7fe6a00bff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:31.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a66c2700 1 -- 172.21.15.203:0/3268759114 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6a0061130 msgr2=0x7fe6a00bff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:31.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a66c2700 1 --2- 172.21.15.203:0/3268759114 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6a0061130 0x7fe6a00bff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a66c2700 1 -- 172.21.15.203:0/3268759114 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6a00bc700 msgr2=0x7fe6a015dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:31.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a66c2700 1 --2- 172.21.15.203:0/3268759114 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6a00bc700 0x7fe6a015dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a66c2700 1 -- 172.21.15.203:0/3268759114 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a00b8330 con 0x7fe6a014f880 2021-01-06T22:17:31.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a56c0700 1 --2- 172.21.15.203:0/3268759114 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6a00bc700 0x7fe6a015dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:17:31.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.390+0000 7fe6a66c2700 1 --2- 172.21.15.203:0/3268759114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a014f880 0x7fe6a0145d40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fe69800b7b0 tx=0x7fe698008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:31.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.394+0000 7fe68dffb700 1 -- 172.21.15.203:0/3268759114 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69800f5e0 con 0x7fe6a014f880 2021-01-06T22:17:31.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.394+0000 7fe68dffb700 1 -- 172.21.15.203:0/3268759114 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69800f740 con 0x7fe6a014f880 2021-01-06T22:17:31.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.394+0000 7fe6a6ec3700 1 -- 172.21.15.203:0/3268759114 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a015d740 con 0x7fe6a014f880 2021-01-06T22:17:31.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.394+0000 7fe68dffb700 1 -- 172.21.15.203:0/3268759114 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69800fae0 con 0x7fe6a014f880 2021-01-06T22:17:31.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.394+0000 7fe6a6ec3700 1 -- 172.21.15.203:0/3268759114 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a014d180 con 0x7fe6a014f880 2021-01-06T22:17:31.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.394+0000 7fe6a6ec3700 1 -- 172.21.15.203:0/3268759114 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a00450a0 con 0x7fe6a014f880 2021-01-06T22:17:31.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.394+0000 7fe68dffb700 1 -- 172.21.15.203:0/3268759114 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fe69801c460 con 0x7fe6a014f880 2021-01-06T22:17:31.404 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.398+0000 7fe68dffb700 1 --2- 172.21.15.203:0/3268759114 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe6800563d0 0x7fe680058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:31.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.398+0000 7fe68dffb700 1 -- 172.21.15.203:0/3268759114 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6761+0+0 (secure 0 0 0) 0x7fe69806f7b0 con 0x7fe6a014f880 2021-01-06T22:17:31.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.398+0000 7fe6a5ec1700 1 --2- 172.21.15.203:0/3268759114 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe6800563d0 0x7fe680058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:31.405 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.398+0000 7fe6a5ec1700 1 --2- 172.21.15.203:0/3268759114 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe6800563d0 0x7fe680058890 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fe69c0092d0 tx=0x7fe69c017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:31.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.402+0000 7fe68dffb700 1 -- 172.21.15.203:0/3268759114 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fe698044d20 con 0x7fe6a014f880 2021-01-06T22:17:31.652 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.650+0000 7fe6a6ec3700 1 -- 172.21.15.203:0/3268759114 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6a005ca60 con 0x7fe6a014f880 2021-01-06T22:17:31.653 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.650+0000 7fe68dffb700 1 -- 172.21.15.203:0/3268759114 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+3564 (secure 0 0 0) 0x7fe698048340 con 0x7fe6a014f880 2021-01-06T22:17:31.653 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:17:31.654 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5632,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:55.638183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:17:31.655 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.650+0000 7fe6a6ec3700 1 -- 172.21.15.203:0/3268759114 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe6800563d0 msgr2=0x7fe680058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:31.655 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.650+0000 7fe6a6ec3700 1 --2- 172.21.15.203:0/3268759114 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe6800563d0 0x7fe680058890 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fe69c0092d0 tx=0x7fe69c017000).stop 2021-01-06T22:17:31.656 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.650+0000 7fe6a6ec3700 1 -- 172.21.15.203:0/3268759114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a014f880 msgr2=0x7fe6a0145d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:31.656 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.650+0000 7fe6a6ec3700 1 --2- 172.21.15.203:0/3268759114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a014f880 0x7fe6a0145d40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fe69800b7b0 tx=0x7fe698008040).stop 2021-01-06T22:17:31.657 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.650+0000 7fe6a6ec3700 1 -- 172.21.15.203:0/3268759114 shutdown_connections 2021-01-06T22:17:31.657 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.650+0000 7fe6a6ec3700 1 --2- 172.21.15.203:0/3268759114 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe6800563d0 0x7fe680058890 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.657 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.650+0000 7fe6a6ec3700 1 --2- 172.21.15.203:0/3268759114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a014f880 0x7fe6a0145d40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.650+0000 7fe6a6ec3700 1 --2- 172.21.15.203:0/3268759114 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6a0061130 0x7fe6a00bff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.650+0000 7fe6a6ec3700 1 --2- 172.21.15.203:0/3268759114 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6a00bc700 0x7fe6a015dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:31.658 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.650+0000 7fe6a6ec3700 1 -- 172.21.15.203:0/3268759114 >> 172.21.15.203:0/3268759114 conn(0x7fe6a00667b0 msgr2=0x7fe6a014d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:31.659 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.654+0000 7fe6a6ec3700 1 -- 172.21.15.203:0/3268759114 shutdown_connections 2021-01-06T22:17:31.659 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:31.654+0000 7fe6a6ec3700 1 -- 172.21.15.203:0/3268759114 wait complete. 2021-01-06T22:17:31.660 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 68 2021-01-06T22:17:31.670 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:17:33.671 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:17:33.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.794+0000 7ff4b359e700 1 Processor -- start 2021-01-06T22:17:33.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.794+0000 7ff4b359e700 1 -- start start 2021-01-06T22:17:33.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b359e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff4b40bed80 0x7ff4b41566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:33.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b359e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b40be410 con 0x7ff4b40b97a0 2021-01-06T22:17:33.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b359e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b4066510 con 0x7ff4b40b9b80 2021-01-06T22:17:33.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b359e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b40667b0 con 0x7ff4b40bed80 2021-01-06T22:17:33.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b2d9d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff4b40bed80 0x7ff4b41566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:33.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b2d9d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff4b40bed80 0x7ff4b41566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52472/0 (socket says 172.21.15.203:52472) 2021-01-06T22:17:33.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b2d9d700 1 -- 172.21.15.203:0/374439109 learned_addr learned my addr 172.21.15.203:0/374439109 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:33.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b2d9d700 1 -- 172.21.15.203:0/374439109 >> v1:172.21.15.203:6789/0 conn(0x7ff4b40b9b80 legacy=0x7ff4b4065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:33.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b2d9d700 1 -- 172.21.15.203:0/374439109 >> v1:172.21.15.91:6789/0 conn(0x7ff4b40b97a0 legacy=0x7ff4b40bd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:33.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b2d9d700 1 -- 172.21.15.203:0/374439109 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b4066a50 con 0x7ff4b40bed80 2021-01-06T22:17:33.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b2d9d700 1 --2- 172.21.15.203:0/374439109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b40bed80 0x7ff4b41566f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7ff4ac009d50 tx=0x7ff4ac009290).ready entity=mon.1 client_cookie=1834b68978a57aba server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:33.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b159a700 1 -- 172.21.15.203:0/374439109 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00bd10 con 0x7ff4b40bed80 2021-01-06T22:17:33.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b159a700 1 -- 172.21.15.203:0/374439109 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac00be70 con 0x7ff4b40bed80 2021-01-06T22:17:33.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b159a700 1 -- 172.21.15.203:0/374439109 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac011610 con 0x7ff4b40bed80 2021-01-06T22:17:33.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b359e700 1 -- 172.21.15.203:0/374439109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b40bed80 msgr2=0x7ff4b41566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:33.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b359e700 1 --2- 172.21.15.203:0/374439109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b40bed80 0x7ff4b41566f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7ff4ac009d50 tx=0x7ff4ac009290).stop 2021-01-06T22:17:33.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b359e700 1 -- 172.21.15.203:0/374439109 shutdown_connections 2021-01-06T22:17:33.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b359e700 1 --2- 172.21.15.203:0/374439109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b40bed80 0x7ff4b41566f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:33.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b359e700 1 -- 172.21.15.203:0/374439109 >> 172.21.15.203:0/374439109 conn(0x7ff4b40648f0 msgr2=0x7ff4b4064cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:33.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b359e700 1 -- 172.21.15.203:0/374439109 shutdown_connections 2021-01-06T22:17:33.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.798+0000 7ff4b359e700 1 -- 172.21.15.203:0/374439109 wait complete. 2021-01-06T22:17:33.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b359e700 1 Processor -- start 2021-01-06T22:17:33.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b359e700 1 -- start start 2021-01-06T22:17:33.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b359e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4b40b97a0 0x7ff4b415bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:33.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b359e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b40b9b80 0x7ff4b40a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:33.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b359e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff4b40bed80 0x7ff4b40a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:33.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b359e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40a1a70 con 0x7ff4b40b97a0 2021-01-06T22:17:33.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b359e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40a6d50 con 0x7ff4b40b9b80 2021-01-06T22:17:33.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b359e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b415c270 con 0x7ff4b40bed80 2021-01-06T22:17:33.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b1d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b40b9b80 0x7ff4b40a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:33.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b1d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b40b9b80 0x7ff4b40a1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52476/0 (socket says 172.21.15.203:52476) 2021-01-06T22:17:33.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b1d9b700 1 -- 172.21.15.203:0/663849294 learned_addr learned my addr 172.21.15.203:0/663849294 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:33.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b2d9d700 1 --2- 172.21.15.203:0/663849294 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff4b40bed80 0x7ff4b40a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:33.969 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b259c700 1 --2- 172.21.15.203:0/663849294 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4b40b97a0 0x7ff4b415bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:33.969 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b1d9b700 1 -- 172.21.15.203:0/663849294 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff4b40bed80 msgr2=0x7ff4b40a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:33.969 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b1d9b700 1 --2- 172.21.15.203:0/663849294 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff4b40bed80 0x7ff4b40a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:33.969 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b1d9b700 1 -- 172.21.15.203:0/663849294 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4b40b97a0 msgr2=0x7ff4b415bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:33.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b1d9b700 1 --2- 172.21.15.203:0/663849294 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4b40b97a0 0x7ff4b415bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:33.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b1d9b700 1 -- 172.21.15.203:0/663849294 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b40a6f80 con 0x7ff4b40b9b80 2021-01-06T22:17:33.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b1d9b700 1 --2- 172.21.15.203:0/663849294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b40b9b80 0x7ff4b40a1090 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7ff4a800c770 tx=0x7ff4a8007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:33.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4a27fc700 1 -- 172.21.15.203:0/663849294 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8007020 con 0x7ff4b40b9b80 2021-01-06T22:17:33.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4a27fc700 1 -- 172.21.15.203:0/663849294 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a8007180 con 0x7ff4b40b9b80 2021-01-06T22:17:33.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b359e700 1 -- 172.21.15.203:0/663849294 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b40a7220 con 0x7ff4b40b9b80 2021-01-06T22:17:33.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4a27fc700 1 -- 172.21.15.203:0/663849294 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800e560 con 0x7ff4b40b9b80 2021-01-06T22:17:33.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.802+0000 7ff4b359e700 1 -- 172.21.15.203:0/663849294 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b40a06b0 con 0x7ff4b40b9b80 2021-01-06T22:17:33.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.806+0000 7ff4b359e700 1 -- 172.21.15.203:0/663849294 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b40450a0 con 0x7ff4b40b9b80 2021-01-06T22:17:33.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.806+0000 7ff4a27fc700 1 -- 172.21.15.203:0/663849294 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff4a8018020 con 0x7ff4b40b9b80 2021-01-06T22:17:33.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.806+0000 7ff4a27fc700 1 --2- 172.21.15.203:0/663849294 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff490056400 0x7ff4900588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:33.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.806+0000 7ff4a27fc700 1 -- 172.21.15.203:0/663849294 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6761+0+0 (secure 0 0 0) 0x7ff4a806f040 con 0x7ff4b40b9b80 2021-01-06T22:17:33.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.806+0000 7ff4b259c700 1 --2- 172.21.15.203:0/663849294 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff490056400 0x7ff4900588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:33.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.806+0000 7ff4b259c700 1 --2- 172.21.15.203:0/663849294 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff490056400 0x7ff4900588c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7ff4a40034f0 tx=0x7ff4a4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:33.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:33.814+0000 7ff4a27fc700 1 -- 172.21.15.203:0/663849294 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff4a80445b0 con 0x7ff4b40b9b80 2021-01-06T22:17:34.042 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:34.038+0000 7ff4b359e700 1 -- 172.21.15.203:0/663849294 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4b405ca60 con 0x7ff4b40b9b80 2021-01-06T22:17:34.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:34.038+0000 7ff4a27fc700 1 -- 172.21.15.203:0/663849294 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+3564 (secure 0 0 0) 0x7ff4a8047bd0 con 0x7ff4b40b9b80 2021-01-06T22:17:34.043 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:17:34.043 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5632,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:55.638183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:17:34.045 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:34.042+0000 7ff4b359e700 1 -- 172.21.15.203:0/663849294 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff490056400 msgr2=0x7ff4900588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:34.045 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:34.042+0000 7ff4b359e700 1 --2- 172.21.15.203:0/663849294 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff490056400 0x7ff4900588c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7ff4a40034f0 tx=0x7ff4a4009040).stop 2021-01-06T22:17:34.045 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:34.042+0000 7ff4b359e700 1 -- 172.21.15.203:0/663849294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b40b9b80 msgr2=0x7ff4b40a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:34.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:34.042+0000 7ff4b359e700 1 --2- 172.21.15.203:0/663849294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b40b9b80 0x7ff4b40a1090 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7ff4a800c770 tx=0x7ff4a8007c40).stop 2021-01-06T22:17:34.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:34.042+0000 7ff4b359e700 1 -- 172.21.15.203:0/663849294 shutdown_connections 2021-01-06T22:17:34.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:34.042+0000 7ff4b359e700 1 --2- 172.21.15.203:0/663849294 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff490056400 0x7ff4900588c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:34.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:34.042+0000 7ff4b359e700 1 --2- 172.21.15.203:0/663849294 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff4b40bed80 0x7ff4b40a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:34.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:34.042+0000 7ff4b359e700 1 --2- 172.21.15.203:0/663849294 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4b40b97a0 0x7ff4b415bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:34.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:34.042+0000 7ff4b359e700 1 --2- 172.21.15.203:0/663849294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b40b9b80 0x7ff4b40a1090 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:34.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:34.042+0000 7ff4b359e700 1 -- 172.21.15.203:0/663849294 >> 172.21.15.203:0/663849294 conn(0x7ff4b40648f0 msgr2=0x7ff4b40c21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:34.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:34.042+0000 7ff4b359e700 1 -- 172.21.15.203:0/663849294 shutdown_connections 2021-01-06T22:17:34.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:34.042+0000 7ff4b359e700 1 -- 172.21.15.203:0/663849294 wait complete. 2021-01-06T22:17:34.048 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 68 2021-01-06T22:17:34.059 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:17:36.061 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:17:36.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.193+0000 7ff62d00d700 1 Processor -- start 2021-01-06T22:17:36.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.193+0000 7ff62d00d700 1 -- start start 2021-01-06T22:17:36.199 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.193+0000 7ff62d00d700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff62814c110 0x7ff6280657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:36.200 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.193+0000 7ff62d00d700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62814ee80 con 0x7ff6280be350 2021-01-06T22:17:36.200 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.193+0000 7ff62d00d700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62814f120 con 0x7ff6280be730 2021-01-06T22:17:36.200 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.193+0000 7ff62d00d700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62814f3c0 con 0x7ff62814c110 2021-01-06T22:17:36.201 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.193+0000 7ff6277fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7ff6280be730 0x7ff62814ba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51084/0 (socket says 172.21.15.203:51084) 2021-01-06T22:17:36.201 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.193+0000 7ff6277fe700 1 -- 172.21.15.203:0/3744253147 learned_addr learned my addr 172.21.15.203:0/3744253147 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:17:36.201 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.193+0000 7ff62c80c700 1 --2- 172.21.15.203:0/3744253147 >> v2:172.21.15.91:3300/0 conn(0x7ff62814c110 0x7ff6280657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:36.202 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.193+0000 7ff626ffd700 1 -- 172.21.15.203:0/3744253147 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475040078 0 0) 0x7ff62814f120 con 0x7ff6280be730 2021-01-06T22:17:36.202 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.193+0000 7ff626ffd700 1 -- 172.21.15.203:0/3744253147 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff614003230 con 0x7ff6280be730 2021-01-06T22:17:36.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.197+0000 7ff62c80c700 1 -- 172.21.15.203:0/3744253147 >> v1:172.21.15.203:6789/0 conn(0x7ff6280be730 legacy=0x7ff62814ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:36.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.197+0000 7ff626ffd700 1 -- 172.21.15.203:0/3744253147 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3309725516 0 0) 0x7ff614003230 con 0x7ff6280be730 2021-01-06T22:17:36.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.197+0000 7ff62c80c700 1 -- 172.21.15.203:0/3744253147 >> v1:172.21.15.91:6789/0 conn(0x7ff6280be350 legacy=0x7ff6280ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:36.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.197+0000 7ff62c80c700 1 -- 172.21.15.203:0/3744253147 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62814f660 con 0x7ff62814c110 2021-01-06T22:17:36.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.197+0000 7ff62c80c700 1 --2- 172.21.15.203:0/3744253147 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff62814c110 0x7ff6280657c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7ff620009c10 tx=0x7ff6200091d0).ready entity=mon.0 client_cookie=31d3042fe2247c48 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:36.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.197+0000 7ff626ffd700 1 -- 172.21.15.203:0/3744253147 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c0014e0 con 0x7ff62814c110 2021-01-06T22:17:36.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.197+0000 7ff626ffd700 1 -- 172.21.15.203:0/3744253147 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62000b790 con 0x7ff62814c110 2021-01-06T22:17:36.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.197+0000 7ff626ffd700 1 -- 172.21.15.203:0/3744253147 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62000bb10 con 0x7ff62814c110 2021-01-06T22:17:36.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.197+0000 7ff62d00d700 1 -- 172.21.15.203:0/3744253147 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff62814c110 msgr2=0x7ff6280657c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:36.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.197+0000 7ff62d00d700 1 --2- 172.21.15.203:0/3744253147 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff62814c110 0x7ff6280657c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7ff620009c10 tx=0x7ff6200091d0).stop 2021-01-06T22:17:36.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.197+0000 7ff62d00d700 1 -- 172.21.15.203:0/3744253147 shutdown_connections 2021-01-06T22:17:36.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.197+0000 7ff62d00d700 1 --2- 172.21.15.203:0/3744253147 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff62814c110 0x7ff6280657c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:36.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.197+0000 7ff62d00d700 1 -- 172.21.15.203:0/3744253147 >> 172.21.15.203:0/3744253147 conn(0x7ff628067e20 msgr2=0x7ff628068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:36.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.197+0000 7ff62d00d700 1 -- 172.21.15.203:0/3744253147 shutdown_connections 2021-01-06T22:17:36.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.197+0000 7ff62d00d700 1 -- 172.21.15.203:0/3744253147 wait complete. 2021-01-06T22:17:36.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff62d00d700 1 Processor -- start 2021-01-06T22:17:36.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff62d00d700 1 -- start start 2021-01-06T22:17:36.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff62d00d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6280be350 0x7ff62815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:36.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff62d00d700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6280be730 0x7ff6280c18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:36.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff62d00d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff62814c110 0x7ff628064f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:36.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff62d00d700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62815f200 con 0x7ff6280be730 2021-01-06T22:17:36.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff62d00d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280bf950 con 0x7ff6280be350 2021-01-06T22:17:36.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff62d00d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280c1da0 con 0x7ff62814c110 2021-01-06T22:17:36.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff627fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6280be350 0x7ff62815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:36.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff627fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6280be350 0x7ff62815ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52494/0 (socket says 172.21.15.203:52494) 2021-01-06T22:17:36.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff627fff700 1 -- 172.21.15.203:0/3068790541 learned_addr learned my addr 172.21.15.203:0/3068790541 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:36.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff6277fe700 1 --2- 172.21.15.203:0/3068790541 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6280be730 0x7ff6280c18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:36.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff62c80c700 1 --2- 172.21.15.203:0/3068790541 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff62814c110 0x7ff628064f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:36.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff627fff700 1 -- 172.21.15.203:0/3068790541 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff62814c110 msgr2=0x7ff628064f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:36.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff627fff700 1 --2- 172.21.15.203:0/3068790541 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff62814c110 0x7ff628064f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:36.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff627fff700 1 -- 172.21.15.203:0/3068790541 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6280be730 msgr2=0x7ff6280c18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:36.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff627fff700 1 --2- 172.21.15.203:0/3068790541 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6280be730 0x7ff6280c18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:36.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff627fff700 1 -- 172.21.15.203:0/3068790541 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6280bfb80 con 0x7ff6280be350 2021-01-06T22:17:36.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff627fff700 1 --2- 172.21.15.203:0/3068790541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6280be350 0x7ff62815ed10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7ff61800c710 tx=0x7ff618007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:36.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff60ffff700 1 -- 172.21.15.203:0/3068790541 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618007020 con 0x7ff6280be350 2021-01-06T22:17:36.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff60ffff700 1 -- 172.21.15.203:0/3068790541 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff618007180 con 0x7ff6280be350 2021-01-06T22:17:36.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff62d00d700 1 -- 172.21.15.203:0/3068790541 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6280bfe20 con 0x7ff6280be350 2021-01-06T22:17:36.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff60ffff700 1 -- 172.21.15.203:0/3068790541 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61800e560 con 0x7ff6280be350 2021-01-06T22:17:36.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.201+0000 7ff62d00d700 1 -- 172.21.15.203:0/3068790541 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62815ae70 con 0x7ff6280be350 2021-01-06T22:17:36.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.205+0000 7ff62d00d700 1 -- 172.21.15.203:0/3068790541 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6280450a0 con 0x7ff6280be350 2021-01-06T22:17:36.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.205+0000 7ff60ffff700 1 -- 172.21.15.203:0/3068790541 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff618018020 con 0x7ff6280be350 2021-01-06T22:17:36.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.205+0000 7ff60ffff700 1 --2- 172.21.15.203:0/3068790541 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6080563d0 0x7ff608058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:36.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.205+0000 7ff60ffff700 1 -- 172.21.15.203:0/3068790541 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6761+0+0 (secure 0 0 0) 0x7ff61806f170 con 0x7ff6280be350 2021-01-06T22:17:36.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.205+0000 7ff6277fe700 1 --2- 172.21.15.203:0/3068790541 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6080563d0 0x7ff608058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:36.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.205+0000 7ff6277fe700 1 --2- 172.21.15.203:0/3068790541 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6080563d0 0x7ff608058890 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7ff628060a60 tx=0x7ff61c000b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:36.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.209+0000 7ff60ffff700 1 -- 172.21.15.203:0/3068790541 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff618044740 con 0x7ff6280be350 2021-01-06T22:17:36.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.453+0000 7ff62d00d700 1 -- 172.21.15.203:0/3068790541 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff62814ab10 con 0x7ff6280be350 2021-01-06T22:17:36.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.453+0000 7ff60ffff700 1 -- 172.21.15.203:0/3068790541 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+3564 (secure 0 0 0) 0x7ff618016030 con 0x7ff6280be350 2021-01-06T22:17:36.459 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:17:36.459 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5632,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:55.638183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:17:36.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.457+0000 7ff62d00d700 1 -- 172.21.15.203:0/3068790541 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6080563d0 msgr2=0x7ff608058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:36.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.457+0000 7ff62d00d700 1 --2- 172.21.15.203:0/3068790541 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6080563d0 0x7ff608058890 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7ff628060a60 tx=0x7ff61c000b80).stop 2021-01-06T22:17:36.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.457+0000 7ff62d00d700 1 -- 172.21.15.203:0/3068790541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6280be350 msgr2=0x7ff62815ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:36.462 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.457+0000 7ff62d00d700 1 --2- 172.21.15.203:0/3068790541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6280be350 0x7ff62815ed10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7ff61800c710 tx=0x7ff618007c40).stop 2021-01-06T22:17:36.462 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.457+0000 7ff62d00d700 1 -- 172.21.15.203:0/3068790541 shutdown_connections 2021-01-06T22:17:36.462 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.457+0000 7ff62d00d700 1 --2- 172.21.15.203:0/3068790541 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff6080563d0 0x7ff608058890 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:36.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.457+0000 7ff62d00d700 1 --2- 172.21.15.203:0/3068790541 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff62814c110 0x7ff628064f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:36.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.457+0000 7ff62d00d700 1 --2- 172.21.15.203:0/3068790541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6280be350 0x7ff62815ed10 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:36.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.457+0000 7ff62d00d700 1 --2- 172.21.15.203:0/3068790541 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6280be730 0x7ff6280c18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:36.463 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.457+0000 7ff62d00d700 1 -- 172.21.15.203:0/3068790541 >> 172.21.15.203:0/3068790541 conn(0x7ff628067e20 msgr2=0x7ff6280ba580 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:36.464 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.457+0000 7ff62d00d700 1 -- 172.21.15.203:0/3068790541 shutdown_connections 2021-01-06T22:17:36.464 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:36.457+0000 7ff62d00d700 1 -- 172.21.15.203:0/3068790541 wait complete. 2021-01-06T22:17:36.464 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 68 2021-01-06T22:17:36.477 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:17:38.478 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:17:38.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f12035ec700 1 Processor -- start 2021-01-06T22:17:38.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f12035ec700 1 -- start start 2021-01-06T22:17:38.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f12035ec700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f11fc0bf100 0x7f11fc0bf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:38.607 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f12035ec700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f11fc064bc0 0x7f11fc14df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:38.607 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f12035ec700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f11fc151560 0x7f11fc147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:38.608 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f12035ec700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc152460 con 0x7f11fc151560 2021-01-06T22:17:38.608 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f12035ec700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc14e740 con 0x7f11fc0bf100 2021-01-06T22:17:38.608 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f12035ec700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc14e9e0 con 0x7f11fc064bc0 2021-01-06T22:17:38.608 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f1201388700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f11fc0bf100 0x7f11fc0bf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:38.609 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f1201388700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f11fc0bf100 0x7f11fc0bf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52506/0 (socket says 172.21.15.203:52506) 2021-01-06T22:17:38.609 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f1201388700 1 -- 172.21.15.203:0/1715749574 learned_addr learned my addr 172.21.15.203:0/1715749574 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:38.609 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f1200b87700 1 --2- 172.21.15.203:0/1715749574 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f11fc064bc0 0x7f11fc14df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:38.609 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f1201b89700 1 --2- 172.21.15.203:0/1715749574 >> v2:172.21.15.91:3300/0 conn(0x7f11fc151560 0x7f11fc147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:38.610 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f1201388700 1 -- 172.21.15.203:0/1715749574 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f11fc064bc0 msgr2=0x7f11fc14df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:38.610 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f1201388700 1 --2- 172.21.15.203:0/1715749574 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f11fc064bc0 0x7f11fc14df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:38.610 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f1201388700 1 -- 172.21.15.203:0/1715749574 >> v2:172.21.15.91:3300/0 conn(0x7f11fc151560 msgr2=0x7f11fc147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:38.610 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f1201388700 1 --2- 172.21.15.203:0/1715749574 >> v2:172.21.15.91:3300/0 conn(0x7f11fc151560 0x7f11fc147bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:38.610 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f1201388700 1 -- 172.21.15.203:0/1715749574 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc14ec80 con 0x7f11fc0bf100 2021-01-06T22:17:38.611 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f1200b87700 1 --2- 172.21.15.203:0/1715749574 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f11fc064bc0 0x7f11fc14df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:17:38.611 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f1201388700 1 --2- 172.21.15.203:0/1715749574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11fc0bf100 0x7f11fc0bf4e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f11fc068760 tx=0x7f11f8006ea0).ready entity=mon.1 client_cookie=a3e676704f3c41af server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:38.614 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f11ebfff700 1 -- 172.21.15.203:0/1715749574 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f8009810 con 0x7f11fc0bf100 2021-01-06T22:17:38.614 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f11ebfff700 1 -- 172.21.15.203:0/1715749574 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f8009970 con 0x7f11fc0bf100 2021-01-06T22:17:38.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.601+0000 7f11ebfff700 1 -- 172.21.15.203:0/1715749574 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f8009cf0 con 0x7f11fc0bf100 2021-01-06T22:17:38.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.605+0000 7f12035ec700 1 -- 172.21.15.203:0/1715749574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11fc0bf100 msgr2=0x7f11fc0bf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:38.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.605+0000 7f12035ec700 1 --2- 172.21.15.203:0/1715749574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11fc0bf100 0x7f11fc0bf4e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f11fc068760 tx=0x7f11f8006ea0).stop 2021-01-06T22:17:38.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.605+0000 7f12035ec700 1 -- 172.21.15.203:0/1715749574 shutdown_connections 2021-01-06T22:17:38.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.605+0000 7f12035ec700 1 --2- 172.21.15.203:0/1715749574 >> v2:172.21.15.91:3300/0 conn(0x7f11fc151560 0x7f11fc147bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:38.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.605+0000 7f12035ec700 1 --2- 172.21.15.203:0/1715749574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11fc0bf100 0x7f11fc0bf4e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:38.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.605+0000 7f12035ec700 1 --2- 172.21.15.203:0/1715749574 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f11fc064bc0 0x7f11fc14df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:38.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.605+0000 7f12035ec700 1 -- 172.21.15.203:0/1715749574 >> 172.21.15.203:0/1715749574 conn(0x7f11fc0602b0 msgr2=0x7f11fc0606c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:38.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.605+0000 7f12035ec700 1 -- 172.21.15.203:0/1715749574 shutdown_connections 2021-01-06T22:17:38.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.605+0000 7f12035ec700 1 -- 172.21.15.203:0/1715749574 wait complete. 2021-01-06T22:17:38.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.605+0000 7f12035ec700 1 Processor -- start 2021-01-06T22:17:38.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.605+0000 7f12035ec700 1 -- start start 2021-01-06T22:17:38.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.605+0000 7f12035ec700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11fc064bc0 0x7f11fc14e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:38.618 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.605+0000 7f12035ec700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f11fc0bf100 0x7f11fc15e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:38.618 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f12035ec700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f11fc151560 0x7f11fc143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:38.618 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f12035ec700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc14b260 con 0x7f11fc151560 2021-01-06T22:17:38.618 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f12035ec700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0c26c0 con 0x7f11fc064bc0 2021-01-06T22:17:38.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f12035ec700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0b3510 con 0x7f11fc0bf100 2021-01-06T22:17:38.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f1201388700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11fc064bc0 0x7f11fc14e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:38.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f1201388700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11fc064bc0 0x7f11fc14e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52514/0 (socket says 172.21.15.203:52514) 2021-01-06T22:17:38.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f1201388700 1 -- 172.21.15.203:0/240964379 learned_addr learned my addr 172.21.15.203:0/240964379 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:38.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f1200b87700 1 --2- 172.21.15.203:0/240964379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f11fc0bf100 0x7f11fc15e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:38.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f1201b89700 1 --2- 172.21.15.203:0/240964379 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f11fc151560 0x7f11fc143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:38.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f1201388700 1 -- 172.21.15.203:0/240964379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f11fc0bf100 msgr2=0x7f11fc15e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:38.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f1201388700 1 --2- 172.21.15.203:0/240964379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f11fc0bf100 0x7f11fc15e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:38.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f1201388700 1 -- 172.21.15.203:0/240964379 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f11fc151560 msgr2=0x7f11fc143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:38.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f1201388700 1 --2- 172.21.15.203:0/240964379 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f11fc151560 0x7f11fc143de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:38.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f1201388700 1 -- 172.21.15.203:0/240964379 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc0b37b0 con 0x7f11fc064bc0 2021-01-06T22:17:38.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f1201388700 1 --2- 172.21.15.203:0/240964379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11fc064bc0 0x7f11fc14e0f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f11f8006ee0 tx=0x7f11f8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:38.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f11e97fa700 1 -- 172.21.15.203:0/240964379 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f8012d60 con 0x7f11fc064bc0 2021-01-06T22:17:38.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f11e97fa700 1 -- 172.21.15.203:0/240964379 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f801b410 con 0x7f11fc064bc0 2021-01-06T22:17:38.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f12035ec700 1 -- 172.21.15.203:0/240964379 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc0b3a50 con 0x7f11fc064bc0 2021-01-06T22:17:38.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f11e97fa700 1 -- 172.21.15.203:0/240964379 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f801b790 con 0x7f11fc064bc0 2021-01-06T22:17:38.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f12035ec700 1 -- 172.21.15.203:0/240964379 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc15da40 con 0x7f11fc064bc0 2021-01-06T22:17:38.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f11e97fa700 1 -- 172.21.15.203:0/240964379 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f11f802a070 con 0x7f11fc064bc0 2021-01-06T22:17:38.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f11e97fa700 1 --2- 172.21.15.203:0/240964379 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f11dc0563d0 0x7f11dc058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:38.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f12035ec700 1 -- 172.21.15.203:0/240964379 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d4004f60 con 0x7f11fc064bc0 2021-01-06T22:17:38.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.609+0000 7f11e97fa700 1 -- 172.21.15.203:0/240964379 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f11f8070be0 con 0x7f11fc064bc0 2021-01-06T22:17:38.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.613+0000 7f1200b87700 1 --2- 172.21.15.203:0/240964379 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f11dc0563d0 0x7f11dc058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:38.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.613+0000 7f1200b87700 1 --2- 172.21.15.203:0/240964379 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f11dc0563d0 0x7f11dc058890 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f11ec006fd0 tx=0x7f11ec008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:38.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.617+0000 7f11e97fa700 1 -- 172.21.15.203:0/240964379 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f11f80461b0 con 0x7f11fc064bc0 2021-01-06T22:17:38.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.853+0000 7f12035ec700 1 -- 172.21.15.203:0/240964379 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11d4005790 con 0x7f11fc064bc0 2021-01-06T22:17:38.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.857+0000 7f11e97fa700 1 -- 172.21.15.203:0/240964379 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+3564 (secure 0 0 0) 0x7f11f80497d0 con 0x7f11fc064bc0 2021-01-06T22:17:38.860 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:17:38.860 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5632,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:55.638183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:17:38.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.857+0000 7f12035ec700 1 -- 172.21.15.203:0/240964379 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f11dc0563d0 msgr2=0x7f11dc058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:38.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.857+0000 7f12035ec700 1 --2- 172.21.15.203:0/240964379 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f11dc0563d0 0x7f11dc058890 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f11ec006fd0 tx=0x7f11ec008040).stop 2021-01-06T22:17:38.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.857+0000 7f12035ec700 1 -- 172.21.15.203:0/240964379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11fc064bc0 msgr2=0x7f11fc14e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:38.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.857+0000 7f12035ec700 1 --2- 172.21.15.203:0/240964379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11fc064bc0 0x7f11fc14e0f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f11f8006ee0 tx=0x7f11f8016000).stop 2021-01-06T22:17:38.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.857+0000 7f12035ec700 1 -- 172.21.15.203:0/240964379 shutdown_connections 2021-01-06T22:17:38.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.857+0000 7f12035ec700 1 --2- 172.21.15.203:0/240964379 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f11dc0563d0 0x7f11dc058890 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:38.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.857+0000 7f12035ec700 1 --2- 172.21.15.203:0/240964379 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f11fc151560 0x7f11fc143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:38.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.857+0000 7f12035ec700 1 --2- 172.21.15.203:0/240964379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11fc064bc0 0x7f11fc14e0f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:38.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.857+0000 7f12035ec700 1 --2- 172.21.15.203:0/240964379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f11fc0bf100 0x7f11fc15e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:38.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.857+0000 7f12035ec700 1 -- 172.21.15.203:0/240964379 >> 172.21.15.203:0/240964379 conn(0x7f11fc0602b0 msgr2=0x7f11fc0c2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:38.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.857+0000 7f12035ec700 1 -- 172.21.15.203:0/240964379 shutdown_connections 2021-01-06T22:17:38.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:38.857+0000 7f12035ec700 1 -- 172.21.15.203:0/240964379 wait complete. 2021-01-06T22:17:38.865 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 68 2021-01-06T22:17:38.878 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:17:40.879 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:17:41.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:40.997+0000 7f054541c700 1 Processor -- start 2021-01-06T22:17:41.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:40.997+0000 7f054541c700 1 -- start start 2021-01-06T22:17:41.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f054541c700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f05400bbb70 0x7f0540068440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:41.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f054541c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0540057060 0x7f054014c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:41.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f054541c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0540152c60 con 0x7f05400bb790 2021-01-06T22:17:41.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f054541c700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05400bc2f0 con 0x7f05400bbb70 2021-01-06T22:17:41.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f054541c700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05400b4580 con 0x7f0540057060 2021-01-06T22:17:41.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053effd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f05400bb790 0x7f0540152550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51116/0 (socket says 172.21.15.203:51116) 2021-01-06T22:17:41.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053effd700 1 -- 172.21.15.203:0/4028692231 learned_addr learned my addr 172.21.15.203:0/4028692231 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:17:41.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 -- 172.21.15.203:0/4028692231 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270667391 0 0) 0x7f0540152c60 con 0x7f05400bb790 2021-01-06T22:17:41.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 -- 172.21.15.203:0/4028692231 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f052c003230 con 0x7f05400bb790 2021-01-06T22:17:41.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053e7fc700 1 --2- 172.21.15.203:0/4028692231 >> v2:172.21.15.91:3300/0 conn(0x7f05400bbb70 0x7f0540068440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:41.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 -- 172.21.15.203:0/4028692231 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 208297871 0 0) 0x7f052c003230 con 0x7f05400bb790 2021-01-06T22:17:41.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053f7fe700 1 --2- 172.21.15.203:0/4028692231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0540057060 0x7f054014c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:41.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 -- 172.21.15.203:0/4028692231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0540057060 msgr2=0x7f054014c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:41.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 --2- 172.21.15.203:0/4028692231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0540057060 0x7f054014c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:41.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 -- 172.21.15.203:0/4028692231 >> v2:172.21.15.91:3300/0 conn(0x7f05400bbb70 msgr2=0x7f0540068440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:41.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 --2- 172.21.15.203:0/4028692231 >> v2:172.21.15.91:3300/0 conn(0x7f05400bbb70 0x7f0540068440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:41.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 -- 172.21.15.203:0/4028692231 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05400b4820 con 0x7f05400bb790 2021-01-06T22:17:41.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 -- 172.21.15.203:0/4028692231 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f05300014e0 con 0x7f05400bb790 2021-01-06T22:17:41.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 -- 172.21.15.203:0/4028692231 >> v1:172.21.15.203:6789/0 conn(0x7f05400bb790 legacy=0x7f0540152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:41.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 --2- 172.21.15.203:0/4028692231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0540057060 0x7f052c003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:41.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 --2- 172.21.15.203:0/4028692231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c003c30 0x7f052c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:41.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 --2- 172.21.15.203:0/4028692231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f052c0049c0 0x7f052c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:41.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 -- 172.21.15.203:0/4028692231 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05400bc2f0 con 0x7f052c0049c0 2021-01-06T22:17:41.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 -- 172.21.15.203:0/4028692231 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05400b4580 con 0x7f052c003c30 2021-01-06T22:17:41.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 -- 172.21.15.203:0/4028692231 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0041f0 con 0x7f0540057060 2021-01-06T22:17:41.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053e7fc700 1 --2- 172.21.15.203:0/4028692231 >> v2:172.21.15.91:3300/0 conn(0x7f05400bbb70 0x7f0540068440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:17:41.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053e7fc700 1 --2- 172.21.15.203:0/4028692231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c003c30 0x7f052c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:41.021 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053effd700 1 --2- 172.21.15.203:0/4028692231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0540057060 0x7f052c003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:41.021 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053f7fe700 1 --2- 172.21.15.203:0/4028692231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f052c0049c0 0x7f052c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:41.021 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053e7fc700 1 -- 172.21.15.203:0/4028692231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0540057060 msgr2=0x7f052c003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:41.021 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053e7fc700 1 --2- 172.21.15.203:0/4028692231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0540057060 0x7f052c003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:41.022 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053e7fc700 1 -- 172.21.15.203:0/4028692231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f052c0049c0 msgr2=0x7f052c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:41.022 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053e7fc700 1 --2- 172.21.15.203:0/4028692231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f052c0049c0 0x7f052c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:41.022 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053e7fc700 1 -- 172.21.15.203:0/4028692231 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c003230 con 0x7f052c003c30 2021-01-06T22:17:41.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053e7fc700 1 --2- 172.21.15.203:0/4028692231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c003c30 0x7f052c0044d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f053400aa90 tx=0x7f0534008040).ready entity=mon.1 client_cookie=bdbaae67df3871de server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:41.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 -- 172.21.15.203:0/4028692231 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053400a160 con 0x7f052c003c30 2021-01-06T22:17:41.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 -- 172.21.15.203:0/4028692231 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053400c9f0 con 0x7f052c003c30 2021-01-06T22:17:41.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053dffb700 1 -- 172.21.15.203:0/4028692231 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053400cd70 con 0x7f052c003c30 2021-01-06T22:17:41.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f054541c700 1 -- 172.21.15.203:0/4028692231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c003c30 msgr2=0x7f052c0044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:41.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f054541c700 1 --2- 172.21.15.203:0/4028692231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c003c30 0x7f052c0044d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f053400aa90 tx=0x7f0534008040).stop 2021-01-06T22:17:41.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f053f7fe700 1 -- 172.21.15.203:0/4028692231 reap_dead start 2021-01-06T22:17:41.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f054541c700 1 -- 172.21.15.203:0/4028692231 shutdown_connections 2021-01-06T22:17:41.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f054541c700 1 -- 172.21.15.203:0/4028692231 >> 172.21.15.203:0/4028692231 conn(0x7f05400c0c50 msgr2=0x7f05400c1030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:41.026 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f054541c700 1 -- 172.21.15.203:0/4028692231 shutdown_connections 2021-01-06T22:17:41.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.001+0000 7f054541c700 1 -- 172.21.15.203:0/4028692231 wait complete. 2021-01-06T22:17:41.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f054541c700 1 Processor -- start 2021-01-06T22:17:41.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f054541c700 1 -- start start 2021-01-06T22:17:41.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f054541c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c003c30 0x7f054014d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:41.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f054541c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f05401508f0 0x7f0540165b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:41.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f054541c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0540151ea0 0x7f0540061380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:41.029 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f054541c700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05401523e0 con 0x7f0540151ea0 2021-01-06T22:17:41.029 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f054541c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0540152680 con 0x7f052c003c30 2021-01-06T22:17:41.029 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f054541c700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0540152920 con 0x7f05401508f0 2021-01-06T22:17:41.029 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f053effd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c003c30 0x7f054014d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:41.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f053effd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c003c30 0x7f054014d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52534/0 (socket says 172.21.15.203:52534) 2021-01-06T22:17:41.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f053effd700 1 -- 172.21.15.203:0/1922647269 learned_addr learned my addr 172.21.15.203:0/1922647269 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:41.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f053e7fc700 1 --2- 172.21.15.203:0/1922647269 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f05401508f0 0x7f0540165b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:41.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f053f7fe700 1 --2- 172.21.15.203:0/1922647269 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0540151ea0 0x7f0540061380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:41.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f053effd700 1 -- 172.21.15.203:0/1922647269 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f05401508f0 msgr2=0x7f0540165b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:41.031 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f053effd700 1 --2- 172.21.15.203:0/1922647269 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f05401508f0 0x7f0540165b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:41.031 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f053effd700 1 -- 172.21.15.203:0/1922647269 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0540151ea0 msgr2=0x7f0540061380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:41.031 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f053effd700 1 --2- 172.21.15.203:0/1922647269 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0540151ea0 0x7f0540061380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:41.031 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f053effd700 1 -- 172.21.15.203:0/1922647269 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0540152bc0 con 0x7f052c003c30 2021-01-06T22:17:41.032 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f053effd700 1 --2- 172.21.15.203:0/1922647269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c003c30 0x7f054014d2d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f0530000f90 tx=0x7f05300014b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:41.032 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f0526ffd700 1 -- 172.21.15.203:0/1922647269 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053000c7e0 con 0x7f052c003c30 2021-01-06T22:17:41.032 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f0526ffd700 1 -- 172.21.15.203:0/1922647269 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053000c940 con 0x7f052c003c30 2021-01-06T22:17:41.032 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f054541c700 1 -- 172.21.15.203:0/1922647269 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05401653c0 con 0x7f052c003c30 2021-01-06T22:17:41.033 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f0526ffd700 1 -- 172.21.15.203:0/1922647269 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053000cce0 con 0x7f052c003c30 2021-01-06T22:17:41.033 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.005+0000 7f054541c700 1 -- 172.21.15.203:0/1922647269 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054014d7c0 con 0x7f052c003c30 2021-01-06T22:17:41.034 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.009+0000 7f054541c700 1 -- 172.21.15.203:0/1922647269 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054005d600 con 0x7f052c003c30 2021-01-06T22:17:41.034 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.009+0000 7f0526ffd700 1 -- 172.21.15.203:0/1922647269 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f0530014640 con 0x7f052c003c30 2021-01-06T22:17:41.035 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.009+0000 7f0526ffd700 1 --2- 172.21.15.203:0/1922647269 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0520056400 0x7f05200588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:41.035 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.009+0000 7f0526ffd700 1 -- 172.21.15.203:0/1922647269 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f053006f3d0 con 0x7f052c003c30 2021-01-06T22:17:41.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.009+0000 7f053e7fc700 1 --2- 172.21.15.203:0/1922647269 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0520056400 0x7f05200588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:41.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.009+0000 7f053e7fc700 1 --2- 172.21.15.203:0/1922647269 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0520056400 0x7f05200588c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f053400c6f0 tx=0x7f0534008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:41.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.013+0000 7f0526ffd700 1 -- 172.21.15.203:0/1922647269 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f0530044940 con 0x7f052c003c30 2021-01-06T22:17:41.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.233+0000 7f054541c700 1 -- 172.21.15.203:0/1922647269 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05400646f0 con 0x7f052c003c30 2021-01-06T22:17:41.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.233+0000 7f0526ffd700 1 -- 172.21.15.203:0/1922647269 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+3564 (secure 0 0 0) 0x7f0530047f60 con 0x7f052c003c30 2021-01-06T22:17:41.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.237+0000 7f054541c700 1 -- 172.21.15.203:0/1922647269 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0520056400 msgr2=0x7f05200588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:41.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.237+0000 7f054541c700 1 --2- 172.21.15.203:0/1922647269 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0520056400 0x7f05200588c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f053400c6f0 tx=0x7f0534008040).stop 2021-01-06T22:17:41.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.237+0000 7f054541c700 1 -- 172.21.15.203:0/1922647269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c003c30 msgr2=0x7f054014d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:41.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.237+0000 7f054541c700 1 --2- 172.21.15.203:0/1922647269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c003c30 0x7f054014d2d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f0530000f90 tx=0x7f05300014b0).stop 2021-01-06T22:17:41.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.237+0000 7f054541c700 1 -- 172.21.15.203:0/1922647269 shutdown_connections 2021-01-06T22:17:41.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.237+0000 7f054541c700 1 --2- 172.21.15.203:0/1922647269 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0520056400 0x7f05200588c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:41.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.237+0000 7f054541c700 1 --2- 172.21.15.203:0/1922647269 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0540151ea0 0x7f0540061380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:41.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.237+0000 7f054541c700 1 --2- 172.21.15.203:0/1922647269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f052c003c30 0x7f054014d2d0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:41.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.237+0000 7f054541c700 1 --2- 172.21.15.203:0/1922647269 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f05401508f0 0x7f0540165b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:41.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.237+0000 7f054541c700 1 -- 172.21.15.203:0/1922647269 >> 172.21.15.203:0/1922647269 conn(0x7f05400c0c50 msgr2=0x7f05400c1030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:41.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.237+0000 7f054541c700 1 -- 172.21.15.203:0/1922647269 shutdown_connections 2021-01-06T22:17:41.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:41.237+0000 7f054541c700 1 -- 172.21.15.203:0/1922647269 wait complete. 2021-01-06T22:17:41.248 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 68 2021-01-06T22:17:41.249 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:17:41.249 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5632,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:55.638183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:17:41.256 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:17:43.258 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:17:43.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac9e70700 1 Processor -- start 2021-01-06T22:17:43.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac9e70700 1 -- start start 2021-01-06T22:17:43.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac9e70700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faac4062880 0x7faac414b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:43.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac9e70700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faac4061470 0x7faac40bb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:43.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac9e70700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac40bbae0 con 0x7faac4062c60 2021-01-06T22:17:43.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac9e70700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac414bd50 con 0x7faac4062880 2021-01-06T22:17:43.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac9e70700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac40602b0 con 0x7faac4061470 2021-01-06T22:17:43.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac2ffd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7faac4062c60 0x7faac4147e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51142/0 (socket says 172.21.15.203:51142) 2021-01-06T22:17:43.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac2ffd700 1 -- 172.21.15.203:0/3255549849 learned_addr learned my addr 172.21.15.203:0/3255549849 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:17:43.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac37fe700 1 --2- 172.21.15.203:0/3255549849 >> v2:172.21.15.91:3300/0 conn(0x7faac4062880 0x7faac414b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:43.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac3fff700 1 --2- 172.21.15.203:0/3255549849 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faac4061470 0x7faac40bb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:43.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac27fc700 1 -- 172.21.15.203:0/3255549849 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614859749 0 0) 0x7faac40bbae0 con 0x7faac4062c60 2021-01-06T22:17:43.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac27fc700 1 -- 172.21.15.203:0/3255549849 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faab0003230 con 0x7faac4062c60 2021-01-06T22:17:43.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac27fc700 1 -- 172.21.15.203:0/3255549849 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2764635007 0 0) 0x7faab0003230 con 0x7faac4062c60 2021-01-06T22:17:43.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac27fc700 1 -- 172.21.15.203:0/3255549849 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faac4061470 msgr2=0x7faac40bb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:43.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac27fc700 1 --2- 172.21.15.203:0/3255549849 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faac4061470 0x7faac40bb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:43.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac27fc700 1 -- 172.21.15.203:0/3255549849 >> v2:172.21.15.91:3300/0 conn(0x7faac4062880 msgr2=0x7faac414b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:43.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac27fc700 1 --2- 172.21.15.203:0/3255549849 >> v2:172.21.15.91:3300/0 conn(0x7faac4062880 0x7faac414b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:43.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac27fc700 1 -- 172.21.15.203:0/3255549849 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac4060550 con 0x7faac4062c60 2021-01-06T22:17:43.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac27fc700 1 -- 172.21.15.203:0/3255549849 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7faab40014e0 con 0x7faac4062c60 2021-01-06T22:17:43.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac27fc700 1 -- 172.21.15.203:0/3255549849 >> v1:172.21.15.203:6789/0 conn(0x7faac4062c60 legacy=0x7faac4147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:43.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac27fc700 1 --2- 172.21.15.203:0/3255549849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab0003720 0x7faab0003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:43.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac27fc700 1 --2- 172.21.15.203:0/3255549849 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faab00044d0 0x7faab0004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:43.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac27fc700 1 --2- 172.21.15.203:0/3255549849 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faac4061470 0x7faab0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:43.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac27fc700 1 -- 172.21.15.203:0/3255549849 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac40602b0 con 0x7faab00044d0 2021-01-06T22:17:43.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac27fc700 1 -- 172.21.15.203:0/3255549849 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab0005380 con 0x7faab0003720 2021-01-06T22:17:43.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac27fc700 1 -- 172.21.15.203:0/3255549849 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab00042e0 con 0x7faac4061470 2021-01-06T22:17:43.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.385+0000 7faac37fe700 1 --2- 172.21.15.203:0/3255549849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab0003720 0x7faab0003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:43.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac3fff700 1 --2- 172.21.15.203:0/3255549849 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faac4061470 0x7faab0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:43.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac2ffd700 1 --2- 172.21.15.203:0/3255549849 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faab00044d0 0x7faab0004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:43.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac37fe700 1 -- 172.21.15.203:0/3255549849 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faac4061470 msgr2=0x7faab0004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:43.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac37fe700 1 --2- 172.21.15.203:0/3255549849 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faac4061470 0x7faab0004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:43.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac37fe700 1 -- 172.21.15.203:0/3255549849 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faab00044d0 msgr2=0x7faab0004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:43.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac37fe700 1 --2- 172.21.15.203:0/3255549849 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faab00044d0 0x7faab0004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:43.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac37fe700 1 -- 172.21.15.203:0/3255549849 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab0003230 con 0x7faab0003720 2021-01-06T22:17:43.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac37fe700 1 --2- 172.21.15.203:0/3255549849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab0003720 0x7faab0003b40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7faab800bb90 tx=0x7faab8008040).ready entity=mon.1 client_cookie=a75642c0831a85fb server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:43.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac27fc700 1 -- 172.21.15.203:0/3255549849 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab800b2b0 con 0x7faab0003720 2021-01-06T22:17:43.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac27fc700 1 -- 172.21.15.203:0/3255549849 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab800ea30 con 0x7faab0003720 2021-01-06T22:17:43.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac27fc700 1 -- 172.21.15.203:0/3255549849 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab800edd0 con 0x7faab0003720 2021-01-06T22:17:43.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac9e70700 1 -- 172.21.15.203:0/3255549849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab0003720 msgr2=0x7faab0003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:43.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac9e70700 1 --2- 172.21.15.203:0/3255549849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab0003720 0x7faab0003b40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7faab800bb90 tx=0x7faab8008040).stop 2021-01-06T22:17:43.406 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac3fff700 1 -- 172.21.15.203:0/3255549849 reap_dead start 2021-01-06T22:17:43.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac9e70700 1 -- 172.21.15.203:0/3255549849 shutdown_connections 2021-01-06T22:17:43.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac9e70700 1 -- 172.21.15.203:0/3255549849 >> 172.21.15.203:0/3255549849 conn(0x7faac40c0c50 msgr2=0x7faac40c1030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:43.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac9e70700 1 -- 172.21.15.203:0/3255549849 shutdown_connections 2021-01-06T22:17:43.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac9e70700 1 -- 172.21.15.203:0/3255549849 wait complete. 2021-01-06T22:17:43.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac9e70700 1 Processor -- start 2021-01-06T22:17:43.408 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.389+0000 7faac9e70700 1 -- start start 2021-01-06T22:17:43.408 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac9e70700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faac413dce0 0x7faac4136a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:43.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac9e70700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faac413e0c0 0x7faac4136ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:43.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac9e70700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faac40b9da0 0x7faac4146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:43.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac9e70700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac41383b0 con 0x7faac413e0c0 2021-01-06T22:17:43.409 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac9e70700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac40ba580 con 0x7faac40b9da0 2021-01-06T22:17:43.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac9e70700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac40ba2e0 con 0x7faac413dce0 2021-01-06T22:17:43.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faac40b9da0 0x7faac4146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:43.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faac40b9da0 0x7faac4146b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52562/0 (socket says 172.21.15.203:52562) 2021-01-06T22:17:43.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac3fff700 1 -- 172.21.15.203:0/4045904175 learned_addr learned my addr 172.21.15.203:0/4045904175 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:43.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac2ffd700 1 --2- 172.21.15.203:0/4045904175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faac413e0c0 0x7faac4136ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:43.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac37fe700 1 --2- 172.21.15.203:0/4045904175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faac413dce0 0x7faac4136a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:43.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac3fff700 1 -- 172.21.15.203:0/4045904175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faac413dce0 msgr2=0x7faac4136a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:43.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac3fff700 1 --2- 172.21.15.203:0/4045904175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faac413dce0 0x7faac4136a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:43.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac3fff700 1 -- 172.21.15.203:0/4045904175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faac413e0c0 msgr2=0x7faac4136ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:43.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac3fff700 1 --2- 172.21.15.203:0/4045904175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faac413e0c0 0x7faac4136ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:43.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac3fff700 1 -- 172.21.15.203:0/4045904175 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac415de50 con 0x7faac40b9da0 2021-01-06T22:17:43.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac3fff700 1 --2- 172.21.15.203:0/4045904175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faac40b9da0 0x7faac4146b30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7faac4063040 tx=0x7faabc008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:43.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faaab7fe700 1 -- 172.21.15.203:0/4045904175 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc00e8a0 con 0x7faac40b9da0 2021-01-06T22:17:43.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faaab7fe700 1 -- 172.21.15.203:0/4045904175 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faabc00ea00 con 0x7faac40b9da0 2021-01-06T22:17:43.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac9e70700 1 -- 172.21.15.203:0/4045904175 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac415e0f0 con 0x7faac40b9da0 2021-01-06T22:17:43.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faaab7fe700 1 -- 172.21.15.203:0/4045904175 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc00edc0 con 0x7faac40b9da0 2021-01-06T22:17:43.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac9e70700 1 -- 172.21.15.203:0/4045904175 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac415e630 con 0x7faac40b9da0 2021-01-06T22:17:43.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.393+0000 7faac9e70700 1 -- 172.21.15.203:0/4045904175 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac415e7f0 con 0x7faac40b9da0 2021-01-06T22:17:43.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.397+0000 7faaab7fe700 1 -- 172.21.15.203:0/4045904175 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7faabc014680 con 0x7faac40b9da0 2021-01-06T22:17:43.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.397+0000 7faaab7fe700 1 --2- 172.21.15.203:0/4045904175 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faaa4056400 0x7faaa40588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:43.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.397+0000 7faaab7fe700 1 -- 172.21.15.203:0/4045904175 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6761+0+0 (secure 0 0 0) 0x7faabc06ff60 con 0x7faac40b9da0 2021-01-06T22:17:43.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.397+0000 7faac37fe700 1 --2- 172.21.15.203:0/4045904175 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faaa4056400 0x7faaa40588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:43.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.397+0000 7faac37fe700 1 --2- 172.21.15.203:0/4045904175 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faaa4056400 0x7faaa40588c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7faab8006cf0 tx=0x7faab801a000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:43.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.401+0000 7faaab7fe700 1 -- 172.21.15.203:0/4045904175 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7faabc045530 con 0x7faac40b9da0 2021-01-06T22:17:43.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.633+0000 7faac9e70700 1 -- 172.21.15.203:0/4045904175 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faac4142170 con 0x7faac40b9da0 2021-01-06T22:17:43.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.637+0000 7faaab7fe700 1 -- 172.21.15.203:0/4045904175 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+3564 (secure 0 0 0) 0x7faabc048b50 con 0x7faac40b9da0 2021-01-06T22:17:43.640 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:17:43.640 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5632,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:55.638183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:17:43.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.637+0000 7faac9e70700 1 -- 172.21.15.203:0/4045904175 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faaa4056400 msgr2=0x7faaa40588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:43.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.637+0000 7faac9e70700 1 --2- 172.21.15.203:0/4045904175 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faaa4056400 0x7faaa40588c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7faab8006cf0 tx=0x7faab801a000).stop 2021-01-06T22:17:43.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.637+0000 7faac9e70700 1 -- 172.21.15.203:0/4045904175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faac40b9da0 msgr2=0x7faac4146b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:43.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.637+0000 7faac9e70700 1 --2- 172.21.15.203:0/4045904175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faac40b9da0 0x7faac4146b30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7faac4063040 tx=0x7faabc008040).stop 2021-01-06T22:17:43.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.637+0000 7faac9e70700 1 -- 172.21.15.203:0/4045904175 shutdown_connections 2021-01-06T22:17:43.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.637+0000 7faac9e70700 1 --2- 172.21.15.203:0/4045904175 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7faaa4056400 0x7faaa40588c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:43.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.637+0000 7faac9e70700 1 --2- 172.21.15.203:0/4045904175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faac40b9da0 0x7faac4146b30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:43.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.637+0000 7faac9e70700 1 --2- 172.21.15.203:0/4045904175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faac413dce0 0x7faac4136a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:43.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.637+0000 7faac9e70700 1 --2- 172.21.15.203:0/4045904175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faac413e0c0 0x7faac4136ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:43.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.637+0000 7faac9e70700 1 -- 172.21.15.203:0/4045904175 >> 172.21.15.203:0/4045904175 conn(0x7faac40c0c50 msgr2=0x7faac413eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:43.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.637+0000 7faac9e70700 1 -- 172.21.15.203:0/4045904175 shutdown_connections 2021-01-06T22:17:43.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:43.637+0000 7faac9e70700 1 -- 172.21.15.203:0/4045904175 wait complete. 2021-01-06T22:17:43.645 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 68 2021-01-06T22:17:43.660 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:17:45.662 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:17:45.788 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa12b0ca700 1 Processor -- start 2021-01-06T22:17:45.788 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa12b0ca700 1 -- start start 2021-01-06T22:17:45.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa12b0ca700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1240c04c0 0x7fa124066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:45.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa12b0ca700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa1240c08a0 0x7fa124147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:45.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa12b0ca700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1240bb860 con 0x7fa1241481b0 2021-01-06T22:17:45.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa12b0ca700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12414b940 con 0x7fa1240c08a0 2021-01-06T22:17:45.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa12b0ca700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1240bba90 con 0x7fa1240c04c0 2021-01-06T22:17:45.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa11bfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa1240c08a0 0x7fa124147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:45.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa11bfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa1240c08a0 0x7fa124147cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52576/0 (socket says 172.21.15.203:52576) 2021-01-06T22:17:45.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa11bfff700 1 -- 172.21.15.203:0/2455381198 learned_addr learned my addr 172.21.15.203:0/2455381198 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:45.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa128e66700 1 --2- 172.21.15.203:0/2455381198 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1240c04c0 0x7fa124066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:45.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa11bfff700 1 -- 172.21.15.203:0/2455381198 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1240c04c0 msgr2=0x7fa124066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:45.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa11bfff700 1 --2- 172.21.15.203:0/2455381198 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1240c04c0 0x7fa124066cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:45.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa11bfff700 1 -- 172.21.15.203:0/2455381198 >> v1:172.21.15.91:6789/0 conn(0x7fa1241481b0 legacy=0x7fa12414ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:45.794 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa11bfff700 1 -- 172.21.15.203:0/2455381198 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1240bbd30 con 0x7fa1240c08a0 2021-01-06T22:17:45.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa128e66700 1 --2- 172.21.15.203:0/2455381198 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1240c04c0 0x7fa124066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:17:45.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa11bfff700 1 --2- 172.21.15.203:0/2455381198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1240c08a0 0x7fa124147cc0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fa11400ba10 tx=0x7fa114006ea0).ready entity=mon.1 client_cookie=2f0593e5724be45f server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:45.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa11b7fe700 1 -- 172.21.15.203:0/2455381198 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114009890 con 0x7fa1240c08a0 2021-01-06T22:17:45.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa11b7fe700 1 -- 172.21.15.203:0/2455381198 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1140099f0 con 0x7fa1240c08a0 2021-01-06T22:17:45.795 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa11b7fe700 1 -- 172.21.15.203:0/2455381198 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114009d70 con 0x7fa1240c08a0 2021-01-06T22:17:45.796 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa12b0ca700 1 -- 172.21.15.203:0/2455381198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1240c08a0 msgr2=0x7fa124147cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:45.796 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa12b0ca700 1 --2- 172.21.15.203:0/2455381198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1240c08a0 0x7fa124147cc0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fa11400ba10 tx=0x7fa114006ea0).stop 2021-01-06T22:17:45.796 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa12b0ca700 1 -- 172.21.15.203:0/2455381198 shutdown_connections 2021-01-06T22:17:45.796 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa12b0ca700 1 --2- 172.21.15.203:0/2455381198 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1240c04c0 0x7fa124066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:45.797 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa12b0ca700 1 --2- 172.21.15.203:0/2455381198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1240c08a0 0x7fa124147cc0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:45.797 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa12b0ca700 1 -- 172.21.15.203:0/2455381198 >> 172.21.15.203:0/2455381198 conn(0x7fa124056cd0 msgr2=0x7fa1240626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:45.797 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa12b0ca700 1 -- 172.21.15.203:0/2455381198 shutdown_connections 2021-01-06T22:17:45.797 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.785+0000 7fa12b0ca700 1 -- 172.21.15.203:0/2455381198 wait complete. 2021-01-06T22:17:45.798 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa12b0ca700 1 Processor -- start 2021-01-06T22:17:45.798 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa12b0ca700 1 -- start start 2021-01-06T22:17:45.798 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa12b0ca700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1240c04c0 0x7fa1240bec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:45.798 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa12b0ca700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1240c08a0 0x7fa124146420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:45.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa12b0ca700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1241481b0 0x7fa124136f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:45.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa12b0ca700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1240bf0f0 con 0x7fa1240c04c0 2021-01-06T22:17:45.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa12b0ca700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1241610a0 con 0x7fa1241481b0 2021-01-06T22:17:45.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa12b0ca700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa124161340 con 0x7fa1240c08a0 2021-01-06T22:17:45.799 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa129667700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1241481b0 0x7fa124136f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:45.800 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa129667700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1241481b0 0x7fa124136f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52584/0 (socket says 172.21.15.203:52584) 2021-01-06T22:17:45.800 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa129667700 1 -- 172.21.15.203:0/3683376908 learned_addr learned my addr 172.21.15.203:0/3683376908 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:45.800 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa128e66700 1 --2- 172.21.15.203:0/3683376908 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1240c04c0 0x7fa1240bec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:45.801 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa11bfff700 1 --2- 172.21.15.203:0/3683376908 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1240c08a0 0x7fa124146420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:45.804 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa129667700 1 -- 172.21.15.203:0/3683376908 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1240c08a0 msgr2=0x7fa124146420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:45.804 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa129667700 1 --2- 172.21.15.203:0/3683376908 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1240c08a0 0x7fa124146420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:45.804 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa129667700 1 -- 172.21.15.203:0/3683376908 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1240c04c0 msgr2=0x7fa1240bec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:45.805 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa129667700 1 --2- 172.21.15.203:0/3683376908 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1240c04c0 0x7fa1240bec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:45.805 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa129667700 1 -- 172.21.15.203:0/3683376908 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1241615e0 con 0x7fa1241481b0 2021-01-06T22:17:45.805 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa129667700 1 --2- 172.21.15.203:0/3683376908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1241481b0 0x7fa124136f90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7fa11c00ca90 tx=0x7fa11c007f90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:45.805 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa118ff9700 1 -- 172.21.15.203:0/3683376908 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00ce60 con 0x7fa1241481b0 2021-01-06T22:17:45.805 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa118ff9700 1 -- 172.21.15.203:0/3683376908 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11c00a7c0 con 0x7fa1241481b0 2021-01-06T22:17:45.806 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa118ff9700 1 -- 172.21.15.203:0/3683376908 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00ab60 con 0x7fa1241481b0 2021-01-06T22:17:45.806 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa12b0ca700 1 -- 172.21.15.203:0/3683376908 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa124161880 con 0x7fa1241481b0 2021-01-06T22:17:45.806 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.789+0000 7fa12b0ca700 1 -- 172.21.15.203:0/3683376908 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa124136af0 con 0x7fa1241481b0 2021-01-06T22:17:45.806 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.793+0000 7fa12b0ca700 1 -- 172.21.15.203:0/3683376908 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa124136cf0 con 0x7fa1241481b0 2021-01-06T22:17:45.807 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.793+0000 7fa118ff9700 1 -- 172.21.15.203:0/3683376908 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa11c019030 con 0x7fa1241481b0 2021-01-06T22:17:45.807 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.793+0000 7fa118ff9700 1 --2- 172.21.15.203:0/3683376908 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa104056400 0x7fa1040588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:45.807 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.793+0000 7fa118ff9700 1 -- 172.21.15.203:0/3683376908 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6761+0+0 (secure 0 0 0) 0x7fa11c06f000 con 0x7fa1241481b0 2021-01-06T22:17:45.807 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.793+0000 7fa128e66700 1 --2- 172.21.15.203:0/3683376908 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa104056400 0x7fa1040588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:45.807 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.793+0000 7fa128e66700 1 --2- 172.21.15.203:0/3683376908 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa104056400 0x7fa1040588c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fa12000aa10 tx=0x7fa120008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:45.808 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:45.797+0000 7fa118ff9700 1 -- 172.21.15.203:0/3683376908 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa124136cf0 con 0x7fa1241481b0 2021-01-06T22:17:46.031 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:46.025+0000 7fa12b0ca700 1 -- 172.21.15.203:0/3683376908 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa124136cf0 con 0x7fa1241481b0 2021-01-06T22:17:46.033 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:46.029+0000 7fa118ff9700 1 -- 172.21.15.203:0/3683376908 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+3564 (secure 0 0 0) 0x7fa124136cf0 con 0x7fa1241481b0 2021-01-06T22:17:46.033 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:17:46.033 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5632,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:16:55.638183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5364},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5364":{"gid":5364,"name":"f","rank":1,"incarnation":57,"state":"up:active","state_seq":32,"addr":"172.21.15.91:6837/617463325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":617463325},{"type":"v1","addr":"172.21.15.91:6837","nonce":617463325}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:17:46.035 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:46.029+0000 7fa12b0ca700 1 -- 172.21.15.203:0/3683376908 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa104056400 msgr2=0x7fa1040588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:46.035 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:46.029+0000 7fa12b0ca700 1 --2- 172.21.15.203:0/3683376908 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa104056400 0x7fa1040588c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fa12000aa10 tx=0x7fa120008040).stop 2021-01-06T22:17:46.035 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:46.029+0000 7fa12b0ca700 1 -- 172.21.15.203:0/3683376908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1241481b0 msgr2=0x7fa124136f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:46.035 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:46.029+0000 7fa12b0ca700 1 --2- 172.21.15.203:0/3683376908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1241481b0 0x7fa124136f90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7fa11c00ca90 tx=0x7fa11c007f90).stop 2021-01-06T22:17:46.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:46.029+0000 7fa12b0ca700 1 -- 172.21.15.203:0/3683376908 shutdown_connections 2021-01-06T22:17:46.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:46.029+0000 7fa12b0ca700 1 --2- 172.21.15.203:0/3683376908 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa104056400 0x7fa1040588c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:46.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:46.029+0000 7fa12b0ca700 1 --2- 172.21.15.203:0/3683376908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1241481b0 0x7fa124136f90 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:46.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:46.029+0000 7fa12b0ca700 1 --2- 172.21.15.203:0/3683376908 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1240c04c0 0x7fa1240bec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:46.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:46.029+0000 7fa12b0ca700 1 --2- 172.21.15.203:0/3683376908 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1240c08a0 0x7fa124146420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:46.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:46.029+0000 7fa12b0ca700 1 -- 172.21.15.203:0/3683376908 >> 172.21.15.203:0/3683376908 conn(0x7fa124056cd0 msgr2=0x7fa124147a90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:46.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:46.029+0000 7fa12b0ca700 1 -- 172.21.15.203:0/3683376908 shutdown_connections 2021-01-06T22:17:46.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:46.029+0000 7fa12b0ca700 1 -- 172.21.15.203:0/3683376908 wait complete. 2021-01-06T22:17:46.037 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 68 2021-01-06T22:17:46.049 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2021-01-06T22:17:47.965 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:17:47.968 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:17:48.049 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:17:48.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.177+0000 7fa02b59e700 1 Processor -- start 2021-01-06T22:17:48.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.177+0000 7fa02b59e700 1 -- start start 2021-01-06T22:17:48.191 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02b59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa02c064380 0x7fa02c0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:48.191 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02b59e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa02c14e310 0x7fa02c1471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:48.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02b59e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa02c064760 0x7fa02c1476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:48.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02b59e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa02c147b90 con 0x7fa02c064760 2021-01-06T22:17:48.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02b59e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa02c0bb2d0 con 0x7fa02c14e310 2021-01-06T22:17:48.192 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02b59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c14a560 con 0x7fa02c064380 2021-01-06T22:17:48.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa021d9b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa02c14e310 0x7fa02c1471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:48.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa021d9b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa02c14e310 0x7fa02c1471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52604/0 (socket says 172.21.15.203:52604) 2021-01-06T22:17:48.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa021d9b700 1 -- 172.21.15.203:0/1846490167 learned_addr learned my addr 172.21.15.203:0/1846490167 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:48.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02ad9d700 1 --2- 172.21.15.203:0/1846490167 >> v2:172.21.15.91:3300/0 conn(0x7fa02c064760 0x7fa02c1476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:48.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02a59c700 1 --2- 172.21.15.203:0/1846490167 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa02c064380 0x7fa02c0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:48.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa021d9b700 1 -- 172.21.15.203:0/1846490167 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa02c064380 msgr2=0x7fa02c0c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:48.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa021d9b700 1 --2- 172.21.15.203:0/1846490167 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa02c064380 0x7fa02c0c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa021d9b700 1 -- 172.21.15.203:0/1846490167 >> v2:172.21.15.91:3300/0 conn(0x7fa02c064760 msgr2=0x7fa02c1476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:48.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa021d9b700 1 --2- 172.21.15.203:0/1846490167 >> v2:172.21.15.91:3300/0 conn(0x7fa02c064760 0x7fa02c1476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa021d9b700 1 -- 172.21.15.203:0/1846490167 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02c14a800 con 0x7fa02c14e310 2021-01-06T22:17:48.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa021d9b700 1 --2- 172.21.15.203:0/1846490167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa02c14e310 0x7fa02c1471b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fa02c066e10 tx=0x7fa014006ea0).ready entity=mon.1 client_cookie=3d4d63df527aff49 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:48.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa029d9b700 1 -- 172.21.15.203:0/1846490167 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014009960 con 0x7fa02c14e310 2021-01-06T22:17:48.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa029d9b700 1 -- 172.21.15.203:0/1846490167 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa014009ac0 con 0x7fa02c14e310 2021-01-06T22:17:48.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa029d9b700 1 -- 172.21.15.203:0/1846490167 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014009e40 con 0x7fa02c14e310 2021-01-06T22:17:48.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02b59e700 1 -- 172.21.15.203:0/1846490167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa02c14e310 msgr2=0x7fa02c1471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:48.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02b59e700 1 --2- 172.21.15.203:0/1846490167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa02c14e310 0x7fa02c1471b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fa02c066e10 tx=0x7fa014006ea0).stop 2021-01-06T22:17:48.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02b59e700 1 -- 172.21.15.203:0/1846490167 shutdown_connections 2021-01-06T22:17:48.198 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02b59e700 1 --2- 172.21.15.203:0/1846490167 >> v2:172.21.15.91:3300/0 conn(0x7fa02c064760 0x7fa02c1476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02b59e700 1 --2- 172.21.15.203:0/1846490167 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa02c064380 0x7fa02c0c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02b59e700 1 --2- 172.21.15.203:0/1846490167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa02c14e310 0x7fa02c1471b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02b59e700 1 -- 172.21.15.203:0/1846490167 >> 172.21.15.203:0/1846490167 conn(0x7fa02c0667b0 msgr2=0x7fa02c0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:48.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02b59e700 1 -- 172.21.15.203:0/1846490167 shutdown_connections 2021-01-06T22:17:48.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.181+0000 7fa02b59e700 1 -- 172.21.15.203:0/1846490167 wait complete. 2021-01-06T22:17:48.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02b59e700 1 Processor -- start 2021-01-06T22:17:48.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02b59e700 1 -- start start 2021-01-06T22:17:48.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02b59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa02c064380 0x7fa02c0be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:48.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02b59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa02c064760 0x7fa02c0b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:48.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02b59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa02c14e310 0x7fa02c0b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:48.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02b59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c146c30 con 0x7fa02c064760 2021-01-06T22:17:48.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02b59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c0bf750 con 0x7fa02c14e310 2021-01-06T22:17:48.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02b59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c0ba470 con 0x7fa02c064380 2021-01-06T22:17:48.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02ad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa02c14e310 0x7fa02c0b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:48.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02ad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa02c14e310 0x7fa02c0b8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52612/0 (socket says 172.21.15.203:52612) 2021-01-06T22:17:48.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02ad9d700 1 -- 172.21.15.203:0/546184919 learned_addr learned my addr 172.21.15.203:0/546184919 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:48.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02a59c700 1 --2- 172.21.15.203:0/546184919 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa02c064380 0x7fa02c0be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:48.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa021d9b700 1 --2- 172.21.15.203:0/546184919 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa02c064760 0x7fa02c0b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:48.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02ad9d700 1 -- 172.21.15.203:0/546184919 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa02c064380 msgr2=0x7fa02c0be8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:48.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02ad9d700 1 --2- 172.21.15.203:0/546184919 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa02c064380 0x7fa02c0be8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02ad9d700 1 -- 172.21.15.203:0/546184919 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa02c064760 msgr2=0x7fa02c0b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:48.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02ad9d700 1 --2- 172.21.15.203:0/546184919 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa02c064760 0x7fa02c0b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02ad9d700 1 -- 172.21.15.203:0/546184919 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02c0ba710 con 0x7fa02c14e310 2021-01-06T22:17:48.211 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02ad9d700 1 --2- 172.21.15.203:0/546184919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa02c14e310 0x7fa02c0b8ed0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fa024006fd0 tx=0x7fa024008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:48.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa022ffd700 1 -- 172.21.15.203:0/546184919 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02400c6a0 con 0x7fa02c14e310 2021-01-06T22:17:48.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa022ffd700 1 -- 172.21.15.203:0/546184919 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02400c800 con 0x7fa02c14e310 2021-01-06T22:17:48.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02b59e700 1 -- 172.21.15.203:0/546184919 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa02c145d10 con 0x7fa02c14e310 2021-01-06T22:17:48.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa022ffd700 1 -- 172.21.15.203:0/546184919 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02400cb80 con 0x7fa02c14e310 2021-01-06T22:17:48.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.185+0000 7fa02b59e700 1 -- 172.21.15.203:0/546184919 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02c146250 con 0x7fa02c14e310 2021-01-06T22:17:48.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.189+0000 7fa02b59e700 1 -- 172.21.15.203:0/546184919 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02c0450a0 con 0x7fa02c14e310 2021-01-06T22:17:48.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.189+0000 7fa022ffd700 1 -- 172.21.15.203:0/546184919 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fa02401d460 con 0x7fa02c14e310 2021-01-06T22:17:48.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.189+0000 7fa022ffd700 1 --2- 172.21.15.203:0/546184919 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa0100563d0 0x7fa010058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:48.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.189+0000 7fa022ffd700 1 -- 172.21.15.203:0/546184919 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fa02406f800 con 0x7fa02c14e310 2021-01-06T22:17:48.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.189+0000 7fa02a59c700 1 --2- 172.21.15.203:0/546184919 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa0100563d0 0x7fa010058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:48.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.189+0000 7fa02a59c700 1 --2- 172.21.15.203:0/546184919 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa0100563d0 0x7fa010058890 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fa02c144dd0 tx=0x7fa01c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:48.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.197+0000 7fa022ffd700 1 -- 172.21.15.203:0/546184919 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fa024044d20 con 0x7fa02c14e310 2021-01-06T22:17:48.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.413+0000 7fa02b59e700 1 -- 172.21.15.203:0/546184919 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa02c05ca60 con 0x7fa02c14e310 2021-01-06T22:17:48.417 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.413+0000 7fa022ffd700 1 -- 172.21.15.203:0/546184919 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+3215 (secure 0 0 0) 0x7fa024048340 con 0x7fa02c14e310 2021-01-06T22:17:48.419 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:17:48.419 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:17:47.611364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:resolve","state_seq":13,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:17:48.420 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.417+0000 7fa02b59e700 1 -- 172.21.15.203:0/546184919 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa0100563d0 msgr2=0x7fa010058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:48.420 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.417+0000 7fa02b59e700 1 --2- 172.21.15.203:0/546184919 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa0100563d0 0x7fa010058890 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fa02c144dd0 tx=0x7fa01c008040).stop 2021-01-06T22:17:48.420 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.417+0000 7fa02b59e700 1 -- 172.21.15.203:0/546184919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa02c14e310 msgr2=0x7fa02c0b8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:48.421 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.417+0000 7fa02b59e700 1 --2- 172.21.15.203:0/546184919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa02c14e310 0x7fa02c0b8ed0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fa024006fd0 tx=0x7fa024008040).stop 2021-01-06T22:17:48.421 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.417+0000 7fa02b59e700 1 -- 172.21.15.203:0/546184919 shutdown_connections 2021-01-06T22:17:48.421 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.417+0000 7fa02b59e700 1 --2- 172.21.15.203:0/546184919 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fa0100563d0 0x7fa010058890 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.421 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.417+0000 7fa02b59e700 1 --2- 172.21.15.203:0/546184919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa02c14e310 0x7fa02c0b8ed0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.417+0000 7fa02b59e700 1 --2- 172.21.15.203:0/546184919 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa02c064380 0x7fa02c0be8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.417+0000 7fa02b59e700 1 --2- 172.21.15.203:0/546184919 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa02c064760 0x7fa02c0b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.417+0000 7fa02b59e700 1 -- 172.21.15.203:0/546184919 >> 172.21.15.203:0/546184919 conn(0x7fa02c0667b0 msgr2=0x7fa02c146f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:48.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.417+0000 7fa02b59e700 1 -- 172.21.15.203:0/546184919 shutdown_connections 2021-01-06T22:17:48.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.417+0000 7fa02b59e700 1 -- 172.21.15.203:0/546184919 wait complete. 2021-01-06T22:17:48.423 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 70 2021-01-06T22:17:48.436 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2021-01-06T22:17:48.437 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2021-01-06T22:17:48.437 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:17:48.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.557+0000 7f894bfff700 1 Processor -- start 2021-01-06T22:17:48.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.557+0000 7f894bfff700 1 -- start start 2021-01-06T22:17:48.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f894bfff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f894c0633f0 0x7f894c14e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:48.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f894bfff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f894c0be1b0 0x7f894c147790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:48.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f894bfff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f894c0be990 con 0x7f894c063010 2021-01-06T22:17:48.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f894bfff700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f894c14ec60 con 0x7f894c0633f0 2021-01-06T22:17:48.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f894bfff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894c0be6f0 con 0x7f894c0be1b0 2021-01-06T22:17:48.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f894a7fc700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f894c0633f0 0x7f894c14e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:48.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f894b7fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f894c0be1b0 0x7f894c147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:48.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f894b7fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f894c0be1b0 0x7f894c147790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:57702/0 (socket says 172.21.15.203:57702) 2021-01-06T22:17:48.568 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f894a7fc700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f894c0633f0 0x7f894c14e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:60466/0 (socket says 172.21.15.203:60466) 2021-01-06T22:17:48.568 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f894b7fe700 1 -- 172.21.15.203:0/49877993 learned_addr learned my addr 172.21.15.203:0/49877993 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:48.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f894a7fc700 1 -- 172.21.15.203:0/49877993 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f894c0be1b0 msgr2=0x7f894c147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:48.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f894a7fc700 1 --2- 172.21.15.203:0/49877993 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f894c0be1b0 0x7f894c147790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f894a7fc700 1 -- 172.21.15.203:0/49877993 >> v1:172.21.15.91:6789/0 conn(0x7f894c063010 legacy=0x7f894c064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:48.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f894a7fc700 1 -- 172.21.15.203:0/49877993 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c0c1050 con 0x7f894c0633f0 2021-01-06T22:17:48.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f894a7fc700 1 --2- 172.21.15.203:0/49877993 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f894c0633f0 0x7f894c14e770 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f894000b980 tx=0x7f8940006de0).ready entity=mon.0 client_cookie=6acd503f6f6b66a7 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:48.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f8949ffb700 1 -- 172.21.15.203:0/49877993 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8940009840 con 0x7f894c0633f0 2021-01-06T22:17:48.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f8949ffb700 1 -- 172.21.15.203:0/49877993 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89400099a0 con 0x7f894c0633f0 2021-01-06T22:17:48.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.561+0000 7f8949ffb700 1 -- 172.21.15.203:0/49877993 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8940009d20 con 0x7f894c0633f0 2021-01-06T22:17:48.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894bfff700 1 -- 172.21.15.203:0/49877993 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f894c0633f0 msgr2=0x7f894c14e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:48.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894bfff700 1 --2- 172.21.15.203:0/49877993 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f894c0633f0 0x7f894c14e770 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f894000b980 tx=0x7f8940006de0).stop 2021-01-06T22:17:48.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894bfff700 1 -- 172.21.15.203:0/49877993 shutdown_connections 2021-01-06T22:17:48.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894bfff700 1 --2- 172.21.15.203:0/49877993 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f894c0be1b0 0x7f894c147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894bfff700 1 --2- 172.21.15.203:0/49877993 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f894c0633f0 0x7f894c14e770 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894bfff700 1 -- 172.21.15.203:0/49877993 >> 172.21.15.203:0/49877993 conn(0x7f894c067e20 msgr2=0x7f894c068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:48.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894bfff700 1 -- 172.21.15.203:0/49877993 shutdown_connections 2021-01-06T22:17:48.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894bfff700 1 -- 172.21.15.203:0/49877993 wait complete. 2021-01-06T22:17:48.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894bfff700 1 Processor -- start 2021-01-06T22:17:48.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894bfff700 1 -- start start 2021-01-06T22:17:48.576 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894bfff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f894c063010 0x7f894c13ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:48.577 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f894c0633f0 0x7f894c149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:48.577 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894bfff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f894c0be1b0 0x7f894c0bb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:48.577 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894bfff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c14a930 con 0x7f894c063010 2021-01-06T22:17:48.578 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894bfff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c14b750 con 0x7f894c0633f0 2021-01-06T22:17:48.578 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894bfff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894c14b4b0 con 0x7f894c0be1b0 2021-01-06T22:17:48.578 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894a7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f894c0633f0 0x7f894c149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:48.578 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894a7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f894c0633f0 0x7f894c149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52624/0 (socket says 172.21.15.203:52624) 2021-01-06T22:17:48.579 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894a7fc700 1 -- 172.21.15.203:0/3039018774 learned_addr learned my addr 172.21.15.203:0/3039018774 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:48.579 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894b7fe700 1 --2- 172.21.15.203:0/3039018774 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f894c0be1b0 0x7f894c0bb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:48.579 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.565+0000 7f894affd700 1 --2- 172.21.15.203:0/3039018774 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f894c063010 0x7f894c13ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:48.579 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.569+0000 7f894a7fc700 1 -- 172.21.15.203:0/3039018774 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f894c0be1b0 msgr2=0x7f894c0bb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:48.580 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.569+0000 7f894a7fc700 1 --2- 172.21.15.203:0/3039018774 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f894c0be1b0 0x7f894c0bb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.580 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.569+0000 7f894a7fc700 1 -- 172.21.15.203:0/3039018774 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f894c063010 msgr2=0x7f894c13ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:48.580 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.569+0000 7f894a7fc700 1 --2- 172.21.15.203:0/3039018774 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f894c063010 0x7f894c13ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.580 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.569+0000 7f894a7fc700 1 -- 172.21.15.203:0/3039018774 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c14ab90 con 0x7f894c0633f0 2021-01-06T22:17:48.580 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.569+0000 7f894a7fc700 1 --2- 172.21.15.203:0/3039018774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f894c0633f0 0x7f894c149040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f8940006e20 tx=0x7f8940016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:48.581 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.569+0000 7f8932ffd700 1 -- 172.21.15.203:0/3039018774 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8940012d40 con 0x7f894c0633f0 2021-01-06T22:17:48.581 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.569+0000 7f8932ffd700 1 -- 172.21.15.203:0/3039018774 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894001c420 con 0x7f894c0633f0 2021-01-06T22:17:48.581 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.569+0000 7f894bfff700 1 -- 172.21.15.203:0/3039018774 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894c14ae30 con 0x7f894c0633f0 2021-01-06T22:17:48.582 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.569+0000 7f8932ffd700 1 -- 172.21.15.203:0/3039018774 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8940012ea0 con 0x7f894c0633f0 2021-01-06T22:17:48.582 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.569+0000 7f894bfff700 1 -- 172.21.15.203:0/3039018774 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894c0c01b0 con 0x7f894c0633f0 2021-01-06T22:17:48.582 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.569+0000 7f894bfff700 1 -- 172.21.15.203:0/3039018774 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f894c0450a0 con 0x7f894c0633f0 2021-01-06T22:17:48.583 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.569+0000 7f8932ffd700 1 -- 172.21.15.203:0/3039018774 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f894000f070 con 0x7f894c0633f0 2021-01-06T22:17:48.583 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.569+0000 7f8932ffd700 1 --2- 172.21.15.203:0/3039018774 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f892c0563d0 0x7f892c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:48.583 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.569+0000 7f8932ffd700 1 -- 172.21.15.203:0/3039018774 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f8940070dd0 con 0x7f894c0633f0 2021-01-06T22:17:48.583 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.569+0000 7f894affd700 1 --2- 172.21.15.203:0/3039018774 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f892c0563d0 0x7f892c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:48.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.573+0000 7f894affd700 1 --2- 172.21.15.203:0/3039018774 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f892c0563d0 0x7f892c058890 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f893c0035f0 tx=0x7f893c007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:48.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.577+0000 7f8932ffd700 1 -- 172.21.15.203:0/3039018774 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f8940046350 con 0x7f894c0633f0 2021-01-06T22:17:48.805 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.801+0000 7f894bfff700 1 -- 172.21.15.203:0/3039018774 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f894c140c30 con 0x7f894c0633f0 2021-01-06T22:17:48.805 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.801+0000 7f8932ffd700 1 -- 172.21.15.203:0/3039018774 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+3217 (secure 0 0 0) 0x7f8940049970 con 0x7f894c0633f0 2021-01-06T22:17:48.806 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:17:48.806 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:17:48.619963+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:17:48.807 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.801+0000 7f894bfff700 1 -- 172.21.15.203:0/3039018774 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f892c0563d0 msgr2=0x7f892c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:48.808 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.801+0000 7f894bfff700 1 --2- 172.21.15.203:0/3039018774 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f892c0563d0 0x7f892c058890 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f893c0035f0 tx=0x7f893c007b80).stop 2021-01-06T22:17:48.808 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.801+0000 7f894bfff700 1 -- 172.21.15.203:0/3039018774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f894c0633f0 msgr2=0x7f894c149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:48.808 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.801+0000 7f894bfff700 1 --2- 172.21.15.203:0/3039018774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f894c0633f0 0x7f894c149040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f8940006e20 tx=0x7f8940016000).stop 2021-01-06T22:17:48.808 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.801+0000 7f894bfff700 1 -- 172.21.15.203:0/3039018774 shutdown_connections 2021-01-06T22:17:48.808 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.801+0000 7f894bfff700 1 --2- 172.21.15.203:0/3039018774 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f892c0563d0 0x7f892c058890 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.809 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.801+0000 7f894bfff700 1 --2- 172.21.15.203:0/3039018774 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f894c0be1b0 0x7f894c0bb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.809 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.801+0000 7f894bfff700 1 --2- 172.21.15.203:0/3039018774 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f894c063010 0x7f894c13ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.809 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.801+0000 7f894bfff700 1 --2- 172.21.15.203:0/3039018774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f894c0633f0 0x7f894c149040 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:48.809 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.801+0000 7f894bfff700 1 -- 172.21.15.203:0/3039018774 >> 172.21.15.203:0/3039018774 conn(0x7f894c067e20 msgr2=0x7f894c0beed0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:48.809 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.805+0000 7f894bfff700 1 -- 172.21.15.203:0/3039018774 shutdown_connections 2021-01-06T22:17:48.810 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:48.805+0000 7f894bfff700 1 -- 172.21.15.203:0/3039018774 wait complete. 2021-01-06T22:17:48.810 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 71 2021-01-06T22:17:48.819 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 71, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 71, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5539": { "addr": "172.21.15.203:6835/1801698156", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6834", "nonce": 1801698156, "type": "v2" }, { "addr": "172.21.15.203:6835", "nonce": 1801698156, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5539, "incarnation": 63, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 11 }, "gid_5632": { "addr": "172.21.15.203:6837/2424499644", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6836", "nonce": 2424499644, "type": "v2" }, { "addr": "172.21.15.203:6837", "nonce": 2424499644, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5632, "incarnation": 69, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:reconnect", "state_seq": 14 } }, "last_failure": 0, "last_failure_osd_epoch": 32, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2021-01-06T22:17:48.619963+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 5539, "mds_1": 5632 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.91:6836/2428882934", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6834", "nonce": 2428882934, "type": "v2" }, { "addr": "172.21.15.91:6836", "nonce": 2428882934, "type": "v1" } ] }, "epoch": 26, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4724, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2021-01-06T22:17:50.820 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:17:50.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.953+0000 7f04d7fff700 1 Processor -- start 2021-01-06T22:17:50.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.953+0000 7f04d7fff700 1 -- start start 2021-01-06T22:17:50.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d7fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f04d80c44f0 0x7f04d80c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:50.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d7fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04d8060fc0 0x7f04d8150e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:50.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d7fff700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d80c0f50 con 0x7f04d80c48d0 2021-01-06T22:17:50.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d7fff700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d8066cf0 con 0x7f04d80c44f0 2021-01-06T22:17:50.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d7fff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d8060750 con 0x7f04d8060fc0 2021-01-06T22:17:50.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d6ffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f04d80c44f0 0x7f04d80c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:50.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04cffff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f04d80c48d0 0x7f04d8150730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51228/0 (socket says 172.21.15.203:51228) 2021-01-06T22:17:50.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04cffff700 1 -- 172.21.15.203:0/3691224445 learned_addr learned my addr 172.21.15.203:0/3691224445 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:17:50.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d77fe700 1 --2- 172.21.15.203:0/3691224445 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04d8060fc0 0x7f04d8150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:50.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d6ffd700 1 -- 172.21.15.203:0/3691224445 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04d8060fc0 msgr2=0x7f04d8150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:50.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d67fc700 1 -- 172.21.15.203:0/3691224445 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459652320 0 0) 0x7f04d80c0f50 con 0x7f04d80c48d0 2021-01-06T22:17:50.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d6ffd700 1 --2- 172.21.15.203:0/3691224445 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04d8060fc0 0x7f04d8150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:50.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d6ffd700 1 -- 172.21.15.203:0/3691224445 >> v1:172.21.15.203:6789/0 conn(0x7f04d80c48d0 legacy=0x7f04d8150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:50.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d6ffd700 1 -- 172.21.15.203:0/3691224445 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d80609f0 con 0x7f04d80c44f0 2021-01-06T22:17:50.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d77fe700 1 --2- 172.21.15.203:0/3691224445 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04d8060fc0 0x7f04d8150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:17:50.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d6ffd700 1 --2- 172.21.15.203:0/3691224445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04d80c44f0 0x7f04d80c1380 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f04c800b980 tx=0x7f04c8006de0).ready entity=mon.1 client_cookie=d7ca1c154cce8c8d server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:50.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d67fc700 1 -- 172.21.15.203:0/3691224445 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c80097b0 con 0x7f04d80c44f0 2021-01-06T22:17:50.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d67fc700 1 -- 172.21.15.203:0/3691224445 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c8009910 con 0x7f04d80c44f0 2021-01-06T22:17:50.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d67fc700 1 -- 172.21.15.203:0/3691224445 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8009c90 con 0x7f04d80c44f0 2021-01-06T22:17:50.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d7fff700 1 -- 172.21.15.203:0/3691224445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04d80c44f0 msgr2=0x7f04d80c1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:50.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d7fff700 1 --2- 172.21.15.203:0/3691224445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04d80c44f0 0x7f04d80c1380 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f04c800b980 tx=0x7f04c8006de0).stop 2021-01-06T22:17:50.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d7fff700 1 -- 172.21.15.203:0/3691224445 shutdown_connections 2021-01-06T22:17:50.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d7fff700 1 --2- 172.21.15.203:0/3691224445 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04d8060fc0 0x7f04d8150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:50.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d7fff700 1 --2- 172.21.15.203:0/3691224445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04d80c44f0 0x7f04d80c1380 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:50.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d7fff700 1 -- 172.21.15.203:0/3691224445 >> 172.21.15.203:0/3691224445 conn(0x7f04d8056cd0 msgr2=0x7f04d8061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:50.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d7fff700 1 -- 172.21.15.203:0/3691224445 shutdown_connections 2021-01-06T22:17:50.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.957+0000 7f04d7fff700 1 -- 172.21.15.203:0/3691224445 wait complete. 2021-01-06T22:17:50.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d7fff700 1 Processor -- start 2021-01-06T22:17:50.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d7fff700 1 -- start start 2021-01-06T22:17:50.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d7fff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04d8060fc0 0x7f04d80c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:50.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d7fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04d80c44f0 0x7f04d815ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:50.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04d80c48d0 0x7f04d815f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:50.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d7fff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d815f610 con 0x7f04d8060fc0 2021-01-06T22:17:50.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d7fff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d80a40e0 con 0x7f04d80c48d0 2021-01-06T22:17:50.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d7fff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d80c3830 con 0x7f04d80c44f0 2021-01-06T22:17:50.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d77fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04d80c48d0 0x7f04d815f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:50.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d77fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04d80c48d0 0x7f04d815f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52642/0 (socket says 172.21.15.203:52642) 2021-01-06T22:17:50.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d77fe700 1 -- 172.21.15.203:0/3799264619 learned_addr learned my addr 172.21.15.203:0/3799264619 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:50.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d6ffd700 1 --2- 172.21.15.203:0/3799264619 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04d8060fc0 0x7f04d80c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:50.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04cffff700 1 --2- 172.21.15.203:0/3799264619 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04d80c44f0 0x7f04d815ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:50.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d77fe700 1 -- 172.21.15.203:0/3799264619 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04d80c44f0 msgr2=0x7f04d815ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:50.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d77fe700 1 --2- 172.21.15.203:0/3799264619 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04d80c44f0 0x7f04d815ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:50.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d77fe700 1 -- 172.21.15.203:0/3799264619 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04d8060fc0 msgr2=0x7f04d80c2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:50.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d77fe700 1 --2- 172.21.15.203:0/3799264619 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04d8060fc0 0x7f04d80c2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:50.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d77fe700 1 -- 172.21.15.203:0/3799264619 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d80c3ad0 con 0x7f04d80c48d0 2021-01-06T22:17:50.975 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.961+0000 7f04d77fe700 1 --2- 172.21.15.203:0/3799264619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04d80c48d0 0x7f04d815f120 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f04d000ebf0 tx=0x7f04d0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:50.975 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.965+0000 7f04ceffd700 1 -- 172.21.15.203:0/3799264619 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d000c8e0 con 0x7f04d80c48d0 2021-01-06T22:17:50.975 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.965+0000 7f04ceffd700 1 -- 172.21.15.203:0/3799264619 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d000ca40 con 0x7f04d80c48d0 2021-01-06T22:17:50.975 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.965+0000 7f04ceffd700 1 -- 172.21.15.203:0/3799264619 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d000cde0 con 0x7f04d80c48d0 2021-01-06T22:17:50.975 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.965+0000 7f04d7fff700 1 -- 172.21.15.203:0/3799264619 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d80a3900 con 0x7f04d80c48d0 2021-01-06T22:17:50.976 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.965+0000 7f04d7fff700 1 -- 172.21.15.203:0/3799264619 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d80a3e40 con 0x7f04d80c48d0 2021-01-06T22:17:50.976 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.965+0000 7f04d7fff700 1 -- 172.21.15.203:0/3799264619 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d815e970 con 0x7f04d80c48d0 2021-01-06T22:17:50.977 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.969+0000 7f04ceffd700 1 -- 172.21.15.203:0/3799264619 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f04d0013640 con 0x7f04d80c48d0 2021-01-06T22:17:50.977 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.969+0000 7f04ceffd700 1 --2- 172.21.15.203:0/3799264619 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f04bc056400 0x7f04bc0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:50.977 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.969+0000 7f04ceffd700 1 -- 172.21.15.203:0/3799264619 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f04d006fe50 con 0x7f04d80c48d0 2021-01-06T22:17:50.978 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.969+0000 7f04d6ffd700 1 --2- 172.21.15.203:0/3799264619 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f04bc056400 0x7f04bc0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:50.978 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.969+0000 7f04d6ffd700 1 --2- 172.21.15.203:0/3799264619 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f04bc056400 0x7f04bc0588c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f04c800b130 tx=0x7f04c8016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:50.978 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:50.973+0000 7f04ceffd700 1 -- 172.21.15.203:0/3799264619 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f04d00453f0 con 0x7f04d80c48d0 2021-01-06T22:17:51.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:51.185+0000 7f04d7fff700 1 -- 172.21.15.203:0/3799264619 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04d815e970 con 0x7f04d80c48d0 2021-01-06T22:17:51.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:51.185+0000 7f04ceffd700 1 -- 172.21.15.203:0/3799264619 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+3214 (secure 0 0 0) 0x7f04d815e970 con 0x7f04d80c48d0 2021-01-06T22:17:51.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:51.185+0000 7f04d7fff700 1 -- 172.21.15.203:0/3799264619 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f04bc056400 msgr2=0x7f04bc0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:51.193 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:51.185+0000 7f04d7fff700 1 --2- 172.21.15.203:0/3799264619 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f04bc056400 0x7f04bc0588c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f04c800b130 tx=0x7f04c8016000).stop 2021-01-06T22:17:51.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:51.185+0000 7f04d7fff700 1 -- 172.21.15.203:0/3799264619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04d80c48d0 msgr2=0x7f04d815f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:51.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:51.185+0000 7f04d7fff700 1 --2- 172.21.15.203:0/3799264619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04d80c48d0 0x7f04d815f120 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f04d000ebf0 tx=0x7f04d0008040).stop 2021-01-06T22:17:51.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:51.185+0000 7f04d7fff700 1 -- 172.21.15.203:0/3799264619 shutdown_connections 2021-01-06T22:17:51.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:51.185+0000 7f04d7fff700 1 --2- 172.21.15.203:0/3799264619 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f04bc056400 0x7f04bc0588c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:51.194 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:51.185+0000 7f04d7fff700 1 --2- 172.21.15.203:0/3799264619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04d80c48d0 0x7f04d815f120 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:51.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:51.185+0000 7f04d7fff700 1 --2- 172.21.15.203:0/3799264619 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04d8060fc0 0x7f04d80c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:51.195 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:51.185+0000 7f04d7fff700 1 --2- 172.21.15.203:0/3799264619 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04d80c44f0 0x7f04d815ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:51.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:51.185+0000 7f04d7fff700 1 -- 172.21.15.203:0/3799264619 >> 172.21.15.203:0/3799264619 conn(0x7f04d8056cd0 msgr2=0x7f04d80bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:51.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:51.185+0000 7f04d7fff700 1 -- 172.21.15.203:0/3799264619 shutdown_connections 2021-01-06T22:17:51.197 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:51.185+0000 7f04d7fff700 1 -- 172.21.15.203:0/3799264619 wait complete. 2021-01-06T22:17:51.197 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:17:51.198 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:17:50.633569+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:17:51.198 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 73 2021-01-06T22:17:51.206 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=1, replacing gid=5364 2021-01-06T22:17:51.206 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 3 secs before reviving mds.f 2021-01-06T22:17:54.211 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2021-01-06T22:17:54.212 INFO:tasks.ceph.mds.f:Restarting daemon 2021-01-06T22:17:54.212 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2021-01-06T22:17:54.215 INFO:tasks.ceph.mds.f:Started 2021-01-06T22:17:54.216 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:17:54.293 INFO:tasks.ceph.mds.f.smithi091.stdout:starting mds.f at 2021-01-06T22:17:54.349 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.345+0000 7f587633b700 1 Processor -- start 2021-01-06T22:17:54.350 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.345+0000 7f587633b700 1 -- start start 2021-01-06T22:17:54.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.345+0000 7f587633b700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f58700bf4e0 0x7f5870067d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:54.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.345+0000 7f587633b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f58700609c0 0x7f587014b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:54.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.345+0000 7f587633b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58700676a0 con 0x7f58700bf8c0 2021-01-06T22:17:54.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.345+0000 7f587633b700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58700bbef0 con 0x7f58700bf4e0 2021-01-06T22:17:54.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.345+0000 7f587633b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5870057510 con 0x7f58700609c0 2021-01-06T22:17:54.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.345+0000 7f5867fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f58700609c0 0x7f587014b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:54.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.345+0000 7f58748d8700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f58700bf8c0 0x7f5870063b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51246/0 (socket says 172.21.15.203:51246) 2021-01-06T22:17:54.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.345+0000 7f58748d8700 1 -- 172.21.15.203:0/387630048 learned_addr learned my addr 172.21.15.203:0/387630048 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:17:54.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f586ffff700 1 --2- 172.21.15.203:0/387630048 >> v2:172.21.15.91:3300/0 conn(0x7f58700bf4e0 0x7f5870067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:54.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f586f7fe700 1 -- 172.21.15.203:0/387630048 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980302746 0 0) 0x7f58700676a0 con 0x7f58700bf8c0 2021-01-06T22:17:54.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f586f7fe700 1 -- 172.21.15.203:0/387630048 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5850003230 con 0x7f58700bf8c0 2021-01-06T22:17:54.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f5867fff700 1 -- 172.21.15.203:0/387630048 >> v2:172.21.15.91:3300/0 conn(0x7f58700bf4e0 msgr2=0x7f5870067d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:54.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f5867fff700 1 --2- 172.21.15.203:0/387630048 >> v2:172.21.15.91:3300/0 conn(0x7f58700bf4e0 0x7f5870067d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:54.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f5867fff700 1 -- 172.21.15.203:0/387630048 >> v1:172.21.15.203:6789/0 conn(0x7f58700bf8c0 legacy=0x7f5870063b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:54.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f5867fff700 1 -- 172.21.15.203:0/387630048 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58700bc150 con 0x7f58700609c0 2021-01-06T22:17:54.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f5867fff700 1 --2- 172.21.15.203:0/387630048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58700609c0 0x7f587014b250 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f5870065f00 tx=0x7f5858006de0).ready entity=mon.1 client_cookie=a1722cf598055242 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:54.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f586f7fe700 1 -- 172.21.15.203:0/387630048 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58580096d0 con 0x7f58700609c0 2021-01-06T22:17:54.356 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f586f7fe700 1 -- 172.21.15.203:0/387630048 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5858009830 con 0x7f58700609c0 2021-01-06T22:17:54.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f586f7fe700 1 -- 172.21.15.203:0/387630048 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5858009bb0 con 0x7f58700609c0 2021-01-06T22:17:54.360 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f587633b700 1 -- 172.21.15.203:0/387630048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58700609c0 msgr2=0x7f587014b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:54.361 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f587633b700 1 --2- 172.21.15.203:0/387630048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58700609c0 0x7f587014b250 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f5870065f00 tx=0x7f5858006de0).stop 2021-01-06T22:17:54.361 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f587633b700 1 -- 172.21.15.203:0/387630048 shutdown_connections 2021-01-06T22:17:54.361 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f587633b700 1 --2- 172.21.15.203:0/387630048 >> v2:172.21.15.91:3300/0 conn(0x7f58700bf4e0 0x7f5870067d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:54.362 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f587633b700 1 --2- 172.21.15.203:0/387630048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58700609c0 0x7f587014b250 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:54.362 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f587633b700 1 -- 172.21.15.203:0/387630048 >> 172.21.15.203:0/387630048 conn(0x7f5870056cd0 msgr2=0x7f58700602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:54.362 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f587633b700 1 -- 172.21.15.203:0/387630048 shutdown_connections 2021-01-06T22:17:54.362 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.349+0000 7f587633b700 1 -- 172.21.15.203:0/387630048 wait complete. 2021-01-06T22:17:54.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f587633b700 1 Processor -- start 2021-01-06T22:17:54.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f587633b700 1 -- start start 2021-01-06T22:17:54.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f587633b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58700609c0 0x7f58700bced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:54.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f587633b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58700bf4e0 0x7f587014f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:54.364 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f587633b700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58700bf8c0 0x7f5870160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:54.364 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f587633b700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58700bdf40 con 0x7f58700bf8c0 2021-01-06T22:17:54.364 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f587633b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58700c2b60 con 0x7f58700609c0 2021-01-06T22:17:54.364 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f587633b700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58700be130 con 0x7f58700bf4e0 2021-01-06T22:17:54.365 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f586ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58700609c0 0x7f58700bced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:54.365 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f586ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58700609c0 0x7f58700bced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52654/0 (socket says 172.21.15.203:52654) 2021-01-06T22:17:54.365 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f586ffff700 1 -- 172.21.15.203:0/1718643097 learned_addr learned my addr 172.21.15.203:0/1718643097 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:54.365 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f58748d8700 1 --2- 172.21.15.203:0/1718643097 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58700bf8c0 0x7f5870160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:54.366 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f5867fff700 1 --2- 172.21.15.203:0/1718643097 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58700bf4e0 0x7f587014f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:54.366 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f586ffff700 1 -- 172.21.15.203:0/1718643097 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58700bf4e0 msgr2=0x7f587014f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:54.366 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f586ffff700 1 --2- 172.21.15.203:0/1718643097 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58700bf4e0 0x7f587014f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:54.366 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f586ffff700 1 -- 172.21.15.203:0/1718643097 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58700bf8c0 msgr2=0x7f5870160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:54.367 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f586ffff700 1 --2- 172.21.15.203:0/1718643097 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58700bf8c0 0x7f5870160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:54.367 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f586ffff700 1 -- 172.21.15.203:0/1718643097 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58700be390 con 0x7f58700609c0 2021-01-06T22:17:54.367 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f58748d8700 1 --2- 172.21.15.203:0/1718643097 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58700bf8c0 0x7f5870160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:17:54.367 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f5867fff700 1 --2- 172.21.15.203:0/1718643097 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58700bf4e0 0x7f587014f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:17:54.368 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f586ffff700 1 --2- 172.21.15.203:0/1718643097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58700609c0 0x7f58700bced0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f586000eb20 tx=0x7f5860006de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:54.368 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f586cff9700 1 -- 172.21.15.203:0/1718643097 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586000ca40 con 0x7f58700609c0 2021-01-06T22:17:54.368 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f586cff9700 1 -- 172.21.15.203:0/1718643097 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586000cba0 con 0x7f58700609c0 2021-01-06T22:17:54.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f587633b700 1 -- 172.21.15.203:0/1718643097 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5870153d60 con 0x7f58700609c0 2021-01-06T22:17:54.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f586cff9700 1 -- 172.21.15.203:0/1718643097 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58600123f0 con 0x7f58700609c0 2021-01-06T22:17:54.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.353+0000 7f587633b700 1 -- 172.21.15.203:0/1718643097 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5870148de0 con 0x7f58700609c0 2021-01-06T22:17:54.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.357+0000 7f587633b700 1 -- 172.21.15.203:0/1718643097 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f587005cfe0 con 0x7f58700609c0 2021-01-06T22:17:54.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.357+0000 7f586cff9700 1 -- 172.21.15.203:0/1718643097 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f5860012760 con 0x7f58700609c0 2021-01-06T22:17:54.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.357+0000 7f586cff9700 1 --2- 172.21.15.203:0/1718643097 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58540563d0 0x7f5854058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:54.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.357+0000 7f586cff9700 1 -- 172.21.15.203:0/1718643097 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f586006fd20 con 0x7f58700609c0 2021-01-06T22:17:54.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.357+0000 7f5867fff700 1 --2- 172.21.15.203:0/1718643097 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58540563d0 0x7f5854058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:54.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.357+0000 7f5867fff700 1 --2- 172.21.15.203:0/1718643097 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58540563d0 0x7f5854058890 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f5858006e20 tx=0x7f5858016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:54.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.365+0000 7f586cff9700 1 -- 172.21.15.203:0/1718643097 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f58600452a0 con 0x7f58700609c0 2021-01-06T22:17:54.580 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.573+0000 7f587633b700 1 -- 172.21.15.203:0/1718643097 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f587005da20 con 0x7f58700609c0 2021-01-06T22:17:54.580 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.577+0000 7f586cff9700 1 -- 172.21.15.203:0/1718643097 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+3214 (secure 0 0 0) 0x7f58600488c0 con 0x7f58700609c0 2021-01-06T22:17:54.580 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:17:54.582 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:17:50.633569+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:17:54.583 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.577+0000 7f587633b700 1 -- 172.21.15.203:0/1718643097 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58540563d0 msgr2=0x7f5854058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:54.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.577+0000 7f587633b700 1 --2- 172.21.15.203:0/1718643097 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58540563d0 0x7f5854058890 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f5858006e20 tx=0x7f5858016000).stop 2021-01-06T22:17:54.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.577+0000 7f587633b700 1 -- 172.21.15.203:0/1718643097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58700609c0 msgr2=0x7f58700bced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:54.584 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.577+0000 7f587633b700 1 --2- 172.21.15.203:0/1718643097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58700609c0 0x7f58700bced0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f586000eb20 tx=0x7f5860006de0).stop 2021-01-06T22:17:54.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.577+0000 7f587633b700 1 -- 172.21.15.203:0/1718643097 shutdown_connections 2021-01-06T22:17:54.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.577+0000 7f587633b700 1 --2- 172.21.15.203:0/1718643097 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f58540563d0 0x7f5854058890 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:54.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.577+0000 7f587633b700 1 --2- 172.21.15.203:0/1718643097 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f58700bf8c0 0x7f5870160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:54.585 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.577+0000 7f587633b700 1 --2- 172.21.15.203:0/1718643097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58700609c0 0x7f58700bced0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:54.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.577+0000 7f587633b700 1 --2- 172.21.15.203:0/1718643097 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58700bf4e0 0x7f587014f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:54.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.577+0000 7f587633b700 1 -- 172.21.15.203:0/1718643097 >> 172.21.15.203:0/1718643097 conn(0x7f5870056cd0 msgr2=0x7f5870067c90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:54.586 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.577+0000 7f587633b700 1 -- 172.21.15.203:0/1718643097 shutdown_connections 2021-01-06T22:17:54.587 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:54.577+0000 7f587633b700 1 -- 172.21.15.203:0/1718643097 wait complete. 2021-01-06T22:17:54.587 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 73 2021-01-06T22:17:54.595 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2021-01-06T22:17:56.597 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:17:56.729 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.721+0000 7fb90bfff700 1 Processor -- start 2021-01-06T22:17:56.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.721+0000 7fb90bfff700 1 -- start start 2021-01-06T22:17:56.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.721+0000 7fb90bfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb90c063010 0x7fb90c14e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:56.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.721+0000 7fb90bfff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb90c0be990 con 0x7fb90c0be1b0 2021-01-06T22:17:56.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.721+0000 7fb90bfff700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb90c0be6f0 con 0x7fb90c0633f0 2021-01-06T22:17:56.730 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.721+0000 7fb90bfff700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb90c14ec60 con 0x7fb90c063010 2021-01-06T22:17:56.731 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.721+0000 7fb903fff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fb90c0633f0 0x7fb90c064b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51258/0 (socket says 172.21.15.203:51258) 2021-01-06T22:17:56.732 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.721+0000 7fb903fff700 1 -- 172.21.15.203:0/1535888435 learned_addr learned my addr 172.21.15.203:0/1535888435 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:17:56.733 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.721+0000 7fb90affd700 1 --2- 172.21.15.203:0/1535888435 >> v2:172.21.15.203:3300/0 conn(0x7fb90c063010 0x7fb90c14e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:56.733 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.721+0000 7fb90a7fc700 1 -- 172.21.15.203:0/1535888435 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551972934 0 0) 0x7fb90c0be6f0 con 0x7fb90c0633f0 2021-01-06T22:17:56.734 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.721+0000 7fb90a7fc700 1 -- 172.21.15.203:0/1535888435 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8ec003230 con 0x7fb90c0633f0 2021-01-06T22:17:56.735 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.721+0000 7fb90affd700 1 -- 172.21.15.203:0/1535888435 >> v1:172.21.15.203:6789/0 conn(0x7fb90c0633f0 legacy=0x7fb90c064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:56.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.721+0000 7fb90affd700 1 -- 172.21.15.203:0/1535888435 >> v1:172.21.15.91:6789/0 conn(0x7fb90c0be1b0 legacy=0x7fb90c147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:56.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.721+0000 7fb90affd700 1 -- 172.21.15.203:0/1535888435 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c0c1050 con 0x7fb90c063010 2021-01-06T22:17:56.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.721+0000 7fb90affd700 1 --2- 172.21.15.203:0/1535888435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c063010 0x7fb90c14e770 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7fb8fc00b980 tx=0x7fb8fc006de0).ready entity=mon.1 client_cookie=3d550ad2bf1fdc34 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:56.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90a7fc700 1 -- 172.21.15.203:0/1535888435 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc009840 con 0x7fb90c063010 2021-01-06T22:17:56.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90a7fc700 1 -- 172.21.15.203:0/1535888435 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8fc0099a0 con 0x7fb90c063010 2021-01-06T22:17:56.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90a7fc700 1 -- 172.21.15.203:0/1535888435 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc009d20 con 0x7fb90c063010 2021-01-06T22:17:56.738 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90bfff700 1 -- 172.21.15.203:0/1535888435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c063010 msgr2=0x7fb90c14e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:56.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90bfff700 1 --2- 172.21.15.203:0/1535888435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c063010 0x7fb90c14e770 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7fb8fc00b980 tx=0x7fb8fc006de0).stop 2021-01-06T22:17:56.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90bfff700 1 -- 172.21.15.203:0/1535888435 shutdown_connections 2021-01-06T22:17:56.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90bfff700 1 --2- 172.21.15.203:0/1535888435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c063010 0x7fb90c14e770 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:56.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90bfff700 1 -- 172.21.15.203:0/1535888435 >> 172.21.15.203:0/1535888435 conn(0x7fb90c067e20 msgr2=0x7fb90c068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:56.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90bfff700 1 -- 172.21.15.203:0/1535888435 shutdown_connections 2021-01-06T22:17:56.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90bfff700 1 -- 172.21.15.203:0/1535888435 wait complete. 2021-01-06T22:17:56.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90bfff700 1 Processor -- start 2021-01-06T22:17:56.751 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90bfff700 1 -- start start 2021-01-06T22:17:56.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90bfff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb90c063010 0x7fb90c147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:56.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90bfff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb90c0633f0 0x7fb90c15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:56.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c0be1b0 0x7fb90c15bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:56.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90bfff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c15c090 con 0x7fb90c0633f0 2021-01-06T22:17:56.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90bfff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c141c10 con 0x7fb90c0be1b0 2021-01-06T22:17:56.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90bfff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c0ba6b0 con 0x7fb90c063010 2021-01-06T22:17:56.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c0be1b0 0x7fb90c15bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:56.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c0be1b0 0x7fb90c15bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52672/0 (socket says 172.21.15.203:52672) 2021-01-06T22:17:56.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90b7fe700 1 -- 172.21.15.203:0/489083236 learned_addr learned my addr 172.21.15.203:0/489083236 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:56.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90affd700 1 --2- 172.21.15.203:0/489083236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb90c063010 0x7fb90c147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:56.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb903fff700 1 --2- 172.21.15.203:0/489083236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb90c0633f0 0x7fb90c15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:56.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90b7fe700 1 -- 172.21.15.203:0/489083236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb90c063010 msgr2=0x7fb90c147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:56.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.725+0000 7fb90b7fe700 1 --2- 172.21.15.203:0/489083236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb90c063010 0x7fb90c147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:56.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.729+0000 7fb90b7fe700 1 -- 172.21.15.203:0/489083236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb90c0633f0 msgr2=0x7fb90c15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:56.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.729+0000 7fb90b7fe700 1 --2- 172.21.15.203:0/489083236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb90c0633f0 0x7fb90c15b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:56.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.729+0000 7fb90b7fe700 1 -- 172.21.15.203:0/489083236 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c141e70 con 0x7fb90c0be1b0 2021-01-06T22:17:56.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.729+0000 7fb90affd700 1 --2- 172.21.15.203:0/489083236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb90c063010 0x7fb90c147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:17:56.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.729+0000 7fb90b7fe700 1 --2- 172.21.15.203:0/489083236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c0be1b0 0x7fb90c15bba0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fb90400cac0 tx=0x7fb904007f00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:56.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.729+0000 7fb902ffd700 1 -- 172.21.15.203:0/489083236 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90400a7f0 con 0x7fb90c0be1b0 2021-01-06T22:17:56.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.729+0000 7fb902ffd700 1 -- 172.21.15.203:0/489083236 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90400a950 con 0x7fb90c0be1b0 2021-01-06T22:17:56.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.729+0000 7fb90bfff700 1 -- 172.21.15.203:0/489083236 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb90c148710 con 0x7fb90c0be1b0 2021-01-06T22:17:56.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.729+0000 7fb90bfff700 1 -- 172.21.15.203:0/489083236 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90c148c50 con 0x7fb90c0be1b0 2021-01-06T22:17:56.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.729+0000 7fb902ffd700 1 -- 172.21.15.203:0/489083236 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90400acd0 con 0x7fb90c0be1b0 2021-01-06T22:17:56.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.729+0000 7fb90bfff700 1 -- 172.21.15.203:0/489083236 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e4004f60 con 0x7fb90c0be1b0 2021-01-06T22:17:56.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.729+0000 7fb902ffd700 1 -- 172.21.15.203:0/489083236 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fb904019030 con 0x7fb90c0be1b0 2021-01-06T22:17:56.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.729+0000 7fb902ffd700 1 --2- 172.21.15.203:0/489083236 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb8f00563d0 0x7fb8f0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:56.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.729+0000 7fb902ffd700 1 -- 172.21.15.203:0/489083236 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fb90406f080 con 0x7fb90c0be1b0 2021-01-06T22:17:56.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.729+0000 7fb90affd700 1 --2- 172.21.15.203:0/489083236 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb8f00563d0 0x7fb8f0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:56.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.729+0000 7fb90affd700 1 --2- 172.21.15.203:0/489083236 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb8f00563d0 0x7fb8f0058890 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fb90c065c10 tx=0x7fb8fc016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:56.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.737+0000 7fb902ffd700 1 -- 172.21.15.203:0/489083236 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fb9040446b0 con 0x7fb90c0be1b0 2021-01-06T22:17:56.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.965+0000 7fb90bfff700 1 -- 172.21.15.203:0/489083236 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8e4005790 con 0x7fb90c0be1b0 2021-01-06T22:17:56.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.965+0000 7fb902ffd700 1 -- 172.21.15.203:0/489083236 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+3214 (secure 0 0 0) 0x7fb904017070 con 0x7fb90c0be1b0 2021-01-06T22:17:56.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.965+0000 7fb90bfff700 1 -- 172.21.15.203:0/489083236 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb8f00563d0 msgr2=0x7fb8f0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:56.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.965+0000 7fb90bfff700 1 --2- 172.21.15.203:0/489083236 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb8f00563d0 0x7fb8f0058890 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fb90c065c10 tx=0x7fb8fc016000).stop 2021-01-06T22:17:56.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.965+0000 7fb90bfff700 1 -- 172.21.15.203:0/489083236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c0be1b0 msgr2=0x7fb90c15bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:56.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.965+0000 7fb90bfff700 1 --2- 172.21.15.203:0/489083236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c0be1b0 0x7fb90c15bba0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fb90400cac0 tx=0x7fb904007f00).stop 2021-01-06T22:17:56.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.965+0000 7fb90bfff700 1 -- 172.21.15.203:0/489083236 shutdown_connections 2021-01-06T22:17:56.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.965+0000 7fb90bfff700 1 --2- 172.21.15.203:0/489083236 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb8f00563d0 0x7fb8f0058890 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:56.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.965+0000 7fb90bfff700 1 --2- 172.21.15.203:0/489083236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c0be1b0 0x7fb90c15bba0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:56.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.965+0000 7fb90bfff700 1 --2- 172.21.15.203:0/489083236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb90c063010 0x7fb90c147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:56.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.965+0000 7fb90bfff700 1 --2- 172.21.15.203:0/489083236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb90c0633f0 0x7fb90c15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:56.980 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.965+0000 7fb90bfff700 1 -- 172.21.15.203:0/489083236 >> 172.21.15.203:0/489083236 conn(0x7fb90c067e20 msgr2=0x7fb90c0b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:56.980 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.965+0000 7fb90bfff700 1 -- 172.21.15.203:0/489083236 shutdown_connections 2021-01-06T22:17:56.980 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:56.965+0000 7fb90bfff700 1 -- 172.21.15.203:0/489083236 wait complete. 2021-01-06T22:17:56.980 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 73 2021-01-06T22:17:56.981 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:17:56.981 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:17:50.633569+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:17:56.987 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2021-01-06T22:17:58.988 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:17:59.128 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f8816700 1 Processor -- start 2021-01-06T22:17:59.128 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f8816700 1 -- start start 2021-01-06T22:17:59.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f8816700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb1f4060fe0 0x7fb1f414fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:59.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f8816700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb1f40a5a60 0x7fb1f4064340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:59.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f8816700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f4067660 con 0x7fb1f40a5680 2021-01-06T22:17:59.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f8816700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f4064ab0 con 0x7fb1f40a5a60 2021-01-06T22:17:59.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f8816700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f41500f0 con 0x7fb1f4060fe0 2021-01-06T22:17:59.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f3fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb1f40a5a60 0x7fb1f4064340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:59.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f3fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb1f40a5a60 0x7fb1f4064340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52684/0 (socket says 172.21.15.203:52684) 2021-01-06T22:17:59.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f3fff700 1 -- 172.21.15.203:0/1383648932 learned_addr learned my addr 172.21.15.203:0/1383648932 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:59.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f2ffd700 1 --2- 172.21.15.203:0/1383648932 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb1f4060fe0 0x7fb1f414fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:59.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f3fff700 1 -- 172.21.15.203:0/1383648932 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb1f4060fe0 msgr2=0x7fb1f414fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:17:59.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f3fff700 1 --2- 172.21.15.203:0/1383648932 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb1f4060fe0 0x7fb1f414fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:59.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f3fff700 1 -- 172.21.15.203:0/1383648932 >> v1:172.21.15.91:6789/0 conn(0x7fb1f40a5680 legacy=0x7fb1f40608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:59.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f3fff700 1 -- 172.21.15.203:0/1383648932 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f4064830 con 0x7fb1f40a5a60 2021-01-06T22:17:59.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f3fff700 1 --2- 172.21.15.203:0/1383648932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1f40a5a60 0x7fb1f4064340 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fb1ec009d50 tx=0x7fb1ec009290).ready entity=mon.1 client_cookie=fc0a37c5958245e7 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:59.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f27fc700 1 -- 172.21.15.203:0/1383648932 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec00bc50 con 0x7fb1f40a5a60 2021-01-06T22:17:59.134 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f27fc700 1 -- 172.21.15.203:0/1383648932 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec00bdb0 con 0x7fb1f40a5a60 2021-01-06T22:17:59.135 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f27fc700 1 -- 172.21.15.203:0/1383648932 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec011610 con 0x7fb1f40a5a60 2021-01-06T22:17:59.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f8816700 1 -- 172.21.15.203:0/1383648932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1f40a5a60 msgr2=0x7fb1f4064340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:59.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f8816700 1 --2- 172.21.15.203:0/1383648932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1f40a5a60 0x7fb1f4064340 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fb1ec009d50 tx=0x7fb1ec009290).stop 2021-01-06T22:17:59.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f8816700 1 -- 172.21.15.203:0/1383648932 shutdown_connections 2021-01-06T22:17:59.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f8816700 1 --2- 172.21.15.203:0/1383648932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1f40a5a60 0x7fb1f4064340 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:59.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f8816700 1 --2- 172.21.15.203:0/1383648932 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb1f4060fe0 0x7fb1f414fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:59.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f8816700 1 -- 172.21.15.203:0/1383648932 >> 172.21.15.203:0/1383648932 conn(0x7fb1f4061530 msgr2=0x7fb1f4063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:59.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f8816700 1 -- 172.21.15.203:0/1383648932 shutdown_connections 2021-01-06T22:17:59.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.125+0000 7fb1f8816700 1 -- 172.21.15.203:0/1383648932 wait complete. 2021-01-06T22:17:59.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f8816700 1 Processor -- start 2021-01-06T22:17:59.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f8816700 1 -- start start 2021-01-06T22:17:59.142 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f8816700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb1f4060fe0 0x7fb1f40b9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:59.142 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f8816700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1f40a5680 0x7fb1f415be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:59.142 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f8816700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1f40a5a60 0x7fb1f4152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:59.142 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f8816700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f415b2c0 con 0x7fb1f40a5680 2021-01-06T22:17:59.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f8816700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f40b98a0 con 0x7fb1f40a5a60 2021-01-06T22:17:59.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f8816700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f40b9b40 con 0x7fb1f4060fe0 2021-01-06T22:17:59.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1f40a5a60 0x7fb1f4152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:59.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1f40a5a60 0x7fb1f4152690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52690/0 (socket says 172.21.15.203:52690) 2021-01-06T22:17:59.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 -- 172.21.15.203:0/1100843924 learned_addr learned my addr 172.21.15.203:0/1100843924 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:17:59.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f2ffd700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1f40a5680 0x7fb1f415be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:59.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f37fe700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb1f4060fe0 0x7fb1f40b9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:59.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 -- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb1f4060fe0 msgr2=0x7fb1f40b9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:59.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb1f4060fe0 0x7fb1f40b9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:59.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 -- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1f40a5680 msgr2=0x7fb1f415be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:59.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1f40a5680 0x7fb1f415be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:59.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 -- 172.21.15.203:0/1100843924 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f415b4b0 con 0x7fb1f40a5a60 2021-01-06T22:17:59.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f37fe700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb1f4060fe0 0x7fb1f40b9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:17:59.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1f40a5a60 0x7fb1f4152690 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fb1ec0092d0 tx=0x7fb1ec017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:59.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 -- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1f40a5a60 msgr2=0x7fb1f4152690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2021-01-06T22:17:59.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 -- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1f40a5a60 msgr2=0x7fb1f4152690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2021-01-06T22:17:59.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1f40a5a60 0x7fb1f4152690 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fb1ec0092d0 tx=0x7fb1ec017000).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:17:59.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1f40a5a60 0x7fb1f4152690 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fb1ec0092d0 tx=0x7fb1ec017000).stop 2021-01-06T22:17:59.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1db7fe700 1 -- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1f40a5a60 msgr2=0x7fb1f4152690 unknown :-1 s=STATE_CLOSED l=1).mark_down 2021-01-06T22:17:59.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1db7fe700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1f40a5a60 0x7fb1f4152690 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:59.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1db7fe700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1f40a5680 0x7fb1d40051e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:59.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1db7fe700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb1f4060fe0 0x7fb1d4007720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:59.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1db7fe700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1d4007c80 0x7fb1d400a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:59.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1db7fe700 1 -- 172.21.15.203:0/1100843924 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d400a7a0 con 0x7fb1f40a5680 2021-01-06T22:17:59.148 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1db7fe700 1 -- 172.21.15.203:0/1100843924 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d400a8e0 con 0x7fb1d4007c80 2021-01-06T22:17:59.148 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1db7fe700 1 -- 172.21.15.203:0/1100843924 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d400aaa0 con 0x7fb1f4060fe0 2021-01-06T22:17:59.148 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1d4007c80 0x7fb1d400a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:59.148 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f37fe700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1f40a5680 0x7fb1d40051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:59.149 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f2ffd700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb1f4060fe0 0x7fb1d4007720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:59.149 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 -- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb1f4060fe0 msgr2=0x7fb1d4007720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:59.149 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb1f4060fe0 0x7fb1d4007720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:59.149 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 -- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1f40a5680 msgr2=0x7fb1d40051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:59.149 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1f40a5680 0x7fb1d40051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:59.150 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 -- 172.21.15.203:0/1100843924 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d400f130 con 0x7fb1d4007c80 2021-01-06T22:17:59.150 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f3fff700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1d4007c80 0x7fb1d400a110 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7fb1ec01c000 tx=0x7fb1ec01b000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:59.150 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1db7fe700 1 -- 172.21.15.203:0/1100843924 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec011e70 con 0x7fb1d4007c80 2021-01-06T22:17:59.150 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1db7fe700 1 -- 172.21.15.203:0/1100843924 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec024600 con 0x7fb1d4007c80 2021-01-06T22:17:59.151 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1f8816700 1 -- 172.21.15.203:0/1100843924 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f415b750 con 0x7fb1d4007c80 2021-01-06T22:17:59.151 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.129+0000 7fb1db7fe700 1 -- 172.21.15.203:0/1100843924 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec0249a0 con 0x7fb1d4007c80 2021-01-06T22:17:59.151 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.133+0000 7fb1f8816700 1 -- 172.21.15.203:0/1100843924 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f40b9560 con 0x7fb1d4007c80 2021-01-06T22:17:59.151 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.133+0000 7fb1f8816700 1 -- 172.21.15.203:0/1100843924 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f40bbdc0 con 0x7fb1d4007c80 2021-01-06T22:17:59.151 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.133+0000 7fb1db7fe700 1 -- 172.21.15.203:0/1100843924 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fb1ec00f070 con 0x7fb1d4007c80 2021-01-06T22:17:59.152 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.133+0000 7fb1db7fe700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb1f40a5a60 0x7fb1d40642a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:17:59.152 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.133+0000 7fb1db7fe700 1 -- 172.21.15.203:0/1100843924 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fb1ec0741f0 con 0x7fb1d4007c80 2021-01-06T22:17:59.152 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.133+0000 7fb1f37fe700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb1f40a5a60 0x7fb1d40642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:17:59.152 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.133+0000 7fb1f37fe700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb1f40a5a60 0x7fb1d40642a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fb1e40036a0 tx=0x7fb1e4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:17:59.153 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.141+0000 7fb1db7fe700 1 -- 172.21.15.203:0/1100843924 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fb1f40bbdc0 con 0x7fb1d4007c80 2021-01-06T22:17:59.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.381+0000 7fb1f8816700 1 -- 172.21.15.203:0/1100843924 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1f40bbdc0 con 0x7fb1d4007c80 2021-01-06T22:17:59.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.381+0000 7fb1db7fe700 1 -- 172.21.15.203:0/1100843924 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+3567 (secure 0 0 0) 0x7fb1f40bbdc0 con 0x7fb1d4007c80 2021-01-06T22:17:59.385 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:17:59.385 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5261,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":74}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:17:50.633569+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:17:59.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.381+0000 7fb1f8816700 1 -- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb1f40a5a60 msgr2=0x7fb1d40642a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:59.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.385+0000 7fb1f8816700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb1f40a5a60 0x7fb1d40642a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fb1e40036a0 tx=0x7fb1e4009040).stop 2021-01-06T22:17:59.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.385+0000 7fb1f8816700 1 -- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1d4007c80 msgr2=0x7fb1d400a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:17:59.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.385+0000 7fb1f8816700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1d4007c80 0x7fb1d400a110 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7fb1ec01c000 tx=0x7fb1ec01b000).stop 2021-01-06T22:17:59.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.385+0000 7fb1f8816700 1 -- 172.21.15.203:0/1100843924 shutdown_connections 2021-01-06T22:17:59.389 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.385+0000 7fb1f8816700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb1f40a5a60 0x7fb1d40642a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:59.389 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.385+0000 7fb1f8816700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1d4007c80 0x7fb1d400a110 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:59.389 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.385+0000 7fb1f8816700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb1f4060fe0 0x7fb1d4007720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:59.389 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.385+0000 7fb1f8816700 1 --2- 172.21.15.203:0/1100843924 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1f40a5680 0x7fb1d40051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:17:59.390 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.385+0000 7fb1f8816700 1 -- 172.21.15.203:0/1100843924 >> 172.21.15.203:0/1100843924 conn(0x7fb1f4061530 msgr2=0x7fb1f40bd080 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:17:59.390 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.385+0000 7fb1f8816700 1 -- 172.21.15.203:0/1100843924 shutdown_connections 2021-01-06T22:17:59.390 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:17:59.385+0000 7fb1f8816700 1 -- 172.21.15.203:0/1100843924 wait complete. 2021-01-06T22:17:59.390 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 74 2021-01-06T22:17:59.402 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2021-01-06T22:17:59.403 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 63 secs before thrashing 2021-01-06T22:18:18.053 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:18:18.057 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:18:48.089 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:18:48.092 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:19:02.406 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:19:02.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f6542286700 1 Processor -- start 2021-01-06T22:19:02.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f6542286700 1 -- start start 2021-01-06T22:19:02.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f6542286700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f653c0b95f0 0x7f653c0c1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:02.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f6542286700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f653c14ec30 0x7f653c1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:02.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f6542286700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f653c0654a0 con 0x7f653c0c13b0 2021-01-06T22:19:02.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f6542286700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653c14ac90 con 0x7f653c0b95f0 2021-01-06T22:19:02.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f6542286700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653c0b9c40 con 0x7f653c14ec30 2021-01-06T22:19:02.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f653bfff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f653c0c13b0 0x7f653c064d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51354/0 (socket says 172.21.15.203:51354) 2021-01-06T22:19:02.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f653bfff700 1 -- 172.21.15.203:0/253282457 learned_addr learned my addr 172.21.15.203:0/253282457 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:19:02.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f6540823700 1 --2- 172.21.15.203:0/253282457 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f653c14ec30 0x7f653c1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:02.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f653b7fe700 1 --2- 172.21.15.203:0/253282457 >> v2:172.21.15.91:3300/0 conn(0x7f653c0b95f0 0x7f653c0c1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:02.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f653affd700 1 -- 172.21.15.203:0/253282457 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620831295 0 0) 0x7f653c0654a0 con 0x7f653c0c13b0 2021-01-06T22:19:02.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f653affd700 1 -- 172.21.15.203:0/253282457 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6528003230 con 0x7f653c0c13b0 2021-01-06T22:19:02.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f653b7fe700 1 -- 172.21.15.203:0/253282457 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f653c14ec30 msgr2=0x7f653c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:02.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f653b7fe700 1 --2- 172.21.15.203:0/253282457 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f653c14ec30 0x7f653c1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:02.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f653b7fe700 1 -- 172.21.15.203:0/253282457 >> v1:172.21.15.203:6789/0 conn(0x7f653c0c13b0 legacy=0x7f653c064d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:02.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f653b7fe700 1 -- 172.21.15.203:0/253282457 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c14ae80 con 0x7f653c0b95f0 2021-01-06T22:19:02.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f653b7fe700 1 --2- 172.21.15.203:0/253282457 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f653c0b95f0 0x7f653c0c1790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f653000b980 tx=0x7f6530006de0).ready entity=mon.0 client_cookie=4379a0063e0c9f8b server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:02.556 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f653affd700 1 -- 172.21.15.203:0/253282457 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65300096d0 con 0x7f653c0b95f0 2021-01-06T22:19:02.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f653affd700 1 -- 172.21.15.203:0/253282457 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6530009830 con 0x7f653c0b95f0 2021-01-06T22:19:02.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f653affd700 1 -- 172.21.15.203:0/253282457 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530009bb0 con 0x7f653c0b95f0 2021-01-06T22:19:02.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f6542286700 1 -- 172.21.15.203:0/253282457 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f653c0b95f0 msgr2=0x7f653c0c1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:02.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f6542286700 1 --2- 172.21.15.203:0/253282457 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f653c0b95f0 0x7f653c0c1790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f653000b980 tx=0x7f6530006de0).stop 2021-01-06T22:19:02.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f6542286700 1 -- 172.21.15.203:0/253282457 shutdown_connections 2021-01-06T22:19:02.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f6542286700 1 --2- 172.21.15.203:0/253282457 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f653c14ec30 0x7f653c1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:02.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f6542286700 1 --2- 172.21.15.203:0/253282457 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f653c0b95f0 0x7f653c0c1790 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:02.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f6542286700 1 -- 172.21.15.203:0/253282457 >> 172.21.15.203:0/253282457 conn(0x7f653c063240 msgr2=0x7f653c067cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:02.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f6542286700 1 -- 172.21.15.203:0/253282457 shutdown_connections 2021-01-06T22:19:02.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.531+0000 7f6542286700 1 -- 172.21.15.203:0/253282457 wait complete. 2021-01-06T22:19:02.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f6542286700 1 Processor -- start 2021-01-06T22:19:02.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f6542286700 1 -- start start 2021-01-06T22:19:02.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f6542286700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f653c0b95f0 0x7f653c149950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:02.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f6542286700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f653c0c13b0 0x7f653c15e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:02.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f6542286700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f653c14ec30 0x7f653c15e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:02.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f6542286700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c14ffc0 con 0x7f653c0c13b0 2021-01-06T22:19:02.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f6542286700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c14f820 con 0x7f653c0b95f0 2021-01-06T22:19:02.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f6542286700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653c14fa80 con 0x7f653c14ec30 2021-01-06T22:19:02.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f653bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f653c0b95f0 0x7f653c149950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:02.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f653bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f653c0b95f0 0x7f653c149950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52766/0 (socket says 172.21.15.203:52766) 2021-01-06T22:19:02.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f653bfff700 1 -- 172.21.15.203:0/790465772 learned_addr learned my addr 172.21.15.203:0/790465772 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:02.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f653b7fe700 1 --2- 172.21.15.203:0/790465772 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f653c0c13b0 0x7f653c15e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:02.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f6540823700 1 --2- 172.21.15.203:0/790465772 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f653c14ec30 0x7f653c15e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:02.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f653bfff700 1 -- 172.21.15.203:0/790465772 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f653c14ec30 msgr2=0x7f653c15e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:02.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f653bfff700 1 --2- 172.21.15.203:0/790465772 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f653c14ec30 0x7f653c15e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:02.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f653bfff700 1 -- 172.21.15.203:0/790465772 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f653c0c13b0 msgr2=0x7f653c15e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:02.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f653bfff700 1 --2- 172.21.15.203:0/790465772 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f653c0c13b0 0x7f653c15e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:02.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f653bfff700 1 -- 172.21.15.203:0/790465772 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c14fd20 con 0x7f653c0b95f0 2021-01-06T22:19:02.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f653bfff700 1 --2- 172.21.15.203:0/790465772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f653c0b95f0 0x7f653c149950 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f652c00c6e0 tx=0x7f652c007ab0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:02.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f6523fff700 1 -- 172.21.15.203:0/790465772 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c0072e0 con 0x7f653c0b95f0 2021-01-06T22:19:02.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f6523fff700 1 -- 172.21.15.203:0/790465772 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652c007440 con 0x7f653c0b95f0 2021-01-06T22:19:02.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f6542286700 1 -- 172.21.15.203:0/790465772 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653c15dc60 con 0x7f653c0b95f0 2021-01-06T22:19:02.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f6523fff700 1 -- 172.21.15.203:0/790465772 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c005810 con 0x7f653c0b95f0 2021-01-06T22:19:02.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.535+0000 7f6542286700 1 -- 172.21.15.203:0/790465772 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653c0bc750 con 0x7f653c0b95f0 2021-01-06T22:19:02.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.539+0000 7f6523fff700 1 -- 172.21.15.203:0/790465772 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f652c010070 con 0x7f653c0b95f0 2021-01-06T22:19:02.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.539+0000 7f6542286700 1 -- 172.21.15.203:0/790465772 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653c14e800 con 0x7f653c0b95f0 2021-01-06T22:19:02.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.539+0000 7f6523fff700 1 --2- 172.21.15.203:0/790465772 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f651c0563d0 0x7f651c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:02.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.539+0000 7f6523fff700 1 -- 172.21.15.203:0/790465772 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f652c06ee20 con 0x7f653c0b95f0 2021-01-06T22:19:02.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.539+0000 7f653b7fe700 1 --2- 172.21.15.203:0/790465772 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f651c0563d0 0x7f651c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:02.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.539+0000 7f653b7fe700 1 --2- 172.21.15.203:0/790465772 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f651c0563d0 0x7f651c058890 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f653000b130 tx=0x7f6530016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:02.567 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.547+0000 7f6523fff700 1 -- 172.21.15.203:0/790465772 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f653c14e800 con 0x7f653c0b95f0 2021-01-06T22:19:02.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.779+0000 7f6542286700 1 -- 172.21.15.203:0/790465772 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f653c14e800 con 0x7f653c0b95f0 2021-01-06T22:19:02.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.779+0000 7f6523fff700 1 -- 172.21.15.203:0/790465772 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+3567 (secure 0 0 0) 0x7f653c14e800 con 0x7f653c0b95f0 2021-01-06T22:19:02.782 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:19:02.782 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":5261,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":74}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:17:50.633569+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:19:02.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.783+0000 7f6542286700 1 -- 172.21.15.203:0/790465772 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f651c0563d0 msgr2=0x7f651c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:02.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.783+0000 7f6542286700 1 --2- 172.21.15.203:0/790465772 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f651c0563d0 0x7f651c058890 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f653000b130 tx=0x7f6530016000).stop 2021-01-06T22:19:02.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.783+0000 7f6542286700 1 -- 172.21.15.203:0/790465772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f653c0b95f0 msgr2=0x7f653c149950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:02.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.783+0000 7f6542286700 1 --2- 172.21.15.203:0/790465772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f653c0b95f0 0x7f653c149950 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f652c00c6e0 tx=0x7f652c007ab0).stop 2021-01-06T22:19:02.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.783+0000 7f6542286700 1 -- 172.21.15.203:0/790465772 shutdown_connections 2021-01-06T22:19:02.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.783+0000 7f6542286700 1 --2- 172.21.15.203:0/790465772 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f651c0563d0 0x7f651c058890 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:02.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.783+0000 7f6542286700 1 --2- 172.21.15.203:0/790465772 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f653c14ec30 0x7f653c15e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:02.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.783+0000 7f6542286700 1 --2- 172.21.15.203:0/790465772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f653c0b95f0 0x7f653c149950 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:02.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.783+0000 7f6542286700 1 --2- 172.21.15.203:0/790465772 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f653c0c13b0 0x7f653c15e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:02.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.783+0000 7f6542286700 1 -- 172.21.15.203:0/790465772 >> 172.21.15.203:0/790465772 conn(0x7f653c063240 msgr2=0x7f653c0574f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:02.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.783+0000 7f6542286700 1 -- 172.21.15.203:0/790465772 shutdown_connections 2021-01-06T22:19:02.787 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.783+0000 7f6542286700 1 -- 172.21.15.203:0/790465772 wait complete. 2021-01-06T22:19:02.787 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 74 2021-01-06T22:19:02.800 INFO:tasks.mds_thrash.fs.[cephfs]:thrashing max_mds: 2 -> 3 2021-01-06T22:19:02.800 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2021-01-06T22:19:02.938 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.927+0000 7efe7f59e700 1 Processor -- start 2021-01-06T22:19:02.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.927+0000 7efe7f59e700 1 -- start start 2021-01-06T22:19:02.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.927+0000 7efe7f59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efe800c49b0 0x7efe801511c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:02.939 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.927+0000 7efe7f59e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7efe80066940 0x7efe800bdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:02.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.927+0000 7efe7f59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe80151ac0 con 0x7efe800c45d0 2021-01-06T22:19:02.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.927+0000 7efe7f59e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe800c1d50 con 0x7efe80066940 2021-01-06T22:19:02.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.927+0000 7efe7f59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe80151d60 con 0x7efe800c49b0 2021-01-06T22:19:02.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe7ed9d700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7efe80066940 0x7efe800bdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:02.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe77fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efe800c49b0 0x7efe801511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:02.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe7ed9d700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7efe80066940 0x7efe800bdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:60628/0 (socket says 172.21.15.203:60628) 2021-01-06T22:19:02.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe7ed9d700 1 -- 172.21.15.203:0/4057886867 learned_addr learned my addr 172.21.15.203:0/4057886867 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:02.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe77fff700 1 -- 172.21.15.203:0/4057886867 >> v2:172.21.15.91:3300/0 conn(0x7efe80066940 msgr2=0x7efe800bdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:02.941 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe77fff700 1 --2- 172.21.15.203:0/4057886867 >> v2:172.21.15.91:3300/0 conn(0x7efe80066940 0x7efe800bdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:02.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe77fff700 1 -- 172.21.15.203:0/4057886867 >> v1:172.21.15.91:6789/0 conn(0x7efe800c45d0 legacy=0x7efe800c1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:02.942 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe77fff700 1 -- 172.21.15.203:0/4057886867 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe80152000 con 0x7efe800c49b0 2021-01-06T22:19:02.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe77fff700 1 --2- 172.21.15.203:0/4057886867 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efe800c49b0 0x7efe801511c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7efe6800b980 tx=0x7efe68006de0).ready entity=mon.2 client_cookie=ad2a02220d3ec2a4 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:02.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe7dd9b700 1 -- 172.21.15.203:0/4057886867 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe680097b0 con 0x7efe800c49b0 2021-01-06T22:19:02.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe7dd9b700 1 -- 172.21.15.203:0/4057886867 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe68009910 con 0x7efe800c49b0 2021-01-06T22:19:02.943 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe7dd9b700 1 -- 172.21.15.203:0/4057886867 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68009c90 con 0x7efe800c49b0 2021-01-06T22:19:02.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe7f59e700 1 -- 172.21.15.203:0/4057886867 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efe800c49b0 msgr2=0x7efe801511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:02.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe7f59e700 1 --2- 172.21.15.203:0/4057886867 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efe800c49b0 0x7efe801511c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7efe6800b980 tx=0x7efe68006de0).stop 2021-01-06T22:19:02.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe7f59e700 1 -- 172.21.15.203:0/4057886867 shutdown_connections 2021-01-06T22:19:02.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe7f59e700 1 --2- 172.21.15.203:0/4057886867 >> v2:172.21.15.91:3300/0 conn(0x7efe80066940 0x7efe800bdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:02.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe7f59e700 1 --2- 172.21.15.203:0/4057886867 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efe800c49b0 0x7efe801511c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:02.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe7f59e700 1 -- 172.21.15.203:0/4057886867 >> 172.21.15.203:0/4057886867 conn(0x7efe800626e0 msgr2=0x7efe80062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:02.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe7f59e700 1 -- 172.21.15.203:0/4057886867 shutdown_connections 2021-01-06T22:19:02.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe7f59e700 1 -- 172.21.15.203:0/4057886867 wait complete. 2021-01-06T22:19:02.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe7f59e700 1 Processor -- start 2021-01-06T22:19:02.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.931+0000 7efe7f59e700 1 -- start start 2021-01-06T22:19:02.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7f59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe80066940 0x7efe8015d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:02.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7f59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efe800c45d0 0x7efe80149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:02.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7f59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efe800c49b0 0x7efe8014a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:02.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7f59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe800bb4c0 con 0x7efe800c45d0 2021-01-06T22:19:02.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7f59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe800bbc30 con 0x7efe80066940 2021-01-06T22:19:02.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7f59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe800be5a0 con 0x7efe800c49b0 2021-01-06T22:19:02.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7e59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe80066940 0x7efe8015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:02.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7e59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe80066940 0x7efe8015d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52786/0 (socket says 172.21.15.203:52786) 2021-01-06T22:19:02.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7e59c700 1 -- 172.21.15.203:0/1579008459 learned_addr learned my addr 172.21.15.203:0/1579008459 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:02.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe77fff700 1 --2- 172.21.15.203:0/1579008459 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efe800c45d0 0x7efe80149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:02.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7ed9d700 1 --2- 172.21.15.203:0/1579008459 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efe800c49b0 0x7efe8014a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:02.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7e59c700 1 -- 172.21.15.203:0/1579008459 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efe800c49b0 msgr2=0x7efe8014a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:02.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7e59c700 1 --2- 172.21.15.203:0/1579008459 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efe800c49b0 0x7efe8014a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:02.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7e59c700 1 -- 172.21.15.203:0/1579008459 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efe800c45d0 msgr2=0x7efe80149b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:02.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7e59c700 1 --2- 172.21.15.203:0/1579008459 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efe800c45d0 0x7efe80149b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:02.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7e59c700 1 -- 172.21.15.203:0/1579008459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe800bb6f0 con 0x7efe80066940 2021-01-06T22:19:02.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7e59c700 1 --2- 172.21.15.203:0/1579008459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe80066940 0x7efe8015d540 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7efe7000c710 tx=0x7efe70007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:02.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe767fc700 1 -- 172.21.15.203:0/1579008459 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70007020 con 0x7efe80066940 2021-01-06T22:19:02.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7f59e700 1 -- 172.21.15.203:0/1579008459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe800bb990 con 0x7efe80066940 2021-01-06T22:19:02.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe767fc700 1 -- 172.21.15.203:0/1579008459 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe70007180 con 0x7efe80066940 2021-01-06T22:19:02.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe767fc700 1 -- 172.21.15.203:0/1579008459 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700064e0 con 0x7efe80066940 2021-01-06T22:19:02.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7f59e700 1 -- 172.21.15.203:0/1579008459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe800be2c0 con 0x7efe80066940 2021-01-06T22:19:02.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.935+0000 7efe7f59e700 1 -- 172.21.15.203:0/1579008459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe800b2aa0 con 0x7efe80066940 2021-01-06T22:19:02.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.939+0000 7efe767fc700 1 -- 172.21.15.203:0/1579008459 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7efe70010070 con 0x7efe80066940 2021-01-06T22:19:02.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.939+0000 7efe767fc700 1 --2- 172.21.15.203:0/1579008459 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7efe640563d0 0x7efe64058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:02.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.939+0000 7efe77fff700 1 --2- 172.21.15.203:0/1579008459 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7efe640563d0 0x7efe64058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:02.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.939+0000 7efe767fc700 1 -- 172.21.15.203:0/1579008459 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7efe7006eb60 con 0x7efe80066940 2021-01-06T22:19:02.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.939+0000 7efe77fff700 1 --2- 172.21.15.203:0/1579008459 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7efe640563d0 0x7efe64058890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7efe6800b130 tx=0x7efe68016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:02.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:02.943+0000 7efe767fc700 1 -- 172.21.15.203:0/1579008459 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7efe70043670 con 0x7efe80066940 2021-01-06T22:19:03.178 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.175+0000 7efe7f59e700 1 -- 172.21.15.203:0/1579008459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7efe800450a0 con 0x7efe80066940 2021-01-06T22:19:03.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.439+0000 7efe767fc700 1 -- 172.21.15.203:0/1579008459 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v75) v1 ==== 105+0+0 (secure 0 0 0) 0x7efe800b2aa0 con 0x7efe80066940 2021-01-06T22:19:03.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.439+0000 7efe7f59e700 1 -- 172.21.15.203:0/1579008459 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7efe640563d0 msgr2=0x7efe64058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:03.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.439+0000 7efe7f59e700 1 --2- 172.21.15.203:0/1579008459 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7efe640563d0 0x7efe64058890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7efe6800b130 tx=0x7efe68016000).stop 2021-01-06T22:19:03.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.439+0000 7efe7f59e700 1 -- 172.21.15.203:0/1579008459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe80066940 msgr2=0x7efe8015d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:03.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.439+0000 7efe7f59e700 1 --2- 172.21.15.203:0/1579008459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe80066940 0x7efe8015d540 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7efe7000c710 tx=0x7efe70007c40).stop 2021-01-06T22:19:03.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.443+0000 7efe7f59e700 1 -- 172.21.15.203:0/1579008459 shutdown_connections 2021-01-06T22:19:03.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.443+0000 7efe7f59e700 1 --2- 172.21.15.203:0/1579008459 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7efe640563d0 0x7efe64058890 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:03.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.443+0000 7efe7f59e700 1 --2- 172.21.15.203:0/1579008459 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efe800c49b0 0x7efe8014a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:03.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.443+0000 7efe7f59e700 1 --2- 172.21.15.203:0/1579008459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe80066940 0x7efe8015d540 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:03.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.443+0000 7efe7f59e700 1 --2- 172.21.15.203:0/1579008459 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efe800c45d0 0x7efe80149b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:03.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.443+0000 7efe7f59e700 1 -- 172.21.15.203:0/1579008459 >> 172.21.15.203:0/1579008459 conn(0x7efe800626e0 msgr2=0x7efe8014f910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:03.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.443+0000 7efe7f59e700 1 -- 172.21.15.203:0/1579008459 shutdown_connections 2021-01-06T22:19:03.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.443+0000 7efe7f59e700 1 -- 172.21.15.203:0/1579008459 wait complete. 2021-01-06T22:19:03.458 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2021-01-06T22:19:03.459 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:19:03.591 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69af59e700 1 Processor -- start 2021-01-06T22:19:03.592 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69af59e700 1 -- start start 2021-01-06T22:19:03.593 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69af59e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f69b0063010 0x7f69b014e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:03.593 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69af59e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f69b00633f0 0x7f69b0064b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:03.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69af59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b00be990 con 0x7f69b00be1b0 2021-01-06T22:19:03.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69af59e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b014ec60 con 0x7f69b00633f0 2021-01-06T22:19:03.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69af59e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b00be6f0 con 0x7f69b0063010 2021-01-06T22:19:03.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69ae59c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f69b0063010 0x7f69b014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:03.594 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69ae59c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f69b0063010 0x7f69b014e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52794/0 (socket says 172.21.15.203:52794) 2021-01-06T22:19:03.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69ae59c700 1 -- 172.21.15.203:0/3976243110 learned_addr learned my addr 172.21.15.203:0/3976243110 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:03.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69add9b700 1 --2- 172.21.15.203:0/3976243110 >> v2:172.21.15.91:3300/0 conn(0x7f69b00633f0 0x7f69b0064b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:03.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69ae59c700 1 -- 172.21.15.203:0/3976243110 >> v2:172.21.15.91:3300/0 conn(0x7f69b00633f0 msgr2=0x7f69b0064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:03.595 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69ae59c700 1 --2- 172.21.15.203:0/3976243110 >> v2:172.21.15.91:3300/0 conn(0x7f69b00633f0 0x7f69b0064b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:03.596 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69ae59c700 1 -- 172.21.15.203:0/3976243110 >> v1:172.21.15.91:6789/0 conn(0x7f69b00be1b0 legacy=0x7f69b0147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:03.596 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69ae59c700 1 -- 172.21.15.203:0/3976243110 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b00c1050 con 0x7f69b0063010 2021-01-06T22:19:03.596 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69ae59c700 1 --2- 172.21.15.203:0/3976243110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69b0063010 0x7f69b014e770 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f69b00667e0 tx=0x7f69a0006ea0).ready entity=mon.1 client_cookie=5e2498eaf0d5dc18 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:03.597 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69ad59a700 1 -- 172.21.15.203:0/3976243110 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a0009890 con 0x7f69b0063010 2021-01-06T22:19:03.597 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69ad59a700 1 -- 172.21.15.203:0/3976243110 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a00099f0 con 0x7f69b0063010 2021-01-06T22:19:03.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69ad59a700 1 -- 172.21.15.203:0/3976243110 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a0009d70 con 0x7f69b0063010 2021-01-06T22:19:03.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69af59e700 1 -- 172.21.15.203:0/3976243110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69b0063010 msgr2=0x7f69b014e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:03.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69af59e700 1 --2- 172.21.15.203:0/3976243110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69b0063010 0x7f69b014e770 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f69b00667e0 tx=0x7f69a0006ea0).stop 2021-01-06T22:19:03.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69af59e700 1 -- 172.21.15.203:0/3976243110 shutdown_connections 2021-01-06T22:19:03.598 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69af59e700 1 --2- 172.21.15.203:0/3976243110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69b0063010 0x7f69b014e770 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:03.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69af59e700 1 --2- 172.21.15.203:0/3976243110 >> v2:172.21.15.91:3300/0 conn(0x7f69b00633f0 0x7f69b0064b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:03.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69af59e700 1 -- 172.21.15.203:0/3976243110 >> 172.21.15.203:0/3976243110 conn(0x7f69b0067e20 msgr2=0x7f69b0068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:03.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69af59e700 1 -- 172.21.15.203:0/3976243110 shutdown_connections 2021-01-06T22:19:03.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.591+0000 7f69af59e700 1 -- 172.21.15.203:0/3976243110 wait complete. 2021-01-06T22:19:03.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69af59e700 1 Processor -- start 2021-01-06T22:19:03.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69af59e700 1 -- start start 2021-01-06T22:19:03.602 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69af59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69b0063010 0x7f69b0148970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:03.602 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69af59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69b00633f0 0x7f69b0148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:03.602 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69af59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69b00be1b0 0x7f69b015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:03.602 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69af59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b015c8d0 con 0x7f69b00be1b0 2021-01-06T22:19:03.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69af59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b015cb70 con 0x7f69b00633f0 2021-01-06T22:19:03.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69af59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b015ce10 con 0x7f69b0063010 2021-01-06T22:19:03.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69add9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69b00633f0 0x7f69b0148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:03.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69add9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69b00633f0 0x7f69b0148e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52802/0 (socket says 172.21.15.203:52802) 2021-01-06T22:19:03.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69add9b700 1 -- 172.21.15.203:0/2651948297 learned_addr learned my addr 172.21.15.203:0/2651948297 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:03.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69aed9d700 1 --2- 172.21.15.203:0/2651948297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69b00be1b0 0x7f69b015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:03.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69ae59c700 1 --2- 172.21.15.203:0/2651948297 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69b0063010 0x7f69b0148970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:03.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69add9b700 1 -- 172.21.15.203:0/2651948297 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69b0063010 msgr2=0x7f69b0148970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:03.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69add9b700 1 --2- 172.21.15.203:0/2651948297 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69b0063010 0x7f69b0148970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:03.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69add9b700 1 -- 172.21.15.203:0/2651948297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69b00be1b0 msgr2=0x7f69b015d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:03.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69add9b700 1 --2- 172.21.15.203:0/2651948297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69b00be1b0 0x7f69b015d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:03.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.595+0000 7f69add9b700 1 -- 172.21.15.203:0/2651948297 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b015d0b0 con 0x7f69b00633f0 2021-01-06T22:19:03.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.599+0000 7f69add9b700 1 --2- 172.21.15.203:0/2651948297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69b00633f0 0x7f69b0148e60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f69a4006750 tx=0x7f69a4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:03.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.599+0000 7f699e7fc700 1 -- 172.21.15.203:0/2651948297 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a4009790 con 0x7f69b00633f0 2021-01-06T22:19:03.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.599+0000 7f699e7fc700 1 -- 172.21.15.203:0/2651948297 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a40098f0 con 0x7f69b00633f0 2021-01-06T22:19:03.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.599+0000 7f69af59e700 1 -- 172.21.15.203:0/2651948297 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b015d350 con 0x7f69b00633f0 2021-01-06T22:19:03.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.599+0000 7f699e7fc700 1 -- 172.21.15.203:0/2651948297 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a4009c70 con 0x7f69b00633f0 2021-01-06T22:19:03.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.599+0000 7f69af59e700 1 -- 172.21.15.203:0/2651948297 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b0147f40 con 0x7f69b00633f0 2021-01-06T22:19:03.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.599+0000 7f69af59e700 1 -- 172.21.15.203:0/2651948297 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b015d4e0 con 0x7f69b00633f0 2021-01-06T22:19:03.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.599+0000 7f699e7fc700 1 -- 172.21.15.203:0/2651948297 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f69a4021060 con 0x7f69b00633f0 2021-01-06T22:19:03.607 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.599+0000 7f699e7fc700 1 --2- 172.21.15.203:0/2651948297 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f698c0563d0 0x7f698c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:03.607 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.599+0000 7f699e7fc700 1 -- 172.21.15.203:0/2651948297 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f69a4071840 con 0x7f69b00633f0 2021-01-06T22:19:03.608 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.599+0000 7f69ae59c700 1 --2- 172.21.15.203:0/2651948297 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f698c0563d0 0x7f698c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:03.608 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.599+0000 7f69ae59c700 1 --2- 172.21.15.203:0/2651948297 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f698c0563d0 0x7f698c058890 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f69a0007fd0 tx=0x7f69a0016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:03.608 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.607+0000 7f699e7fc700 1 -- 172.21.15.203:0/2651948297 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f69a4046d60 con 0x7f69b00633f0 2021-01-06T22:19:03.835 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.831+0000 7f69af59e700 1 -- 172.21.15.203:0/2651948297 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69b00450a0 con 0x7f69b00633f0 2021-01-06T22:19:03.836 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.835+0000 7f699e7fc700 1 -- 172.21.15.203:0/2651948297 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+3582 (secure 0 0 0) 0x7f69a401f090 con 0x7f69b00633f0 2021-01-06T22:19:03.836 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:19:03.836 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:19:03.435378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5539,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:starting","state_seq":2,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:19:03.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.835+0000 7f69af59e700 1 -- 172.21.15.203:0/2651948297 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f698c0563d0 msgr2=0x7f698c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:03.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.835+0000 7f69af59e700 1 --2- 172.21.15.203:0/2651948297 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f698c0563d0 0x7f698c058890 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f69a0007fd0 tx=0x7f69a0016000).stop 2021-01-06T22:19:03.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.835+0000 7f69af59e700 1 -- 172.21.15.203:0/2651948297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69b00633f0 msgr2=0x7f69b0148e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:03.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.835+0000 7f69af59e700 1 --2- 172.21.15.203:0/2651948297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69b00633f0 0x7f69b0148e60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f69a4006750 tx=0x7f69a4008040).stop 2021-01-06T22:19:03.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.835+0000 7f69af59e700 1 -- 172.21.15.203:0/2651948297 shutdown_connections 2021-01-06T22:19:03.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.835+0000 7f69af59e700 1 --2- 172.21.15.203:0/2651948297 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f698c0563d0 0x7f698c058890 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:03.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.835+0000 7f69af59e700 1 --2- 172.21.15.203:0/2651948297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69b00be1b0 0x7f69b015d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:03.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.835+0000 7f69af59e700 1 --2- 172.21.15.203:0/2651948297 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69b0063010 0x7f69b0148970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:03.840 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.835+0000 7f69af59e700 1 --2- 172.21.15.203:0/2651948297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69b00633f0 0x7f69b0148e60 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:03.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.835+0000 7f69af59e700 1 -- 172.21.15.203:0/2651948297 >> 172.21.15.203:0/2651948297 conn(0x7f69b0067e20 msgr2=0x7f69b00c0290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:03.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.835+0000 7f69af59e700 1 -- 172.21.15.203:0/2651948297 shutdown_connections 2021-01-06T22:19:03.841 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:03.835+0000 7f69af59e700 1 -- 172.21.15.203:0/2651948297 wait complete. 2021-01-06T22:19:03.842 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 76 2021-01-06T22:19:03.853 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 76, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 76, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5261": { "addr": "172.21.15.91:6837/2136238890", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6835", "nonce": 2136238890, "type": "v2" }, { "addr": "172.21.15.91:6837", "nonce": 2136238890, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5261, "incarnation": 76, "join_fscid": -1, "name": "f", "rank": 2, "state": "up:starting", "state_seq": 2 }, "gid_5539": { "addr": "172.21.15.203:6835/1801698156", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6834", "nonce": 1801698156, "type": "v2" }, { "addr": "172.21.15.203:6835", "nonce": 1801698156, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5539, "incarnation": 63, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 11 }, "gid_5632": { "addr": "172.21.15.203:6837/2424499644", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6836", "nonce": 2424499644, "type": "v2" }, { "addr": "172.21.15.203:6837", "nonce": 2424499644, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5632, "incarnation": 69, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 16 } }, "last_failure": 0, "last_failure_osd_epoch": 32, "max_file_size": 1099511627776, "max_mds": 3, "metadata_pool": 3, "modified": "2021-01-06T22:19:03.435378+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5539, "mds_1": 5632, "mds_2": 5261 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.91:6836/2428882934", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6834", "nonce": 2428882934, "type": "v2" }, { "addr": "172.21.15.91:6836", "nonce": 2428882934, "type": "v1" } ] }, "epoch": 26, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4724, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2021-01-06T22:19:05.855 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:19:05.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.983+0000 7f8824a1c700 1 Processor -- start 2021-01-06T22:19:05.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.983+0000 7f8824a1c700 1 -- start start 2021-01-06T22:19:05.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.983+0000 7f8824a1c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f88200633f0 0x7f882014e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:05.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.983+0000 7f8824a1c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f88200be1b0 0x7f8820147790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:05.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.983+0000 7f8824a1c700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88200be990 con 0x7f8820063010 2021-01-06T22:19:05.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.983+0000 7f8824a1c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f882014ec60 con 0x7f88200633f0 2021-01-06T22:19:05.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.983+0000 7f8824a1c700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88200be6f0 con 0x7f88200be1b0 2021-01-06T22:19:05.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.983+0000 7f8817fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f88200633f0 0x7f882014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:05.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.983+0000 7f8817fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f88200633f0 0x7f882014e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52812/0 (socket says 172.21.15.203:52812) 2021-01-06T22:19:05.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.983+0000 7f8817fff700 1 -- 172.21.15.203:0/273706043 learned_addr learned my addr 172.21.15.203:0/273706043 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:05.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.983+0000 7f881ffff700 1 --2- 172.21.15.203:0/273706043 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f88200be1b0 0x7f8820147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:05.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.983+0000 7f8817fff700 1 -- 172.21.15.203:0/273706043 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f88200be1b0 msgr2=0x7f8820147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:05.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.983+0000 7f8817fff700 1 --2- 172.21.15.203:0/273706043 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f88200be1b0 0x7f8820147790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:05.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.983+0000 7f8817fff700 1 -- 172.21.15.203:0/273706043 >> v1:172.21.15.91:6789/0 conn(0x7f8820063010 legacy=0x7f8820064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:05.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.983+0000 7f8817fff700 1 -- 172.21.15.203:0/273706043 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88200c1050 con 0x7f88200633f0 2021-01-06T22:19:05.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.983+0000 7f8817fff700 1 --2- 172.21.15.203:0/273706043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88200633f0 0x7f882014e770 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f880800b9e0 tx=0x7f8808006ea0).ready entity=mon.1 client_cookie=9535bc6820f412a8 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:05.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f881effd700 1 -- 172.21.15.203:0/273706043 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88080098a0 con 0x7f88200633f0 2021-01-06T22:19:05.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f881effd700 1 -- 172.21.15.203:0/273706043 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8808009a00 con 0x7f88200633f0 2021-01-06T22:19:05.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f881effd700 1 -- 172.21.15.203:0/273706043 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808009d80 con 0x7f88200633f0 2021-01-06T22:19:05.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8824a1c700 1 -- 172.21.15.203:0/273706043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88200633f0 msgr2=0x7f882014e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:05.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8824a1c700 1 --2- 172.21.15.203:0/273706043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88200633f0 0x7f882014e770 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f880800b9e0 tx=0x7f8808006ea0).stop 2021-01-06T22:19:05.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8824a1c700 1 -- 172.21.15.203:0/273706043 shutdown_connections 2021-01-06T22:19:05.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8824a1c700 1 --2- 172.21.15.203:0/273706043 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f88200be1b0 0x7f8820147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:05.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8824a1c700 1 --2- 172.21.15.203:0/273706043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88200633f0 0x7f882014e770 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:05.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8824a1c700 1 -- 172.21.15.203:0/273706043 >> 172.21.15.203:0/273706043 conn(0x7f8820067e20 msgr2=0x7f8820068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:05.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8824a1c700 1 -- 172.21.15.203:0/273706043 shutdown_connections 2021-01-06T22:19:05.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8824a1c700 1 -- 172.21.15.203:0/273706043 wait complete. 2021-01-06T22:19:05.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8824a1c700 1 Processor -- start 2021-01-06T22:19:05.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8824a1c700 1 -- start start 2021-01-06T22:19:05.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8824a1c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8820063010 0x7f88200c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:05.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8824a1c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88200633f0 0x7f882014ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:05.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8824a1c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f88200be1b0 0x7f882015c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:05.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8824a1c700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8820148ef0 con 0x7f88200be1b0 2021-01-06T22:19:05.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8824a1c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882014a200 con 0x7f88200633f0 2021-01-06T22:19:05.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8824a1c700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8820148710 con 0x7f8820063010 2021-01-06T22:19:05.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8817fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88200633f0 0x7f882014ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:05.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8817fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88200633f0 0x7f882014ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52818/0 (socket says 172.21.15.203:52818) 2021-01-06T22:19:05.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f8817fff700 1 -- 172.21.15.203:0/2639833171 learned_addr learned my addr 172.21.15.203:0/2639833171 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:05.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.987+0000 7f881f7fe700 1 --2- 172.21.15.203:0/2639833171 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8820063010 0x7f88200c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:05.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f881ffff700 1 --2- 172.21.15.203:0/2639833171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f88200be1b0 0x7f882015c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:06.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f8817fff700 1 -- 172.21.15.203:0/2639833171 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8820063010 msgr2=0x7f88200c0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:06.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f8817fff700 1 --2- 172.21.15.203:0/2639833171 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8820063010 0x7f88200c0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:06.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f8817fff700 1 -- 172.21.15.203:0/2639833171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f88200be1b0 msgr2=0x7f882015c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:06.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f8817fff700 1 --2- 172.21.15.203:0/2639833171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f88200be1b0 0x7f882015c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:06.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f8817fff700 1 -- 172.21.15.203:0/2639833171 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88201489b0 con 0x7f88200633f0 2021-01-06T22:19:06.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f881f7fe700 1 --2- 172.21.15.203:0/2639833171 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8820063010 0x7f88200c0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:19:06.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f8817fff700 1 --2- 172.21.15.203:0/2639833171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88200633f0 0x7f882014ad90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f8808010000 tx=0x7f8808016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:06.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f88177fe700 1 -- 172.21.15.203:0/2639833171 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808012e40 con 0x7f88200633f0 2021-01-06T22:19:06.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f88177fe700 1 -- 172.21.15.203:0/2639833171 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f880801c420 con 0x7f88200633f0 2021-01-06T22:19:06.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f8824a1c700 1 -- 172.21.15.203:0/2639833171 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8820148c50 con 0x7f88200633f0 2021-01-06T22:19:06.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f88177fe700 1 -- 172.21.15.203:0/2639833171 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880801c7c0 con 0x7f88200633f0 2021-01-06T22:19:06.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f8824a1c700 1 -- 172.21.15.203:0/2639833171 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882014a690 con 0x7f88200633f0 2021-01-06T22:19:06.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f88177fe700 1 -- 172.21.15.203:0/2639833171 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f880800f070 con 0x7f88200633f0 2021-01-06T22:19:06.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f8824a1c700 1 -- 172.21.15.203:0/2639833171 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88200450a0 con 0x7f88200633f0 2021-01-06T22:19:06.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f88177fe700 1 --2- 172.21.15.203:0/2639833171 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f88040563d0 0x7f8804058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:06.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f88177fe700 1 -- 172.21.15.203:0/2639833171 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f8808071090 con 0x7f88200633f0 2021-01-06T22:19:06.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.991+0000 7f881f7fe700 1 --2- 172.21.15.203:0/2639833171 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f88040563d0 0x7f8804058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:06.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.995+0000 7f881f7fe700 1 --2- 172.21.15.203:0/2639833171 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f88040563d0 0x7f8804058890 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f8810001530 tx=0x7f8810009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:06.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:05.999+0000 7f88177fe700 1 -- 172.21.15.203:0/2639833171 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f8808046610 con 0x7f88200633f0 2021-01-06T22:19:06.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.227+0000 7f8824a1c700 1 -- 172.21.15.203:0/2639833171 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88200b7fd0 con 0x7f88200633f0 2021-01-06T22:19:06.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.227+0000 7f88177fe700 1 -- 172.21.15.203:0/2639833171 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+3581 (secure 0 0 0) 0x7f8808049c30 con 0x7f88200633f0 2021-01-06T22:19:06.229 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:19:06.229 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:19:04.439746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5539,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:active","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:19:06.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.227+0000 7f8824a1c700 1 -- 172.21.15.203:0/2639833171 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f88040563d0 msgr2=0x7f8804058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:06.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.227+0000 7f8824a1c700 1 --2- 172.21.15.203:0/2639833171 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f88040563d0 0x7f8804058890 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f8810001530 tx=0x7f8810009040).stop 2021-01-06T22:19:06.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.227+0000 7f8824a1c700 1 -- 172.21.15.203:0/2639833171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88200633f0 msgr2=0x7f882014ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:06.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.227+0000 7f8824a1c700 1 --2- 172.21.15.203:0/2639833171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88200633f0 0x7f882014ad90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f8808010000 tx=0x7f8808016000).stop 2021-01-06T22:19:06.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.227+0000 7f8824a1c700 1 -- 172.21.15.203:0/2639833171 shutdown_connections 2021-01-06T22:19:06.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.227+0000 7f8824a1c700 1 --2- 172.21.15.203:0/2639833171 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f88040563d0 0x7f8804058890 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:06.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.227+0000 7f8824a1c700 1 --2- 172.21.15.203:0/2639833171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f88200be1b0 0x7f882015c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:06.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.227+0000 7f8824a1c700 1 --2- 172.21.15.203:0/2639833171 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8820063010 0x7f88200c0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:06.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.227+0000 7f8824a1c700 1 --2- 172.21.15.203:0/2639833171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88200633f0 0x7f882014ad90 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:06.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.227+0000 7f8824a1c700 1 -- 172.21.15.203:0/2639833171 >> 172.21.15.203:0/2639833171 conn(0x7f8820067e20 msgr2=0x7f882014f560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:06.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.227+0000 7f8824a1c700 1 -- 172.21.15.203:0/2639833171 shutdown_connections 2021-01-06T22:19:06.234 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.231+0000 7f8824a1c700 1 -- 172.21.15.203:0/2639833171 wait complete. 2021-01-06T22:19:06.234 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 77 2021-01-06T22:19:06.246 INFO:tasks.mds_thrash.fs.[cephfs]:mds cluster has 3 alive and active, now stable! 2021-01-06T22:19:06.246 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=0) 2021-01-06T22:19:06.247 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-01-06T22:19:06.247 INFO:teuthology.orchestra.run:waiting for 300 2021-01-06T22:19:06.258 INFO:tasks.ceph.mds.c:Stopped 2021-01-06T22:19:06.258 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:19:06.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.419+0000 7f521bfff700 1 Processor -- start 2021-01-06T22:19:06.422 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.419+0000 7f521bfff700 1 -- start start 2021-01-06T22:19:06.425 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.419+0000 7f521bfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f521c1513f0 0x7f521c0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:06.425 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.419+0000 7f521bfff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f521c14ad90 0x7f521c14b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:06.425 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.419+0000 7f521bfff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f521c0bacb0 con 0x7f521c0bccb0 2021-01-06T22:19:06.426 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.419+0000 7f521bfff700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521c0bd160 con 0x7f521c14ad90 2021-01-06T22:19:06.426 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.419+0000 7f521bfff700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521c14b680 con 0x7f521c1513f0 2021-01-06T22:19:06.426 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.419+0000 7f521affd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f521c1513f0 0x7f521c0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:06.427 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.419+0000 7f521affd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f521c1513f0 0x7f521c0c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52826/0 (socket says 172.21.15.203:52826) 2021-01-06T22:19:06.427 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.419+0000 7f521affd700 1 -- 172.21.15.203:0/1527263278 learned_addr learned my addr 172.21.15.203:0/1527263278 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:06.427 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.419+0000 7f521b7fe700 1 --2- 172.21.15.203:0/1527263278 >> v2:172.21.15.91:3300/0 conn(0x7f521c14ad90 0x7f521c14b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:06.427 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521affd700 1 -- 172.21.15.203:0/1527263278 >> v2:172.21.15.91:3300/0 conn(0x7f521c14ad90 msgr2=0x7f521c14b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:06.427 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521affd700 1 --2- 172.21.15.203:0/1527263278 >> v2:172.21.15.91:3300/0 conn(0x7f521c14ad90 0x7f521c14b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:06.428 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521affd700 1 -- 172.21.15.203:0/1527263278 >> v1:172.21.15.91:6789/0 conn(0x7f521c0bccb0 legacy=0x7f521c1517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:06.428 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521affd700 1 -- 172.21.15.203:0/1527263278 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c152000 con 0x7f521c1513f0 2021-01-06T22:19:06.428 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521affd700 1 --2- 172.21.15.203:0/1527263278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f521c1513f0 0x7f521c0c0fa0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f521c061ee0 tx=0x7f520c006ea0).ready entity=mon.1 client_cookie=acdda764e4bdd0b4 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:06.428 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521a7fc700 1 -- 172.21.15.203:0/1527263278 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c009890 con 0x7f521c1513f0 2021-01-06T22:19:06.429 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521a7fc700 1 -- 172.21.15.203:0/1527263278 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c0099f0 con 0x7f521c1513f0 2021-01-06T22:19:06.429 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521a7fc700 1 -- 172.21.15.203:0/1527263278 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c009d70 con 0x7f521c1513f0 2021-01-06T22:19:06.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521bfff700 1 -- 172.21.15.203:0/1527263278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f521c1513f0 msgr2=0x7f521c0c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:06.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521bfff700 1 --2- 172.21.15.203:0/1527263278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f521c1513f0 0x7f521c0c0fa0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f521c061ee0 tx=0x7f520c006ea0).stop 2021-01-06T22:19:06.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521bfff700 1 -- 172.21.15.203:0/1527263278 shutdown_connections 2021-01-06T22:19:06.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521bfff700 1 --2- 172.21.15.203:0/1527263278 >> v2:172.21.15.91:3300/0 conn(0x7f521c14ad90 0x7f521c14b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:06.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521bfff700 1 --2- 172.21.15.203:0/1527263278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f521c1513f0 0x7f521c0c0fa0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:06.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521bfff700 1 -- 172.21.15.203:0/1527263278 >> 172.21.15.203:0/1527263278 conn(0x7f521c0667b0 msgr2=0x7f521c0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:06.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521bfff700 1 -- 172.21.15.203:0/1527263278 shutdown_connections 2021-01-06T22:19:06.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521bfff700 1 -- 172.21.15.203:0/1527263278 wait complete. 2021-01-06T22:19:06.433 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521bfff700 1 Processor -- start 2021-01-06T22:19:06.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.423+0000 7f521bfff700 1 -- start start 2021-01-06T22:19:06.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f521c0bccb0 0x7f521c0b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:06.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521bfff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f521c14ad90 0x7f521c15e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:06.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521bfff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f521c1513f0 0x7f521c161770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:06.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521bfff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0b2670 con 0x7f521c14ad90 2021-01-06T22:19:06.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521bfff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0b28a0 con 0x7f521c0bccb0 2021-01-06T22:19:06.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521bfff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0b2b40 con 0x7f521c1513f0 2021-01-06T22:19:06.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f521c0bccb0 0x7f521c0b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:06.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f521c0bccb0 0x7f521c0b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52832/0 (socket says 172.21.15.203:52832) 2021-01-06T22:19:06.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521affd700 1 -- 172.21.15.203:0/1586178585 learned_addr learned my addr 172.21.15.203:0/1586178585 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:06.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f52127fc700 1 --2- 172.21.15.203:0/1586178585 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f521c14ad90 0x7f521c15e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:06.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521b7fe700 1 --2- 172.21.15.203:0/1586178585 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f521c1513f0 0x7f521c161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:06.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521affd700 1 -- 172.21.15.203:0/1586178585 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f521c1513f0 msgr2=0x7f521c161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:06.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521affd700 1 --2- 172.21.15.203:0/1586178585 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f521c1513f0 0x7f521c161770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:06.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521affd700 1 -- 172.21.15.203:0/1586178585 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f521c14ad90 msgr2=0x7f521c15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:06.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521affd700 1 --2- 172.21.15.203:0/1586178585 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f521c14ad90 0x7f521c15e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:06.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521affd700 1 -- 172.21.15.203:0/1586178585 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c15d920 con 0x7f521c0bccb0 2021-01-06T22:19:06.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521b7fe700 1 --2- 172.21.15.203:0/1586178585 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f521c1513f0 0x7f521c161770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:19:06.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521affd700 1 --2- 172.21.15.203:0/1586178585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f521c0bccb0 0x7f521c0b3060 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f521c14b7c0 tx=0x7f520c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:06.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f52137fe700 1 -- 172.21.15.203:0/1586178585 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c012d90 con 0x7f521c0bccb0 2021-01-06T22:19:06.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f52137fe700 1 -- 172.21.15.203:0/1586178585 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c01c420 con 0x7f521c0bccb0 2021-01-06T22:19:06.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521bfff700 1 -- 172.21.15.203:0/1586178585 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c15dbc0 con 0x7f521c0bccb0 2021-01-06T22:19:06.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f52137fe700 1 -- 172.21.15.203:0/1586178585 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c01c7e0 con 0x7f521c0bccb0 2021-01-06T22:19:06.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521bfff700 1 -- 172.21.15.203:0/1586178585 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521c0b2de0 con 0x7f521c0bccb0 2021-01-06T22:19:06.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f521bfff700 1 -- 172.21.15.203:0/1586178585 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521c0450a0 con 0x7f521c0bccb0 2021-01-06T22:19:06.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.427+0000 7f52137fe700 1 -- 172.21.15.203:0/1586178585 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f520c00f070 con 0x7f521c0bccb0 2021-01-06T22:19:06.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.431+0000 7f52137fe700 1 --2- 172.21.15.203:0/1586178585 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f52000563d0 0x7f5200058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:06.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.431+0000 7f52137fe700 1 -- 172.21.15.203:0/1586178585 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f520c070d70 con 0x7f521c0bccb0 2021-01-06T22:19:06.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.431+0000 7f52127fc700 1 --2- 172.21.15.203:0/1586178585 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f52000563d0 0x7f5200058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:06.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.431+0000 7f52127fc700 1 --2- 172.21.15.203:0/1586178585 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f52000563d0 0x7f5200058890 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f521c061370 tx=0x7f5204007b60).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:06.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.435+0000 7f52137fe700 1 -- 172.21.15.203:0/1586178585 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f520c0462f0 con 0x7f521c0bccb0 2021-01-06T22:19:06.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.643+0000 7f521bfff700 1 -- 172.21.15.203:0/1586178585 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f521c0bacb0 con 0x7f521c0bccb0 2021-01-06T22:19:06.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.643+0000 7f52137fe700 1 -- 172.21.15.203:0/1586178585 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+3581 (secure 0 0 0) 0x7f521c0bacb0 con 0x7f521c0bccb0 2021-01-06T22:19:06.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.647+0000 7f521bfff700 1 -- 172.21.15.203:0/1586178585 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f52000563d0 msgr2=0x7f5200058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:06.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.647+0000 7f521bfff700 1 --2- 172.21.15.203:0/1586178585 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f52000563d0 0x7f5200058890 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f521c061370 tx=0x7f5204007b60).stop 2021-01-06T22:19:06.774 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.647+0000 7f521bfff700 1 -- 172.21.15.203:0/1586178585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f521c0bccb0 msgr2=0x7f521c0b3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:06.774 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.647+0000 7f521bfff700 1 --2- 172.21.15.203:0/1586178585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f521c0bccb0 0x7f521c0b3060 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f521c14b7c0 tx=0x7f520c016000).stop 2021-01-06T22:19:06.774 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.647+0000 7f521bfff700 1 -- 172.21.15.203:0/1586178585 shutdown_connections 2021-01-06T22:19:06.774 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.647+0000 7f521bfff700 1 --2- 172.21.15.203:0/1586178585 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f52000563d0 0x7f5200058890 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:06.774 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.647+0000 7f521bfff700 1 --2- 172.21.15.203:0/1586178585 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f521c1513f0 0x7f521c161770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:06.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.647+0000 7f521bfff700 1 --2- 172.21.15.203:0/1586178585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f521c0bccb0 0x7f521c0b3060 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:06.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.647+0000 7f521bfff700 1 --2- 172.21.15.203:0/1586178585 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f521c14ad90 0x7f521c15e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:06.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.647+0000 7f521bfff700 1 -- 172.21.15.203:0/1586178585 >> 172.21.15.203:0/1586178585 conn(0x7f521c0667b0 msgr2=0x7f521c14a610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:06.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.647+0000 7f521bfff700 1 -- 172.21.15.203:0/1586178585 shutdown_connections 2021-01-06T22:19:06.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:06.647+0000 7f521bfff700 1 -- 172.21.15.203:0/1586178585 wait complete. 2021-01-06T22:19:06.776 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 77 2021-01-06T22:19:06.776 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:19:06.776 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:19:04.439746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5539,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:active","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:19:06.777 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:19:08.778 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:19:08.914 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.911+0000 7f09c69b5700 1 Processor -- start 2021-01-06T22:19:08.915 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.911+0000 7f09c69b5700 1 -- start start 2021-01-06T22:19:08.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c69b5700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09c00be730 0x7f09c0065690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:08.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c69b5700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c00bba40 con 0x7f09c00be350 2021-01-06T22:19:08.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c69b5700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c00bbca0 con 0x7f09c0062e70 2021-01-06T22:19:08.916 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c69b5700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c01563d0 con 0x7f09c00be730 2021-01-06T22:19:08.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c61b4700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f09c0062e70 0x7f09c014ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51452/0 (socket says 172.21.15.203:51452) 2021-01-06T22:19:08.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c61b4700 1 -- 172.21.15.203:0/2233187096 learned_addr learned my addr 172.21.15.203:0/2233187096 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:19:08.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c51b2700 1 --2- 172.21.15.203:0/2233187096 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09c00be730 0x7f09c0065690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:08.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 -- 172.21.15.203:0/2233187096 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800778449 0 0) 0x7f09c00bbca0 con 0x7f09c0062e70 2021-01-06T22:19:08.917 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 -- 172.21.15.203:0/2233187096 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09a8003230 con 0x7f09c0062e70 2021-01-06T22:19:08.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 -- 172.21.15.203:0/2233187096 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4263984483 0 0) 0x7f09a8003230 con 0x7f09c0062e70 2021-01-06T22:19:08.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 -- 172.21.15.203:0/2233187096 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09c00be730 msgr2=0x7f09c0065690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:08.920 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 --2- 172.21.15.203:0/2233187096 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09c00be730 0x7f09c0065690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:08.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 -- 172.21.15.203:0/2233187096 >> v1:172.21.15.91:6789/0 conn(0x7f09c00be350 legacy=0x7f09c0152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:08.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 -- 172.21.15.203:0/2233187096 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c00bbf40 con 0x7f09c0062e70 2021-01-06T22:19:08.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 -- 172.21.15.203:0/2233187096 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f09b8001860 con 0x7f09c0062e70 2021-01-06T22:19:08.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 -- 172.21.15.203:0/2233187096 >> v1:172.21.15.203:6789/0 conn(0x7f09c0062e70 legacy=0x7f09c014ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:08.921 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 --2- 172.21.15.203:0/2233187096 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09a8003720 0x7f09a8003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:08.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 --2- 172.21.15.203:0/2233187096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a80040a0 0x7f09a8004570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:08.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 --2- 172.21.15.203:0/2233187096 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09a8004ba0 0x7f09a800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:08.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 -- 172.21.15.203:0/2233187096 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c01563d0 con 0x7f09a8004ba0 2021-01-06T22:19:08.922 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 -- 172.21.15.203:0/2233187096 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a80051a0 con 0x7f09a80040a0 2021-01-06T22:19:08.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 -- 172.21.15.203:0/2233187096 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a8005360 con 0x7f09a8003720 2021-01-06T22:19:08.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c51b2700 1 --2- 172.21.15.203:0/2233187096 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09c00be730 0x7f09c0065690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:19:08.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c51b2700 1 --2- 172.21.15.203:0/2233187096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a80040a0 0x7f09a8004570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:08.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c59b3700 1 --2- 172.21.15.203:0/2233187096 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09a8003720 0x7f09a8003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:08.923 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c61b4700 1 --2- 172.21.15.203:0/2233187096 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09a8004ba0 0x7f09a800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:08.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c51b2700 1 -- 172.21.15.203:0/2233187096 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09a8003720 msgr2=0x7f09a8003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:08.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c51b2700 1 --2- 172.21.15.203:0/2233187096 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09a8003720 0x7f09a8003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:08.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c51b2700 1 -- 172.21.15.203:0/2233187096 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09a8004ba0 msgr2=0x7f09a800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:08.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c51b2700 1 --2- 172.21.15.203:0/2233187096 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09a8004ba0 0x7f09a800b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:08.924 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c51b2700 1 -- 172.21.15.203:0/2233187096 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a8003230 con 0x7f09a80040a0 2021-01-06T22:19:08.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c51b2700 1 --2- 172.21.15.203:0/2233187096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a80040a0 0x7f09a8004570 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f09c00beb90 tx=0x7f09b0006de0).ready entity=mon.1 client_cookie=be3bcc4496d32b45 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:08.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 -- 172.21.15.203:0/2233187096 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0016070 con 0x7f09a80040a0 2021-01-06T22:19:08.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 -- 172.21.15.203:0/2233187096 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b000e8a0 con 0x7f09a80040a0 2021-01-06T22:19:08.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c49b1700 1 -- 172.21.15.203:0/2233187096 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b000ec20 con 0x7f09a80040a0 2021-01-06T22:19:08.925 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c69b5700 1 -- 172.21.15.203:0/2233187096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a80040a0 msgr2=0x7f09a8004570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:08.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c69b5700 1 --2- 172.21.15.203:0/2233187096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a80040a0 0x7f09a8004570 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f09c00beb90 tx=0x7f09b0006de0).stop 2021-01-06T22:19:08.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c61b4700 1 -- 172.21.15.203:0/2233187096 reap_dead start 2021-01-06T22:19:08.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c69b5700 1 -- 172.21.15.203:0/2233187096 shutdown_connections 2021-01-06T22:19:08.926 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c69b5700 1 -- 172.21.15.203:0/2233187096 >> 172.21.15.203:0/2233187096 conn(0x7f09c0067e20 msgr2=0x7f09c0068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:08.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c69b5700 1 -- 172.21.15.203:0/2233187096 shutdown_connections 2021-01-06T22:19:08.927 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.915+0000 7f09c69b5700 1 -- 172.21.15.203:0/2233187096 wait complete. 2021-01-06T22:19:08.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.919+0000 7f09c69b5700 1 Processor -- start 2021-01-06T22:19:08.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.919+0000 7f09c69b5700 1 -- start start 2021-01-06T22:19:08.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.919+0000 7f09c69b5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09c0148d00 0x7f09c00b98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:08.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.919+0000 7f09c69b5700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09c00c1cb0 0x7f09c00acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:08.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.919+0000 7f09c69b5700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09c01490e0 0x7f09c01547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:08.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.919+0000 7f09c69b5700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00ae5d0 con 0x7f09c00c1cb0 2021-01-06T22:19:08.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.919+0000 7f09c69b5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00b9e90 con 0x7f09c0148d00 2021-01-06T22:19:08.930 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.919+0000 7f09c69b5700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c0161c70 con 0x7f09c01490e0 2021-01-06T22:19:08.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.919+0000 7f09c59b3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09c0148d00 0x7f09c00b98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:08.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.919+0000 7f09c59b3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09c0148d00 0x7f09c00b98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52866/0 (socket says 172.21.15.203:52866) 2021-01-06T22:19:08.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.919+0000 7f09c59b3700 1 -- 172.21.15.203:0/3567482875 learned_addr learned my addr 172.21.15.203:0/3567482875 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:08.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.919+0000 7f09c61b4700 1 --2- 172.21.15.203:0/3567482875 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09c01490e0 0x7f09c01547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:08.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.919+0000 7f09c51b2700 1 --2- 172.21.15.203:0/3567482875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09c00c1cb0 0x7f09c00acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:08.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09c59b3700 1 -- 172.21.15.203:0/3567482875 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09c01490e0 msgr2=0x7f09c01547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:08.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09c59b3700 1 --2- 172.21.15.203:0/3567482875 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09c01490e0 0x7f09c01547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:08.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09c59b3700 1 -- 172.21.15.203:0/3567482875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09c00c1cb0 msgr2=0x7f09c00acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:08.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09c59b3700 1 --2- 172.21.15.203:0/3567482875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09c00c1cb0 0x7f09c00acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:08.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09c59b3700 1 -- 172.21.15.203:0/3567482875 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c0161f10 con 0x7f09c0148d00 2021-01-06T22:19:08.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09c51b2700 1 --2- 172.21.15.203:0/3567482875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09c00c1cb0 0x7f09c00acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:19:08.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09c59b3700 1 --2- 172.21.15.203:0/3567482875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09c0148d00 0x7f09c00b98e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f09bc00fb50 tx=0x7f09bc007b80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:08.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09adffb700 1 -- 172.21.15.203:0/3567482875 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc0029f0 con 0x7f09c0148d00 2021-01-06T22:19:08.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09adffb700 1 -- 172.21.15.203:0/3567482875 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc002b50 con 0x7f09c0148d00 2021-01-06T22:19:08.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09c69b5700 1 -- 172.21.15.203:0/3567482875 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c01621b0 con 0x7f09c0148d00 2021-01-06T22:19:08.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09adffb700 1 -- 172.21.15.203:0/3567482875 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc014450 con 0x7f09c0148d00 2021-01-06T22:19:08.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09c69b5700 1 -- 172.21.15.203:0/3567482875 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c00ad410 con 0x7f09c0148d00 2021-01-06T22:19:08.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09c69b5700 1 -- 172.21.15.203:0/3567482875 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c00450a0 con 0x7f09c0148d00 2021-01-06T22:19:08.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09adffb700 1 -- 172.21.15.203:0/3567482875 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f09bc01b030 con 0x7f09c0148d00 2021-01-06T22:19:08.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09adffb700 1 --2- 172.21.15.203:0/3567482875 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f09a00563d0 0x7f09a0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:08.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09adffb700 1 -- 172.21.15.203:0/3567482875 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f09bc06f610 con 0x7f09c0148d00 2021-01-06T22:19:08.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09c51b2700 1 --2- 172.21.15.203:0/3567482875 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f09a00563d0 0x7f09a0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:08.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.923+0000 7f09c51b2700 1 --2- 172.21.15.203:0/3567482875 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f09a00563d0 0x7f09a0058890 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f09c005fb70 tx=0x7f09b0018000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:08.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:08.931+0000 7f09adffb700 1 -- 172.21.15.203:0/3567482875 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f09bc044c40 con 0x7f09c0148d00 2021-01-06T22:19:09.137 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:09.135+0000 7f09c69b5700 1 -- 172.21.15.203:0/3567482875 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09c0155280 con 0x7f09c0148d00 2021-01-06T22:19:09.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:09.135+0000 7f09adffb700 1 -- 172.21.15.203:0/3567482875 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+3581 (secure 0 0 0) 0x7f09bc048260 con 0x7f09c0148d00 2021-01-06T22:19:09.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:09.139+0000 7f09c69b5700 1 -- 172.21.15.203:0/3567482875 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f09a00563d0 msgr2=0x7f09a0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:09.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:09.139+0000 7f09c69b5700 1 --2- 172.21.15.203:0/3567482875 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f09a00563d0 0x7f09a0058890 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f09c005fb70 tx=0x7f09b0018000).stop 2021-01-06T22:19:09.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:09.139+0000 7f09c69b5700 1 -- 172.21.15.203:0/3567482875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09c0148d00 msgr2=0x7f09c00b98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:09.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:09.139+0000 7f09c69b5700 1 --2- 172.21.15.203:0/3567482875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09c0148d00 0x7f09c00b98e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f09bc00fb50 tx=0x7f09bc007b80).stop 2021-01-06T22:19:09.233 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:09.139+0000 7f09c69b5700 1 -- 172.21.15.203:0/3567482875 shutdown_connections 2021-01-06T22:19:09.234 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:09.139+0000 7f09c69b5700 1 --2- 172.21.15.203:0/3567482875 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f09a00563d0 0x7f09a0058890 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:09.234 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:09.139+0000 7f09c69b5700 1 --2- 172.21.15.203:0/3567482875 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f09c01490e0 0x7f09c01547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:09.234 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:09.139+0000 7f09c69b5700 1 --2- 172.21.15.203:0/3567482875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09c0148d00 0x7f09c00b98e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:09.234 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:09.139+0000 7f09c69b5700 1 --2- 172.21.15.203:0/3567482875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09c00c1cb0 0x7f09c00acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:09.235 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:09.139+0000 7f09c69b5700 1 -- 172.21.15.203:0/3567482875 >> 172.21.15.203:0/3567482875 conn(0x7f09c0067e20 msgr2=0x7f09c00602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:09.235 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:09.139+0000 7f09c69b5700 1 -- 172.21.15.203:0/3567482875 shutdown_connections 2021-01-06T22:19:09.235 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:09.139+0000 7f09c69b5700 1 -- 172.21.15.203:0/3567482875 wait complete. 2021-01-06T22:19:09.235 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 77 2021-01-06T22:19:09.237 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:19:09.237 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:19:04.439746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5539,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:active","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:19:09.238 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:19:11.240 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:19:11.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642f59e700 1 Processor -- start 2021-01-06T22:19:11.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642f59e700 1 -- start start 2021-01-06T22:19:11.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642f59e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f64300c2380 0x7f64300a7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:11.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642f59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64300bd9d0 con 0x7f64300be6d0 2021-01-06T22:19:11.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642f59e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64300a7710 con 0x7f64300c2760 2021-01-06T22:19:11.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642f59e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64300a79b0 con 0x7f64300c2380 2021-01-06T22:19:11.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f6427fff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f64300c2760 0x7f64300bd2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51474/0 (socket says 172.21.15.203:51474) 2021-01-06T22:19:11.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f6427fff700 1 -- 172.21.15.203:0/3733455287 learned_addr learned my addr 172.21.15.203:0/3733455287 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:19:11.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642e59c700 1 --2- 172.21.15.203:0/3733455287 >> v2:172.21.15.91:3300/0 conn(0x7f64300c2380 0x7f64300a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:11.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642dd9b700 1 -- 172.21.15.203:0/3733455287 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445669325 0 0) 0x7f64300a7710 con 0x7f64300c2760 2021-01-06T22:19:11.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642dd9b700 1 -- 172.21.15.203:0/3733455287 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6410003230 con 0x7f64300c2760 2021-01-06T22:19:11.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642dd9b700 1 -- 172.21.15.203:0/3733455287 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1201861387 0 0) 0x7f6410003230 con 0x7f64300c2760 2021-01-06T22:19:11.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642dd9b700 1 -- 172.21.15.203:0/3733455287 >> v2:172.21.15.91:3300/0 conn(0x7f64300c2380 msgr2=0x7f64300a7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:11.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642dd9b700 1 --2- 172.21.15.203:0/3733455287 >> v2:172.21.15.91:3300/0 conn(0x7f64300c2380 0x7f64300a7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:11.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642dd9b700 1 -- 172.21.15.203:0/3733455287 >> v1:172.21.15.91:6789/0 conn(0x7f64300be6d0 legacy=0x7f643014e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:11.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642dd9b700 1 -- 172.21.15.203:0/3733455287 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64300a7c50 con 0x7f64300c2760 2021-01-06T22:19:11.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642dd9b700 1 -- 172.21.15.203:0/3733455287 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f64180014e0 con 0x7f64300c2760 2021-01-06T22:19:11.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642dd9b700 1 -- 172.21.15.203:0/3733455287 >> v1:172.21.15.203:6789/0 conn(0x7f64300c2760 legacy=0x7f64300bd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:11.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642dd9b700 1 --2- 172.21.15.203:0/3733455287 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6410003720 0x7f6410003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:11.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642dd9b700 1 --2- 172.21.15.203:0/3733455287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64100044d0 0x7f6410004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:11.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642dd9b700 1 --2- 172.21.15.203:0/3733455287 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6410004e40 0x7f641000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:11.389 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642dd9b700 1 -- 172.21.15.203:0/3733455287 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64300a79b0 con 0x7f6410003720 2021-01-06T22:19:11.389 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642dd9b700 1 -- 172.21.15.203:0/3733455287 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64100041c0 con 0x7f64100044d0 2021-01-06T22:19:11.389 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642dd9b700 1 -- 172.21.15.203:0/3733455287 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641000bc10 con 0x7f6410004e40 2021-01-06T22:19:11.389 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f6427fff700 1 --2- 172.21.15.203:0/3733455287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64100044d0 0x7f6410004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:11.389 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642e59c700 1 --2- 172.21.15.203:0/3733455287 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6410003720 0x7f6410003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:11.390 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642ed9d700 1 --2- 172.21.15.203:0/3733455287 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6410004e40 0x7f641000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:11.390 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f6427fff700 1 -- 172.21.15.203:0/3733455287 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6410004e40 msgr2=0x7f641000b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:11.390 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f6427fff700 1 --2- 172.21.15.203:0/3733455287 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6410004e40 0x7f641000b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:11.390 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f6427fff700 1 -- 172.21.15.203:0/3733455287 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6410003720 msgr2=0x7f6410003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:11.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f6427fff700 1 --2- 172.21.15.203:0/3733455287 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6410003720 0x7f6410003b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:11.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f6427fff700 1 -- 172.21.15.203:0/3733455287 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6410003230 con 0x7f64100044d0 2021-01-06T22:19:11.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642ed9d700 1 --2- 172.21.15.203:0/3733455287 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6410004e40 0x7f641000b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:19:11.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642ed9d700 1 -- 172.21.15.203:0/3733455287 reap_dead start 2021-01-06T22:19:11.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f6427fff700 1 --2- 172.21.15.203:0/3733455287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64100044d0 0x7f6410004950 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f6418004dd0 tx=0x7f64300bdf50).ready entity=mon.1 client_cookie=686bebef68ef7516 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:11.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.379+0000 7f642dd9b700 1 -- 172.21.15.203:0/3733455287 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418008170 con 0x7f64100044d0 2021-01-06T22:19:11.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.383+0000 7f642dd9b700 1 -- 172.21.15.203:0/3733455287 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64180082d0 con 0x7f64100044d0 2021-01-06T22:19:11.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.383+0000 7f642dd9b700 1 -- 172.21.15.203:0/3733455287 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418005a40 con 0x7f64100044d0 2021-01-06T22:19:11.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.383+0000 7f642f59e700 1 -- 172.21.15.203:0/3733455287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64100044d0 msgr2=0x7f6410004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:11.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.383+0000 7f642f59e700 1 --2- 172.21.15.203:0/3733455287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64100044d0 0x7f6410004950 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f6418004dd0 tx=0x7f64300bdf50).stop 2021-01-06T22:19:11.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.383+0000 7f642f59e700 1 -- 172.21.15.203:0/3733455287 shutdown_connections 2021-01-06T22:19:11.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.383+0000 7f642f59e700 1 --2- 172.21.15.203:0/3733455287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64100044d0 0x7f6410004950 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:11.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.383+0000 7f642f59e700 1 -- 172.21.15.203:0/3733455287 >> 172.21.15.203:0/3733455287 conn(0x7f64300680f0 msgr2=0x7f6430056cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:11.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.383+0000 7f642f59e700 1 -- 172.21.15.203:0/3733455287 shutdown_connections 2021-01-06T22:19:11.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.383+0000 7f642f59e700 1 -- 172.21.15.203:0/3733455287 wait complete. 2021-01-06T22:19:11.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.383+0000 7f642f59e700 1 Processor -- start 2021-01-06T22:19:11.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.383+0000 7f642f59e700 1 -- start start 2021-01-06T22:19:11.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f642f59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f643014c910 0x7f643009ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:11.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f642f59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64300b9b30 0x7f64300ba1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:11.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f642f59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f643014ccf0 0x7f64300ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:11.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f642f59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64300bbb70 con 0x7f643014ccf0 2021-01-06T22:19:11.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f642f59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64300babc0 con 0x7f64300b9b30 2021-01-06T22:19:11.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f642f59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643009f530 con 0x7f643014c910 2021-01-06T22:19:11.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f6427fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64300b9b30 0x7f64300ba1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:11.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f6427fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64300b9b30 0x7f64300ba1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52892/0 (socket says 172.21.15.203:52892) 2021-01-06T22:19:11.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f6427fff700 1 -- 172.21.15.203:0/2173176062 learned_addr learned my addr 172.21.15.203:0/2173176062 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:11.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f642e59c700 1 --2- 172.21.15.203:0/2173176062 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f643014c910 0x7f643009ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:11.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f642ed9d700 1 --2- 172.21.15.203:0/2173176062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f643014ccf0 0x7f64300ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:11.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f6427fff700 1 -- 172.21.15.203:0/2173176062 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f643014c910 msgr2=0x7f643009ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:11.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f6427fff700 1 --2- 172.21.15.203:0/2173176062 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f643014c910 0x7f643009ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:11.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f6427fff700 1 -- 172.21.15.203:0/2173176062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f643014ccf0 msgr2=0x7f64300ba6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:11.399 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f6427fff700 1 --2- 172.21.15.203:0/2173176062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f643014ccf0 0x7f64300ba6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:11.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f6427fff700 1 -- 172.21.15.203:0/2173176062 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64301490c0 con 0x7f64300b9b30 2021-01-06T22:19:11.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f642ed9d700 1 --2- 172.21.15.203:0/2173176062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f643014ccf0 0x7f64300ba6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:19:11.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f642e59c700 1 --2- 172.21.15.203:0/2173176062 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f643014c910 0x7f643009ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:19:11.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f6427fff700 1 --2- 172.21.15.203:0/2173176062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64300b9b30 0x7f64300ba1e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f6418007000 tx=0x7f6418016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:11.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f64267fc700 1 -- 172.21.15.203:0/2173176062 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64180013e0 con 0x7f64300b9b30 2021-01-06T22:19:11.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f64267fc700 1 -- 172.21.15.203:0/2173176062 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6418001540 con 0x7f64300b9b30 2021-01-06T22:19:11.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f642f59e700 1 -- 172.21.15.203:0/2173176062 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6430149360 con 0x7f64300b9b30 2021-01-06T22:19:11.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f64267fc700 1 -- 172.21.15.203:0/2173176062 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418015620 con 0x7f64300b9b30 2021-01-06T22:19:11.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f642f59e700 1 -- 172.21.15.203:0/2173176062 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64301498a0 con 0x7f64300b9b30 2021-01-06T22:19:11.401 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.387+0000 7f642f59e700 1 -- 172.21.15.203:0/2173176062 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64300450a0 con 0x7f64300b9b30 2021-01-06T22:19:11.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.391+0000 7f64267fc700 1 -- 172.21.15.203:0/2173176062 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f6418011030 con 0x7f64300b9b30 2021-01-06T22:19:11.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.391+0000 7f64267fc700 1 --2- 172.21.15.203:0/2173176062 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6414056400 0x7f64140588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:11.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.391+0000 7f64267fc700 1 -- 172.21.15.203:0/2173176062 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f6418070bf0 con 0x7f64300b9b30 2021-01-06T22:19:11.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.391+0000 7f642e59c700 1 --2- 172.21.15.203:0/2173176062 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6414056400 0x7f64140588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:11.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.391+0000 7f642e59c700 1 --2- 172.21.15.203:0/2173176062 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6414056400 0x7f64140588c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f6420006a80 tx=0x7f6420008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:11.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.395+0000 7f64267fc700 1 -- 172.21.15.203:0/2173176062 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f64180461b0 con 0x7f64300b9b30 2021-01-06T22:19:11.599 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.595+0000 7f642f59e700 1 -- 172.21.15.203:0/2173176062 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f643009f270 con 0x7f64300b9b30 2021-01-06T22:19:11.600 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.599+0000 7f64267fc700 1 -- 172.21.15.203:0/2173176062 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+3581 (secure 0 0 0) 0x7f641801b030 con 0x7f64300b9b30 2021-01-06T22:19:11.600 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:19:11.600 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:19:04.439746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5539,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:active","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:19:11.601 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.599+0000 7f642f59e700 1 -- 172.21.15.203:0/2173176062 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6414056400 msgr2=0x7f64140588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:11.602 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.599+0000 7f642f59e700 1 --2- 172.21.15.203:0/2173176062 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6414056400 0x7f64140588c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f6420006a80 tx=0x7f6420008040).stop 2021-01-06T22:19:11.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.599+0000 7f642f59e700 1 -- 172.21.15.203:0/2173176062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64300b9b30 msgr2=0x7f64300ba1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:11.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.599+0000 7f642f59e700 1 --2- 172.21.15.203:0/2173176062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64300b9b30 0x7f64300ba1e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f6418007000 tx=0x7f6418016000).stop 2021-01-06T22:19:11.603 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.599+0000 7f642f59e700 1 -- 172.21.15.203:0/2173176062 shutdown_connections 2021-01-06T22:19:11.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.599+0000 7f642f59e700 1 --2- 172.21.15.203:0/2173176062 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6414056400 0x7f64140588c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:11.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.599+0000 7f642f59e700 1 --2- 172.21.15.203:0/2173176062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f643014ccf0 0x7f64300ba6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:11.604 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.599+0000 7f642f59e700 1 --2- 172.21.15.203:0/2173176062 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f643014c910 0x7f643009ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:11.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.599+0000 7f642f59e700 1 --2- 172.21.15.203:0/2173176062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64300b9b30 0x7f64300ba1e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:11.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.599+0000 7f642f59e700 1 -- 172.21.15.203:0/2173176062 >> 172.21.15.203:0/2173176062 conn(0x7f64300680f0 msgr2=0x7f643014d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:11.605 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.599+0000 7f642f59e700 1 -- 172.21.15.203:0/2173176062 shutdown_connections 2021-01-06T22:19:11.606 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:11.599+0000 7f642f59e700 1 -- 172.21.15.203:0/2173176062 wait complete. 2021-01-06T22:19:11.606 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 77 2021-01-06T22:19:11.615 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:19:13.617 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:19:13.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83658a8700 1 Processor -- start 2021-01-06T22:19:13.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83658a8700 1 -- start start 2021-01-06T22:19:13.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83658a8700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f83600bb3f0 0x7f83600ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:13.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83658a8700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f83600a5e50 0x7f83600669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:13.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83658a8700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f83600bb7d0 0x7f83600602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:13.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83658a8700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83600646b0 con 0x7f83600bb3f0 2021-01-06T22:19:13.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83658a8700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8360152cc0 con 0x7f83600bb7d0 2021-01-06T22:19:13.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83658a8700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8360064450 con 0x7f83600a5e50 2021-01-06T22:19:13.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83650a7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f83600bb7d0 0x7f83600602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:13.756 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83650a7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f83600bb7d0 0x7f83600602b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52906/0 (socket says 172.21.15.203:52906) 2021-01-06T22:19:13.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83650a7700 1 -- 172.21.15.203:0/1222557756 learned_addr learned my addr 172.21.15.203:0/1222557756 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:13.757 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83648a6700 1 --2- 172.21.15.203:0/1222557756 >> v2:172.21.15.91:3300/0 conn(0x7f83600bb3f0 0x7f83600ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:13.758 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f835ffff700 1 --2- 172.21.15.203:0/1222557756 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f83600a5e50 0x7f83600669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:13.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83650a7700 1 -- 172.21.15.203:0/1222557756 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f83600a5e50 msgr2=0x7f83600669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:13.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83650a7700 1 --2- 172.21.15.203:0/1222557756 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f83600a5e50 0x7f83600669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:13.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83650a7700 1 -- 172.21.15.203:0/1222557756 >> v2:172.21.15.91:3300/0 conn(0x7f83600bb3f0 msgr2=0x7f83600ba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:13.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83650a7700 1 --2- 172.21.15.203:0/1222557756 >> v2:172.21.15.91:3300/0 conn(0x7f83600bb3f0 0x7f83600ba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:13.760 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83650a7700 1 -- 172.21.15.203:0/1222557756 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8360068530 con 0x7f83600bb7d0 2021-01-06T22:19:13.760 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f83650a7700 1 --2- 172.21.15.203:0/1222557756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83600bb7d0 0x7f83600602b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f83600a62d0 tx=0x7f8358006ea0).ready entity=mon.1 client_cookie=f09ca83eb7646682 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:13.760 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f835f7fe700 1 -- 172.21.15.203:0/1222557756 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83580096e0 con 0x7f83600bb7d0 2021-01-06T22:19:13.760 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f835f7fe700 1 -- 172.21.15.203:0/1222557756 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8358009840 con 0x7f83600bb7d0 2021-01-06T22:19:13.760 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.747+0000 7f835f7fe700 1 -- 172.21.15.203:0/1222557756 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8358009bc0 con 0x7f83600bb7d0 2021-01-06T22:19:13.761 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.751+0000 7f83658a8700 1 -- 172.21.15.203:0/1222557756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83600bb7d0 msgr2=0x7f83600602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:13.761 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.751+0000 7f83658a8700 1 --2- 172.21.15.203:0/1222557756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83600bb7d0 0x7f83600602b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f83600a62d0 tx=0x7f8358006ea0).stop 2021-01-06T22:19:13.761 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.751+0000 7f83658a8700 1 -- 172.21.15.203:0/1222557756 shutdown_connections 2021-01-06T22:19:13.761 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.751+0000 7f83658a8700 1 --2- 172.21.15.203:0/1222557756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83600bb7d0 0x7f83600602b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:13.761 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.751+0000 7f83658a8700 1 --2- 172.21.15.203:0/1222557756 >> v2:172.21.15.91:3300/0 conn(0x7f83600bb3f0 0x7f83600ba9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:13.762 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.751+0000 7f83658a8700 1 --2- 172.21.15.203:0/1222557756 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f83600a5e50 0x7f83600669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:13.762 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.751+0000 7f83658a8700 1 -- 172.21.15.203:0/1222557756 >> 172.21.15.203:0/1222557756 conn(0x7f8360061530 msgr2=0x7f8360063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:13.762 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.751+0000 7f83658a8700 1 -- 172.21.15.203:0/1222557756 shutdown_connections 2021-01-06T22:19:13.762 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.751+0000 7f83658a8700 1 -- 172.21.15.203:0/1222557756 wait complete. 2021-01-06T22:19:13.768 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.751+0000 7f83658a8700 1 Processor -- start 2021-01-06T22:19:13.768 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.751+0000 7f83658a8700 1 -- start start 2021-01-06T22:19:13.768 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83658a8700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83600a5e50 0x7f83601615a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:13.768 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83658a8700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f83600bb3f0 0x7f8360060450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:13.769 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83658a8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83600bb7d0 0x7f8360162b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:13.769 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83658a8700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8360162590 con 0x7f83600a5e50 2021-01-06T22:19:13.769 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83658a8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8360154310 con 0x7f83600bb7d0 2021-01-06T22:19:13.769 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83658a8700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83601545b0 con 0x7f83600bb3f0 2021-01-06T22:19:13.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83650a7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83600bb7d0 0x7f8360162b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:13.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83650a7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83600bb7d0 0x7f8360162b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52912/0 (socket says 172.21.15.203:52912) 2021-01-06T22:19:13.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83650a7700 1 -- 172.21.15.203:0/1316850228 learned_addr learned my addr 172.21.15.203:0/1316850228 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:13.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83648a6700 1 --2- 172.21.15.203:0/1316850228 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83600a5e50 0x7f83601615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:13.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f835ffff700 1 --2- 172.21.15.203:0/1316850228 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f83600bb3f0 0x7f8360060450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:13.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83650a7700 1 -- 172.21.15.203:0/1316850228 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f83600bb3f0 msgr2=0x7f8360060450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:13.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83650a7700 1 --2- 172.21.15.203:0/1316850228 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f83600bb3f0 0x7f8360060450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:13.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83650a7700 1 -- 172.21.15.203:0/1316850228 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83600a5e50 msgr2=0x7f83601615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:13.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83650a7700 1 --2- 172.21.15.203:0/1316850228 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83600a5e50 0x7f83601615a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:13.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83650a7700 1 -- 172.21.15.203:0/1316850228 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8360154850 con 0x7f83600bb7d0 2021-01-06T22:19:13.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83650a7700 1 --2- 172.21.15.203:0/1316850228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83600bb7d0 0x7f8360162b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f8358009fd0 tx=0x7f8358016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:13.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f835cff9700 1 -- 172.21.15.203:0/1316850228 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83580096e0 con 0x7f83600bb7d0 2021-01-06T22:19:13.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f835cff9700 1 -- 172.21.15.203:0/1316850228 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835801c420 con 0x7f83600bb7d0 2021-01-06T22:19:13.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f835cff9700 1 -- 172.21.15.203:0/1316850228 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835801c7a0 con 0x7f83600bb7d0 2021-01-06T22:19:13.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83658a8700 1 -- 172.21.15.203:0/1316850228 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8360154af0 con 0x7f83600bb7d0 2021-01-06T22:19:13.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.755+0000 7f83658a8700 1 -- 172.21.15.203:0/1316850228 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8360155030 con 0x7f83600bb7d0 2021-01-06T22:19:13.774 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.759+0000 7f83658a8700 1 -- 172.21.15.203:0/1316850228 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83601551f0 con 0x7f83600bb7d0 2021-01-06T22:19:13.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.759+0000 7f835cff9700 1 -- 172.21.15.203:0/1316850228 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f835800f070 con 0x7f83600bb7d0 2021-01-06T22:19:13.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.759+0000 7f835cff9700 1 --2- 172.21.15.203:0/1316850228 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f83400563d0 0x7f8340058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:13.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.759+0000 7f835cff9700 1 -- 172.21.15.203:0/1316850228 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f8358070ed0 con 0x7f83600bb7d0 2021-01-06T22:19:13.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.759+0000 7f83648a6700 1 --2- 172.21.15.203:0/1316850228 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f83400563d0 0x7f8340058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:13.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.759+0000 7f83648a6700 1 --2- 172.21.15.203:0/1316850228 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f83400563d0 0x7f8340058890 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f8350009cd0 tx=0x7f83500091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:13.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.767+0000 7f835cff9700 1 -- 172.21.15.203:0/1316850228 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f8358046450 con 0x7f83600bb7d0 2021-01-06T22:19:13.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.963+0000 7f83658a8700 1 -- 172.21.15.203:0/1316850228 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83600450a0 con 0x7f83600bb7d0 2021-01-06T22:19:13.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.963+0000 7f835cff9700 1 -- 172.21.15.203:0/1316850228 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+3581 (secure 0 0 0) 0x7f8358049a70 con 0x7f83600bb7d0 2021-01-06T22:19:13.967 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:19:13.967 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:19:04.439746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5539,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:active","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:19:13.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.967+0000 7f83658a8700 1 -- 172.21.15.203:0/1316850228 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f83400563d0 msgr2=0x7f8340058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:13.969 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.967+0000 7f83658a8700 1 --2- 172.21.15.203:0/1316850228 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f83400563d0 0x7f8340058890 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f8350009cd0 tx=0x7f83500091d0).stop 2021-01-06T22:19:13.969 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.967+0000 7f83658a8700 1 -- 172.21.15.203:0/1316850228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83600bb7d0 msgr2=0x7f8360162b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:13.969 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.967+0000 7f83658a8700 1 --2- 172.21.15.203:0/1316850228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83600bb7d0 0x7f8360162b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f8358009fd0 tx=0x7f8358016000).stop 2021-01-06T22:19:13.969 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.967+0000 7f83658a8700 1 -- 172.21.15.203:0/1316850228 shutdown_connections 2021-01-06T22:19:13.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.967+0000 7f83658a8700 1 --2- 172.21.15.203:0/1316850228 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f83400563d0 0x7f8340058890 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:13.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.967+0000 7f83658a8700 1 --2- 172.21.15.203:0/1316850228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83600bb7d0 0x7f8360162b50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:13.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.967+0000 7f83658a8700 1 --2- 172.21.15.203:0/1316850228 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83600a5e50 0x7f83601615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:13.970 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.967+0000 7f83658a8700 1 --2- 172.21.15.203:0/1316850228 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f83600bb3f0 0x7f8360060450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:13.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.967+0000 7f83658a8700 1 -- 172.21.15.203:0/1316850228 >> 172.21.15.203:0/1316850228 conn(0x7f8360061530 msgr2=0x7f836014f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:13.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.967+0000 7f83658a8700 1 -- 172.21.15.203:0/1316850228 shutdown_connections 2021-01-06T22:19:13.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:13.967+0000 7f83658a8700 1 -- 172.21.15.203:0/1316850228 wait complete. 2021-01-06T22:19:13.971 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 77 2021-01-06T22:19:13.982 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:19:15.984 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:19:16.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.115+0000 7f0d67fff700 1 Processor -- start 2021-01-06T22:19:16.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d67fff700 1 -- start start 2021-01-06T22:19:16.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d67fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0d68067c10 0x7f0d68067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:16.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d67fff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d6814ddd0 con 0x7f0d68151c60 2021-01-06T22:19:16.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d67fff700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d68152110 con 0x7f0d6814f320 2021-01-06T22:19:16.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d67fff700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6814e000 con 0x7f0d68067c10 2021-01-06T22:19:16.122 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d677fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f0d6814f320 0x7f0d680bc9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51514/0 (socket says 172.21.15.203:51514) 2021-01-06T22:19:16.122 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d677fe700 1 -- 172.21.15.203:0/263603573 learned_addr learned my addr 172.21.15.203:0/263603573 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:19:16.122 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d66ffd700 1 --2- 172.21.15.203:0/263603573 >> v2:172.21.15.91:3300/0 conn(0x7f0d68067c10 0x7f0d68067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:16.122 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d667fc700 1 -- 172.21.15.203:0/263603573 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631221728 0 0) 0x7f0d68152110 con 0x7f0d6814f320 2021-01-06T22:19:16.122 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d667fc700 1 -- 172.21.15.203:0/263603573 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d48003230 con 0x7f0d6814f320 2021-01-06T22:19:16.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d667fc700 1 -- 172.21.15.203:0/263603573 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2620886557 0 0) 0x7f0d48003230 con 0x7f0d6814f320 2021-01-06T22:19:16.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d667fc700 1 -- 172.21.15.203:0/263603573 >> v2:172.21.15.91:3300/0 conn(0x7f0d68067c10 msgr2=0x7f0d68067ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:16.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d667fc700 1 --2- 172.21.15.203:0/263603573 >> v2:172.21.15.91:3300/0 conn(0x7f0d68067c10 0x7f0d68067ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:16.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d667fc700 1 -- 172.21.15.203:0/263603573 >> v1:172.21.15.91:6789/0 conn(0x7f0d68151c60 legacy=0x7f0d68064550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:16.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d667fc700 1 -- 172.21.15.203:0/263603573 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6814e2a0 con 0x7f0d6814f320 2021-01-06T22:19:16.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d667fc700 1 -- 172.21.15.203:0/263603573 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f0d60001860 con 0x7f0d6814f320 2021-01-06T22:19:16.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d667fc700 1 -- 172.21.15.203:0/263603573 >> v1:172.21.15.203:6789/0 conn(0x7f0d6814f320 legacy=0x7f0d680bc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:16.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d667fc700 1 --2- 172.21.15.203:0/263603573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d48003770 0x7f0d48003b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:16.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d667fc700 1 --2- 172.21.15.203:0/263603573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d480044d0 0x7f0d48004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:16.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d667fc700 1 --2- 172.21.15.203:0/263603573 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d48004e40 0x7f0d4800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:16.128 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d667fc700 1 -- 172.21.15.203:0/263603573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6814e000 con 0x7f0d480044d0 2021-01-06T22:19:16.128 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d667fc700 1 -- 172.21.15.203:0/263603573 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d480052f0 con 0x7f0d48003770 2021-01-06T22:19:16.128 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d667fc700 1 -- 172.21.15.203:0/263603573 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4800bc10 con 0x7f0d48004e40 2021-01-06T22:19:16.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d66ffd700 1 --2- 172.21.15.203:0/263603573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d48003770 0x7f0d48003b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:16.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d677fe700 1 --2- 172.21.15.203:0/263603573 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d48004e40 0x7f0d4800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:16.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d5e7fc700 1 --2- 172.21.15.203:0/263603573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d480044d0 0x7f0d48004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:16.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d66ffd700 1 -- 172.21.15.203:0/263603573 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d48004e40 msgr2=0x7f0d4800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:16.129 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d66ffd700 1 --2- 172.21.15.203:0/263603573 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d48004e40 0x7f0d4800b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:16.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d66ffd700 1 -- 172.21.15.203:0/263603573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d480044d0 msgr2=0x7f0d48004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:16.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d66ffd700 1 --2- 172.21.15.203:0/263603573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d480044d0 0x7f0d48004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:16.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d677fe700 1 -- 172.21.15.203:0/263603573 reap_dead start 2021-01-06T22:19:16.130 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d66ffd700 1 -- 172.21.15.203:0/263603573 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d48003230 con 0x7f0d48003770 2021-01-06T22:19:16.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.119+0000 7f0d66ffd700 1 --2- 172.21.15.203:0/263603573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d48003770 0x7f0d48003b90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f0d5800bb90 tx=0x7f0d58008040).ready entity=mon.1 client_cookie=fd6cb353e133a85e server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:16.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d667fc700 1 -- 172.21.15.203:0/263603573 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5800b2b0 con 0x7f0d48003770 2021-01-06T22:19:16.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d667fc700 1 -- 172.21.15.203:0/263603573 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5800e960 con 0x7f0d48003770 2021-01-06T22:19:16.131 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d667fc700 1 -- 172.21.15.203:0/263603573 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5800ed00 con 0x7f0d48003770 2021-01-06T22:19:16.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d67fff700 1 -- 172.21.15.203:0/263603573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d48003770 msgr2=0x7f0d48003b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:16.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d67fff700 1 --2- 172.21.15.203:0/263603573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d48003770 0x7f0d48003b90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f0d5800bb90 tx=0x7f0d58008040).stop 2021-01-06T22:19:16.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d67fff700 1 -- 172.21.15.203:0/263603573 shutdown_connections 2021-01-06T22:19:16.132 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d67fff700 1 --2- 172.21.15.203:0/263603573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d48003770 0x7f0d48003b90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:16.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d67fff700 1 -- 172.21.15.203:0/263603573 >> 172.21.15.203:0/263603573 conn(0x7f0d68061530 msgr2=0x7f0d68063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:16.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d67fff700 1 -- 172.21.15.203:0/263603573 shutdown_connections 2021-01-06T22:19:16.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d67fff700 1 -- 172.21.15.203:0/263603573 wait complete. 2021-01-06T22:19:16.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d67fff700 1 Processor -- start 2021-01-06T22:19:16.133 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d67fff700 1 -- start start 2021-01-06T22:19:16.136 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d67fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d48003770 0x7f0d680bf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:16.136 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d67fff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d680a48d0 0x7f0d680a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:16.136 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d67fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d68143eb0 0x7f0d68144330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:16.136 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d67fff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680a5240 con 0x7f0d680a48d0 2021-01-06T22:19:16.137 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d67fff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680bf900 con 0x7f0d48003770 2021-01-06T22:19:16.137 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d67fff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680bf6b0 con 0x7f0d68143eb0 2021-01-06T22:19:16.137 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d66ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d48003770 0x7f0d680bf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:16.137 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d66ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d48003770 0x7f0d680bf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52928/0 (socket says 172.21.15.203:52928) 2021-01-06T22:19:16.137 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d66ffd700 1 -- 172.21.15.203:0/2951387991 learned_addr learned my addr 172.21.15.203:0/2951387991 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:16.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d5e7fc700 1 --2- 172.21.15.203:0/2951387991 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d680a48d0 0x7f0d680a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:16.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d677fe700 1 --2- 172.21.15.203:0/2951387991 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d68143eb0 0x7f0d68144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:16.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d66ffd700 1 -- 172.21.15.203:0/2951387991 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d68143eb0 msgr2=0x7f0d68144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:16.138 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d66ffd700 1 --2- 172.21.15.203:0/2951387991 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d68143eb0 0x7f0d68144330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:16.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d66ffd700 1 -- 172.21.15.203:0/2951387991 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d680a48d0 msgr2=0x7f0d680a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:16.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d66ffd700 1 --2- 172.21.15.203:0/2951387991 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d680a48d0 0x7f0d680a4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:16.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d66ffd700 1 -- 172.21.15.203:0/2951387991 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d680b4760 con 0x7f0d48003770 2021-01-06T22:19:16.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.123+0000 7f0d66ffd700 1 --2- 172.21.15.203:0/2951387991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d48003770 0x7f0d680bf1c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f0d58012970 tx=0x7f0d5801a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:16.139 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.127+0000 7f0d5f7fe700 1 -- 172.21.15.203:0/2951387991 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d58012e00 con 0x7f0d48003770 2021-01-06T22:19:16.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.127+0000 7f0d5f7fe700 1 -- 172.21.15.203:0/2951387991 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5801b420 con 0x7f0d48003770 2021-01-06T22:19:16.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.127+0000 7f0d67fff700 1 -- 172.21.15.203:0/2951387991 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d680b4a00 con 0x7f0d48003770 2021-01-06T22:19:16.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.127+0000 7f0d5f7fe700 1 -- 172.21.15.203:0/2951387991 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5801b7e0 con 0x7f0d48003770 2021-01-06T22:19:16.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.127+0000 7f0d67fff700 1 -- 172.21.15.203:0/2951387991 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d680a4430 con 0x7f0d48003770 2021-01-06T22:19:16.140 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.127+0000 7f0d5f7fe700 1 -- 172.21.15.203:0/2951387991 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f0d5802c070 con 0x7f0d48003770 2021-01-06T22:19:16.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.127+0000 7f0d67fff700 1 -- 172.21.15.203:0/2951387991 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d680b4ea0 con 0x7f0d48003770 2021-01-06T22:19:16.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.127+0000 7f0d5f7fe700 1 --2- 172.21.15.203:0/2951387991 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0d4c0563d0 0x7f0d4c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:16.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.127+0000 7f0d5e7fc700 1 --2- 172.21.15.203:0/2951387991 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0d4c0563d0 0x7f0d4c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:16.141 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.127+0000 7f0d5f7fe700 1 -- 172.21.15.203:0/2951387991 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f0d58070bd0 con 0x7f0d48003770 2021-01-06T22:19:16.142 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.127+0000 7f0d5e7fc700 1 --2- 172.21.15.203:0/2951387991 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0d4c0563d0 0x7f0d4c058890 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f0d50000bc0 tx=0x7f0d50007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:16.142 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.135+0000 7f0d5f7fe700 1 -- 172.21.15.203:0/2951387991 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f0d680b4ea0 con 0x7f0d48003770 2021-01-06T22:19:16.341 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.339+0000 7f0d67fff700 1 -- 172.21.15.203:0/2951387991 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d680a4270 con 0x7f0d48003770 2021-01-06T22:19:16.342 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.339+0000 7f0d5f7fe700 1 -- 172.21.15.203:0/2951387991 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+3581 (secure 0 0 0) 0x7f0d58046150 con 0x7f0d48003770 2021-01-06T22:19:16.342 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:19:16.343 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:19:04.439746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5539,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:active","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:19:16.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.343+0000 7f0d67fff700 1 -- 172.21.15.203:0/2951387991 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0d4c0563d0 msgr2=0x7f0d4c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:16.344 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.343+0000 7f0d67fff700 1 --2- 172.21.15.203:0/2951387991 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0d4c0563d0 0x7f0d4c058890 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f0d50000bc0 tx=0x7f0d50007b80).stop 2021-01-06T22:19:16.345 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.343+0000 7f0d67fff700 1 -- 172.21.15.203:0/2951387991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d48003770 msgr2=0x7f0d680bf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:16.345 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.343+0000 7f0d67fff700 1 --2- 172.21.15.203:0/2951387991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d48003770 0x7f0d680bf1c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f0d58012970 tx=0x7f0d5801a000).stop 2021-01-06T22:19:16.345 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.343+0000 7f0d67fff700 1 -- 172.21.15.203:0/2951387991 shutdown_connections 2021-01-06T22:19:16.345 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.343+0000 7f0d67fff700 1 --2- 172.21.15.203:0/2951387991 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f0d4c0563d0 0x7f0d4c058890 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:16.345 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.343+0000 7f0d67fff700 1 --2- 172.21.15.203:0/2951387991 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d68143eb0 0x7f0d68144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:16.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.343+0000 7f0d67fff700 1 --2- 172.21.15.203:0/2951387991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d48003770 0x7f0d680bf1c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:16.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.343+0000 7f0d67fff700 1 --2- 172.21.15.203:0/2951387991 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d680a48d0 0x7f0d680a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:16.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.343+0000 7f0d67fff700 1 -- 172.21.15.203:0/2951387991 >> 172.21.15.203:0/2951387991 conn(0x7f0d68061530 msgr2=0x7f0d6814f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:16.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.343+0000 7f0d67fff700 1 -- 172.21.15.203:0/2951387991 shutdown_connections 2021-01-06T22:19:16.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:16.343+0000 7f0d67fff700 1 -- 172.21.15.203:0/2951387991 wait complete. 2021-01-06T22:19:16.347 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 77 2021-01-06T22:19:16.357 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:19:18.226 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:19:18.229 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:19:18.358 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:19:18.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d4221c700 1 Processor -- start 2021-01-06T22:19:18.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d4221c700 1 -- start start 2021-01-06T22:19:18.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d4221c700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1d3c14e680 0x7f1d3c063730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:18.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d4221c700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3c14eb50 con 0x7f1d3c0bf670 2021-01-06T22:19:18.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d4221c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3c0c0f80 con 0x7f1d3c0bf290 2021-01-06T22:19:18.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d4221c700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d3c0c1220 con 0x7f1d3c14e680 2021-01-06T22:19:18.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3b7fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f1d3c0bf290 0x7f1d3c14df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51538/0 (socket says 172.21.15.203:51538) 2021-01-06T22:19:18.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3b7fe700 1 -- 172.21.15.203:0/734771349 learned_addr learned my addr 172.21.15.203:0/734771349 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:19:18.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3affd700 1 --2- 172.21.15.203:0/734771349 >> v2:172.21.15.91:3300/0 conn(0x7f1d3c14e680 0x7f1d3c063730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:18.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3a7fc700 1 -- 172.21.15.203:0/734771349 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763093703 0 0) 0x7f1d3c0c0f80 con 0x7f1d3c0bf290 2021-01-06T22:19:18.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3a7fc700 1 -- 172.21.15.203:0/734771349 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d20003230 con 0x7f1d3c0bf290 2021-01-06T22:19:18.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3a7fc700 1 -- 172.21.15.203:0/734771349 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1996399143 0 0) 0x7f1d20003230 con 0x7f1d3c0bf290 2021-01-06T22:19:18.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3a7fc700 1 -- 172.21.15.203:0/734771349 >> v2:172.21.15.91:3300/0 conn(0x7f1d3c14e680 msgr2=0x7f1d3c063730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:18.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3a7fc700 1 --2- 172.21.15.203:0/734771349 >> v2:172.21.15.91:3300/0 conn(0x7f1d3c14e680 0x7f1d3c063730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:18.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3a7fc700 1 -- 172.21.15.203:0/734771349 >> v1:172.21.15.91:6789/0 conn(0x7f1d3c0bf670 legacy=0x7f1d3c147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:18.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3a7fc700 1 -- 172.21.15.203:0/734771349 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3c0c14c0 con 0x7f1d3c0bf290 2021-01-06T22:19:18.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3a7fc700 1 -- 172.21.15.203:0/734771349 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f1d34001860 con 0x7f1d3c0bf290 2021-01-06T22:19:18.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3a7fc700 1 -- 172.21.15.203:0/734771349 >> v1:172.21.15.203:6789/0 conn(0x7f1d3c0bf290 legacy=0x7f1d3c14df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:18.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3a7fc700 1 --2- 172.21.15.203:0/734771349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d20003720 0x7f1d20003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:18.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3a7fc700 1 --2- 172.21.15.203:0/734771349 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d200044d0 0x7f1d20004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:18.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3a7fc700 1 --2- 172.21.15.203:0/734771349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d20004e40 0x7f1d2000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:18.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3a7fc700 1 -- 172.21.15.203:0/734771349 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c0c1220 con 0x7f1d20004e40 2021-01-06T22:19:18.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3a7fc700 1 -- 172.21.15.203:0/734771349 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d20004250 con 0x7f1d20003720 2021-01-06T22:19:18.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3a7fc700 1 -- 172.21.15.203:0/734771349 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2000bc10 con 0x7f1d200044d0 2021-01-06T22:19:18.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3b7fe700 1 --2- 172.21.15.203:0/734771349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d20003720 0x7f1d20003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:18.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3affd700 1 --2- 172.21.15.203:0/734771349 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d200044d0 0x7f1d20004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:18.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3bfff700 1 --2- 172.21.15.203:0/734771349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d20004e40 0x7f1d2000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:18.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3affd700 1 -- 172.21.15.203:0/734771349 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d200044d0 msgr2=0x7f1d20004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2021-01-06T22:19:18.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3affd700 1 -- 172.21.15.203:0/734771349 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d200044d0 msgr2=0x7f1d20004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:19:18.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3affd700 1 --2- 172.21.15.203:0/734771349 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d200044d0 0x7f1d20004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:19:18.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3affd700 1 --2- 172.21.15.203:0/734771349 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d200044d0 0x7f1d20004950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-01-06T22:19:18.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3b7fe700 1 -- 172.21.15.203:0/734771349 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d200044d0 msgr2=0x7f1d20004950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-01-06T22:19:18.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3b7fe700 1 --2- 172.21.15.203:0/734771349 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d200044d0 0x7f1d20004950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:18.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3b7fe700 1 -- 172.21.15.203:0/734771349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d20004e40 msgr2=0x7f1d2000b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:18.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3b7fe700 1 --2- 172.21.15.203:0/734771349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d20004e40 0x7f1d2000b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:18.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3bfff700 1 -- 172.21.15.203:0/734771349 reap_dead start 2021-01-06T22:19:18.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.483+0000 7f1d3b7fe700 1 -- 172.21.15.203:0/734771349 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d20003230 con 0x7f1d20003720 2021-01-06T22:19:18.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.487+0000 7f1d3b7fe700 1 --2- 172.21.15.203:0/734771349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d20003720 0x7f1d20003b40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f1d3400c600 tx=0x7f1d3c0c0e00).ready entity=mon.1 client_cookie=cfb212c829192b58 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:18.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.487+0000 7f1d3a7fc700 1 -- 172.21.15.203:0/734771349 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34005b70 con 0x7f1d20003720 2021-01-06T22:19:18.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.487+0000 7f1d3a7fc700 1 -- 172.21.15.203:0/734771349 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d34005cd0 con 0x7f1d20003720 2021-01-06T22:19:18.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.487+0000 7f1d3a7fc700 1 -- 172.21.15.203:0/734771349 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34007300 con 0x7f1d20003720 2021-01-06T22:19:18.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.487+0000 7f1d4221c700 1 -- 172.21.15.203:0/734771349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d20003720 msgr2=0x7f1d20003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:18.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.487+0000 7f1d4221c700 1 --2- 172.21.15.203:0/734771349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d20003720 0x7f1d20003b40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f1d3400c600 tx=0x7f1d3c0c0e00).stop 2021-01-06T22:19:18.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.487+0000 7f1d4221c700 1 -- 172.21.15.203:0/734771349 shutdown_connections 2021-01-06T22:19:18.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.487+0000 7f1d4221c700 1 --2- 172.21.15.203:0/734771349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d20003720 0x7f1d20003b40 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:18.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.487+0000 7f1d4221c700 1 -- 172.21.15.203:0/734771349 >> 172.21.15.203:0/734771349 conn(0x7f1d3c0626e0 msgr2=0x7f1d3c061a30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:18.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.487+0000 7f1d4221c700 1 -- 172.21.15.203:0/734771349 shutdown_connections 2021-01-06T22:19:18.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.487+0000 7f1d4221c700 1 -- 172.21.15.203:0/734771349 wait complete. 2021-01-06T22:19:18.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.487+0000 7f1d4221c700 1 Processor -- start 2021-01-06T22:19:18.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.487+0000 7f1d4221c700 1 -- start start 2021-01-06T22:19:18.500 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d4221c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d20003720 0x7f1d3c151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:18.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d4221c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d3c151580 0x7f1d3c0ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:18.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d4221c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d3c0c44f0 0x7f1d3c0c3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:18.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d4221c700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c151d60 con 0x7f1d3c151580 2021-01-06T22:19:18.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d4221c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c151ac0 con 0x7f1d20003720 2021-01-06T22:19:18.501 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d4221c700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c15d6d0 con 0x7f1d3c0c44f0 2021-01-06T22:19:18.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d3b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d20003720 0x7f1d3c151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:18.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d3b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d20003720 0x7f1d3c151f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52952/0 (socket says 172.21.15.203:52952) 2021-01-06T22:19:18.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d3b7fe700 1 -- 172.21.15.203:0/740390059 learned_addr learned my addr 172.21.15.203:0/740390059 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:18.502 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d3affd700 1 --2- 172.21.15.203:0/740390059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d3c151580 0x7f1d3c0ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:18.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d3bfff700 1 --2- 172.21.15.203:0/740390059 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d3c0c44f0 0x7f1d3c0c3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:18.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d3b7fe700 1 -- 172.21.15.203:0/740390059 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d3c0c44f0 msgr2=0x7f1d3c0c3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:18.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d3b7fe700 1 --2- 172.21.15.203:0/740390059 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d3c0c44f0 0x7f1d3c0c3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:18.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d3b7fe700 1 -- 172.21.15.203:0/740390059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d3c151580 msgr2=0x7f1d3c0ba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:18.503 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d3b7fe700 1 --2- 172.21.15.203:0/740390059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d3c151580 0x7f1d3c0ba3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:18.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d3b7fe700 1 -- 172.21.15.203:0/740390059 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3c15d970 con 0x7f1d20003720 2021-01-06T22:19:18.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d3b7fe700 1 --2- 172.21.15.203:0/740390059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d20003720 0x7f1d3c151f50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f1d200043e0 tx=0x7f1d34015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:18.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d337fe700 1 -- 172.21.15.203:0/740390059 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34007880 con 0x7f1d20003720 2021-01-06T22:19:18.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d337fe700 1 -- 172.21.15.203:0/740390059 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d34001210 con 0x7f1d20003720 2021-01-06T22:19:18.504 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d4221c700 1 -- 172.21.15.203:0/740390059 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3c15dc10 con 0x7f1d20003720 2021-01-06T22:19:18.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d337fe700 1 -- 172.21.15.203:0/740390059 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34013b00 con 0x7f1d20003720 2021-01-06T22:19:18.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d4221c700 1 -- 172.21.15.203:0/740390059 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3c15e350 con 0x7f1d20003720 2021-01-06T22:19:18.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.491+0000 7f1d4221c700 1 -- 172.21.15.203:0/740390059 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d3c05cfe0 con 0x7f1d20003720 2021-01-06T22:19:18.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.495+0000 7f1d337fe700 1 -- 172.21.15.203:0/740390059 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f1d3401d070 con 0x7f1d20003720 2021-01-06T22:19:18.505 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.495+0000 7f1d337fe700 1 --2- 172.21.15.203:0/740390059 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d140563d0 0x7f1d14058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:18.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.495+0000 7f1d337fe700 1 -- 172.21.15.203:0/740390059 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f1d34005e40 con 0x7f1d20003720 2021-01-06T22:19:18.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.495+0000 7f1d3affd700 1 --2- 172.21.15.203:0/740390059 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d140563d0 0x7f1d14058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:18.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.495+0000 7f1d3affd700 1 --2- 172.21.15.203:0/740390059 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d140563d0 0x7f1d14058890 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f1d3c14e2a0 tx=0x7f1d24008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:18.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.503+0000 7f1d337fe700 1 -- 172.21.15.203:0/740390059 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f1d34046150 con 0x7f1d20003720 2021-01-06T22:19:18.702 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.699+0000 7f1d4221c700 1 -- 172.21.15.203:0/740390059 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d3c0bede0 con 0x7f1d20003720 2021-01-06T22:19:18.703 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.703+0000 7f1d337fe700 1 -- 172.21.15.203:0/740390059 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+3581 (secure 0 0 0) 0x7f1d3401a030 con 0x7f1d20003720 2021-01-06T22:19:18.703 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:19:18.704 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:19:04.439746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5539,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:active","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:19:18.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.703+0000 7f1d4221c700 1 -- 172.21.15.203:0/740390059 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d140563d0 msgr2=0x7f1d14058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:18.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.703+0000 7f1d4221c700 1 --2- 172.21.15.203:0/740390059 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d140563d0 0x7f1d14058890 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f1d3c14e2a0 tx=0x7f1d24008040).stop 2021-01-06T22:19:18.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.703+0000 7f1d4221c700 1 -- 172.21.15.203:0/740390059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d20003720 msgr2=0x7f1d3c151f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:18.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.703+0000 7f1d4221c700 1 --2- 172.21.15.203:0/740390059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d20003720 0x7f1d3c151f50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f1d200043e0 tx=0x7f1d34015000).stop 2021-01-06T22:19:18.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.703+0000 7f1d4221c700 1 -- 172.21.15.203:0/740390059 shutdown_connections 2021-01-06T22:19:18.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.703+0000 7f1d4221c700 1 --2- 172.21.15.203:0/740390059 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f1d140563d0 0x7f1d14058890 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:18.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.703+0000 7f1d4221c700 1 --2- 172.21.15.203:0/740390059 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d3c0c44f0 0x7f1d3c0c3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:18.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.703+0000 7f1d4221c700 1 --2- 172.21.15.203:0/740390059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d20003720 0x7f1d3c151f50 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:18.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.703+0000 7f1d4221c700 1 --2- 172.21.15.203:0/740390059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d3c151580 0x7f1d3c0ba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:18.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.703+0000 7f1d4221c700 1 -- 172.21.15.203:0/740390059 >> 172.21.15.203:0/740390059 conn(0x7f1d3c0626e0 msgr2=0x7f1d3c1452f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:18.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.703+0000 7f1d4221c700 1 -- 172.21.15.203:0/740390059 shutdown_connections 2021-01-06T22:19:18.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:18.703+0000 7f1d4221c700 1 -- 172.21.15.203:0/740390059 wait complete. 2021-01-06T22:19:18.708 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 77 2021-01-06T22:19:18.718 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:19:20.719 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:19:20.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f759e700 1 Processor -- start 2021-01-06T22:19:20.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f759e700 1 -- start start 2021-01-06T22:19:20.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f759e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f77f80bb3f0 0x7f77f80ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:20.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f759e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f80646b0 con 0x7f77f80bb7d0 2021-01-06T22:19:20.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f759e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f8152cc0 con 0x7f77f80a5e50 2021-01-06T22:19:20.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f759e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f80670c0 con 0x7f77f80bb3f0 2021-01-06T22:19:20.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f5d9b700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f77f80a5e50 0x7f77f80669b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51568/0 (socket says 172.21.15.203:51568) 2021-01-06T22:19:20.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f5d9b700 1 -- 172.21.15.203:0/1178474044 learned_addr learned my addr 172.21.15.203:0/1178474044 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:19:20.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f659c700 1 --2- 172.21.15.203:0/1178474044 >> v2:172.21.15.91:3300/0 conn(0x7f77f80bb3f0 0x7f77f80ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:20.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 -- 172.21.15.203:0/1178474044 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2049800023 0 0) 0x7f77f8152cc0 con 0x7f77f80a5e50 2021-01-06T22:19:20.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 -- 172.21.15.203:0/1178474044 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77dc003230 con 0x7f77f80a5e50 2021-01-06T22:19:20.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 -- 172.21.15.203:0/1178474044 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2603382174 0 0) 0x7f77dc003230 con 0x7f77f80a5e50 2021-01-06T22:19:20.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 -- 172.21.15.203:0/1178474044 >> v2:172.21.15.91:3300/0 conn(0x7f77f80bb3f0 msgr2=0x7f77f80ba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:20.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 --2- 172.21.15.203:0/1178474044 >> v2:172.21.15.91:3300/0 conn(0x7f77f80bb3f0 0x7f77f80ba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:20.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 -- 172.21.15.203:0/1178474044 >> v1:172.21.15.91:6789/0 conn(0x7f77f80bb7d0 legacy=0x7f77f814eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:20.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 -- 172.21.15.203:0/1178474044 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f8064450 con 0x7f77f80a5e50 2021-01-06T22:19:20.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 -- 172.21.15.203:0/1178474044 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f77ec0014e0 con 0x7f77f80a5e50 2021-01-06T22:19:20.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 -- 172.21.15.203:0/1178474044 >> v1:172.21.15.203:6789/0 conn(0x7f77f80a5e50 legacy=0x7f77f80669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:20.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 --2- 172.21.15.203:0/1178474044 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77dc003720 0x7f77dc003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:20.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 --2- 172.21.15.203:0/1178474044 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77dc0044d0 0x7f77dc004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:20.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 --2- 172.21.15.203:0/1178474044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77dc004e40 0x7f77dc00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:20.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 -- 172.21.15.203:0/1178474044 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f80670c0 con 0x7f77dc003720 2021-01-06T22:19:20.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 -- 172.21.15.203:0/1178474044 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc004250 con 0x7f77dc004e40 2021-01-06T22:19:20.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 -- 172.21.15.203:0/1178474044 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc00bc10 con 0x7f77dc0044d0 2021-01-06T22:19:20.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f6d9d700 1 --2- 172.21.15.203:0/1178474044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77dc004e40 0x7f77dc00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:20.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f5d9b700 1 --2- 172.21.15.203:0/1178474044 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77dc0044d0 0x7f77dc004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:20.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f659c700 1 --2- 172.21.15.203:0/1178474044 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77dc003720 0x7f77dc003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:20.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f6d9d700 1 -- 172.21.15.203:0/1178474044 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77dc0044d0 msgr2=0x7f77dc004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:20.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f6d9d700 1 --2- 172.21.15.203:0/1178474044 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77dc0044d0 0x7f77dc004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:20.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f6d9d700 1 -- 172.21.15.203:0/1178474044 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77dc003720 msgr2=0x7f77dc003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:20.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f6d9d700 1 --2- 172.21.15.203:0/1178474044 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77dc003720 0x7f77dc003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:20.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f6d9d700 1 -- 172.21.15.203:0/1178474044 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc003230 con 0x7f77dc004e40 2021-01-06T22:19:20.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f6d9d700 1 -- 172.21.15.203:0/1178474044 reap_dead start 2021-01-06T22:19:20.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f6d9d700 1 --2- 172.21.15.203:0/1178474044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77dc004e40 0x7f77dc00b720 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f77f0001e70 tx=0x7f77f0007f00).ready entity=mon.1 client_cookie=58601d4ebac94716 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:20.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 -- 172.21.15.203:0/1178474044 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0001540 con 0x7f77dc004e40 2021-01-06T22:19:20.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 -- 172.21.15.203:0/1178474044 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f0003030 con 0x7f77dc004e40 2021-01-06T22:19:20.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.851+0000 7f77f559a700 1 -- 172.21.15.203:0/1178474044 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f000cdd0 con 0x7f77dc004e40 2021-01-06T22:19:20.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.855+0000 7f77f759e700 1 -- 172.21.15.203:0/1178474044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77dc004e40 msgr2=0x7f77dc00b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:20.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.855+0000 7f77f759e700 1 --2- 172.21.15.203:0/1178474044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77dc004e40 0x7f77dc00b720 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f77f0001e70 tx=0x7f77f0007f00).stop 2021-01-06T22:19:20.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.855+0000 7f77f759e700 1 -- 172.21.15.203:0/1178474044 shutdown_connections 2021-01-06T22:19:20.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.855+0000 7f77f759e700 1 --2- 172.21.15.203:0/1178474044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77dc004e40 0x7f77dc00b720 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:20.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.855+0000 7f77f759e700 1 -- 172.21.15.203:0/1178474044 >> 172.21.15.203:0/1178474044 conn(0x7f77f8061530 msgr2=0x7f77f8063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:20.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.855+0000 7f77f759e700 1 -- 172.21.15.203:0/1178474044 shutdown_connections 2021-01-06T22:19:20.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.855+0000 7f77f759e700 1 -- 172.21.15.203:0/1178474044 wait complete. 2021-01-06T22:19:20.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.855+0000 7f77f759e700 1 Processor -- start 2021-01-06T22:19:20.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.855+0000 7f77f759e700 1 -- start start 2021-01-06T22:19:20.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f759e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77dc004e40 0x7f77f81658b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:20.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f759e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f809e4c0 0x7f77f81621d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:20.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f759e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f80607c0 0x7f77f8060c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:20.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f759e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f80a2bb0 con 0x7f77f80607c0 2021-01-06T22:19:20.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f759e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f8165da0 con 0x7f77f809e4c0 2021-01-06T22:19:20.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f759e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f80a2de0 con 0x7f77dc004e40 2021-01-06T22:19:20.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f5d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f809e4c0 0x7f77f81621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:20.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f5d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f809e4c0 0x7f77f81621d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52984/0 (socket says 172.21.15.203:52984) 2021-01-06T22:19:20.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f5d9b700 1 -- 172.21.15.203:0/818242670 learned_addr learned my addr 172.21.15.203:0/818242670 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:20.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f659c700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77dc004e40 0x7f77f81658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:20.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f6d9d700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f80607c0 0x7f77f8060c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:20.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f5d9b700 1 -- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77dc004e40 msgr2=0x7f77f81658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:20.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f5d9b700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77dc004e40 0x7f77f81658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:20.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f5d9b700 1 -- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f80607c0 msgr2=0x7f77f8060c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:20.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f5d9b700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f80607c0 0x7f77f8060c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:20.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f5d9b700 1 -- 172.21.15.203:0/818242670 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f80a3080 con 0x7f77f809e4c0 2021-01-06T22:19:20.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f6d9d700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f80607c0 0x7f77f8060c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:19:20.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f659c700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77dc004e40 0x7f77f81658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:19:20.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f5d9b700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f809e4c0 0x7f77f81621d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f77ec00ea80 tx=0x7f77f81521a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:20.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f5d9b700 1 -- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f809e4c0 msgr2=0x7f77f81621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2021-01-06T22:19:20.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f5d9b700 1 -- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f809e4c0 msgr2=0x7f77f81621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2021-01-06T22:19:20.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f5d9b700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f809e4c0 0x7f77f81621d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f77ec00ea80 tx=0x7f77f81521a0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:19:20.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f5d9b700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f809e4c0 0x7f77f81621d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f77ec00ea80 tx=0x7f77f81521a0).stop 2021-01-06T22:19:20.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77e67fc700 1 -- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f809e4c0 msgr2=0x7f77f81621d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2021-01-06T22:19:20.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77e67fc700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f809e4c0 0x7f77f81621d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:20.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77e67fc700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d40031e0 0x7f77d4005630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:20.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77e67fc700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77dc004e40 0x7f77d4007cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:20.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77e67fc700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f80607c0 0x7f77d400a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:20.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77e67fc700 1 -- 172.21.15.203:0/818242670 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d400a790 con 0x7f77f80607c0 2021-01-06T22:19:20.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77e67fc700 1 -- 172.21.15.203:0/818242670 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d400a8d0 con 0x7f77d40031e0 2021-01-06T22:19:20.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77e67fc700 1 -- 172.21.15.203:0/818242670 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d400aa90 con 0x7f77dc004e40 2021-01-06T22:19:20.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f659c700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d40031e0 0x7f77d4005630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:20.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f6d9d700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f80607c0 0x7f77d400a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:20.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f659c700 1 -- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77dc004e40 msgr2=0x7f77d4007cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:20.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f5d9b700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77dc004e40 0x7f77d4007cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:20.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f659c700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77dc004e40 0x7f77d4007cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:20.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f659c700 1 -- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f80607c0 msgr2=0x7f77d400a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:20.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f659c700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f80607c0 0x7f77d400a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:20.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f659c700 1 -- 172.21.15.203:0/818242670 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d400ad10 con 0x7f77d40031e0 2021-01-06T22:19:20.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f6d9d700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f80607c0 0x7f77d400a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:19:20.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f5d9b700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77dc004e40 0x7f77d4007cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:19:20.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f659c700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d40031e0 0x7f77d4005630 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f77e80116a0 tx=0x7f77e8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:20.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77e67fc700 1 -- 172.21.15.203:0/818242670 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e800b200 con 0x7f77d40031e0 2021-01-06T22:19:20.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f759e700 1 -- 172.21.15.203:0/818242670 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f80a3320 con 0x7f77d40031e0 2021-01-06T22:19:20.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77e67fc700 1 -- 172.21.15.203:0/818242670 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e80136e0 con 0x7f77d40031e0 2021-01-06T22:19:20.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77e67fc700 1 -- 172.21.15.203:0/818242670 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8013a60 con 0x7f77d40031e0 2021-01-06T22:19:20.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f759e700 1 -- 172.21.15.203:0/818242670 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f8161d30 con 0x7f77d40031e0 2021-01-06T22:19:20.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.859+0000 7f77f759e700 1 -- 172.21.15.203:0/818242670 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f80450a0 con 0x7f77d40031e0 2021-01-06T22:19:20.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.863+0000 7f77e67fc700 1 -- 172.21.15.203:0/818242670 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f77e8012040 con 0x7f77d40031e0 2021-01-06T22:19:20.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.863+0000 7f77e67fc700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f77f809e4c0 0x7f77d405fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:20.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.863+0000 7f77e67fc700 1 -- 172.21.15.203:0/818242670 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f77e806fe60 con 0x7f77d40031e0 2021-01-06T22:19:20.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.863+0000 7f77f5d9b700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f77f809e4c0 0x7f77d405fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:20.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.863+0000 7f77f5d9b700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f77f809e4c0 0x7f77d405fe40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f77ec0013b0 tx=0x7f77ec009000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:20.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:20.867+0000 7f77e67fc700 1 -- 172.21.15.203:0/818242670 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f77e8045350 con 0x7f77d40031e0 2021-01-06T22:19:21.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:21.067+0000 7f77f759e700 1 -- 172.21.15.203:0/818242670 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77f809e940 con 0x7f77d40031e0 2021-01-06T22:19:21.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:21.067+0000 7f77e67fc700 1 -- 172.21.15.203:0/818242670 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+3581 (secure 0 0 0) 0x7f77e8048970 con 0x7f77d40031e0 2021-01-06T22:19:21.070 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:19:21.070 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:19:04.439746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5539,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:active","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5539":{"gid":5539,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6835/1801698156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1801698156},{"type":"v1","addr":"172.21.15.203:6835","nonce":1801698156}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:19:21.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:21.071+0000 7f77f759e700 1 -- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f77f809e4c0 msgr2=0x7f77d405fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:21.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:21.071+0000 7f77f759e700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f77f809e4c0 0x7f77d405fe40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f77ec0013b0 tx=0x7f77ec009000).stop 2021-01-06T22:19:21.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:21.071+0000 7f77f759e700 1 -- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d40031e0 msgr2=0x7f77d4005630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:21.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:21.071+0000 7f77f759e700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d40031e0 0x7f77d4005630 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f77e80116a0 tx=0x7f77e8008040).stop 2021-01-06T22:19:21.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:21.071+0000 7f77f759e700 1 -- 172.21.15.203:0/818242670 shutdown_connections 2021-01-06T22:19:21.072 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:21.071+0000 7f77f759e700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f77f809e4c0 0x7f77d405fe40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:21.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:21.071+0000 7f77f759e700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f80607c0 0x7f77d400a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:21.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:21.071+0000 7f77f759e700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77dc004e40 0x7f77d4007cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:21.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:21.071+0000 7f77f759e700 1 --2- 172.21.15.203:0/818242670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d40031e0 0x7f77d4005630 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:21.073 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:21.071+0000 7f77f759e700 1 -- 172.21.15.203:0/818242670 >> 172.21.15.203:0/818242670 conn(0x7f77f8061530 msgr2=0x7f77f80ba6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:21.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:21.071+0000 7f77f759e700 1 -- 172.21.15.203:0/818242670 shutdown_connections 2021-01-06T22:19:21.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:21.071+0000 7f77f759e700 1 -- 172.21.15.203:0/818242670 wait complete. 2021-01-06T22:19:21.074 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 77 2021-01-06T22:19:21.084 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:19:23.085 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:19:23.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.851+0000 7f5c74806700 1 Processor -- start 2021-01-06T22:19:23.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.851+0000 7f5c74806700 1 -- start start 2021-01-06T22:19:23.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.851+0000 7f5c74806700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5c70067340 0x7f5c70068110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:23.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.851+0000 7f5c74806700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5c70067720 0x7f5c700602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:23.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.851+0000 7f5c74806700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c700bba10 con 0x7f5c7014b4f0 2021-01-06T22:19:23.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.851+0000 7f5c74806700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c700bb230 con 0x7f5c70067340 2021-01-06T22:19:23.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.851+0000 7f5c74806700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c700bb4d0 con 0x7f5c70067720 2021-01-06T22:19:23.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.851+0000 7f5c6ed9d700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5c70067340 0x7f5c70068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:23.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.851+0000 7f5c6e59c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5c70067720 0x7f5c700602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:23.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.851+0000 7f5c6ed9d700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5c70067340 0x7f5c70068110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:60854/0 (socket says 172.21.15.203:60854) 2021-01-06T22:19:23.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.851+0000 7f5c6ed9d700 1 -- 172.21.15.203:0/3944523720 learned_addr learned my addr 172.21.15.203:0/3944523720 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:23.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.851+0000 7f5c6e59c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5c70067720 0x7f5c700602b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53010/0 (socket says 172.21.15.203:53010) 2021-01-06T22:19:23.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c6e59c700 1 -- 172.21.15.203:0/3944523720 >> v2:172.21.15.91:3300/0 conn(0x7f5c70067340 msgr2=0x7f5c70068110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:23.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c6e59c700 1 --2- 172.21.15.203:0/3944523720 >> v2:172.21.15.91:3300/0 conn(0x7f5c70067340 0x7f5c70068110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:23.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c6e59c700 1 -- 172.21.15.203:0/3944523720 >> v1:172.21.15.91:6789/0 conn(0x7f5c7014b4f0 legacy=0x7f5c70147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:23.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c6e59c700 1 -- 172.21.15.203:0/3944523720 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c700bb770 con 0x7f5c70067720 2021-01-06T22:19:23.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c6ed9d700 1 --2- 172.21.15.203:0/3944523720 >> v2:172.21.15.91:3300/0 conn(0x7f5c70067340 0x7f5c70068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:19:23.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c6e59c700 1 --2- 172.21.15.203:0/3944523720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c70067720 0x7f5c700602b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f5c7005c920 tx=0x7f5c58006de0).ready entity=mon.1 client_cookie=e9d646aebe1f7bef server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:23.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c6dd9b700 1 -- 172.21.15.203:0/3944523720 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c58009750 con 0x7f5c70067720 2021-01-06T22:19:23.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c6dd9b700 1 -- 172.21.15.203:0/3944523720 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c580098b0 con 0x7f5c70067720 2021-01-06T22:19:23.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c6dd9b700 1 -- 172.21.15.203:0/3944523720 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c58009c30 con 0x7f5c70067720 2021-01-06T22:19:23.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c74806700 1 -- 172.21.15.203:0/3944523720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c70067720 msgr2=0x7f5c700602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:23.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c74806700 1 --2- 172.21.15.203:0/3944523720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c70067720 0x7f5c700602b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f5c7005c920 tx=0x7f5c58006de0).stop 2021-01-06T22:19:23.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c74806700 1 -- 172.21.15.203:0/3944523720 shutdown_connections 2021-01-06T22:19:23.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c74806700 1 --2- 172.21.15.203:0/3944523720 >> v2:172.21.15.91:3300/0 conn(0x7f5c70067340 0x7f5c70068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:23.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c74806700 1 --2- 172.21.15.203:0/3944523720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c70067720 0x7f5c700602b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:23.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c74806700 1 -- 172.21.15.203:0/3944523720 >> 172.21.15.203:0/3944523720 conn(0x7f5c700667b0 msgr2=0x7f5c700c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:23.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c74806700 1 -- 172.21.15.203:0/3944523720 shutdown_connections 2021-01-06T22:19:23.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.855+0000 7f5c74806700 1 -- 172.21.15.203:0/3944523720 wait complete. 2021-01-06T22:19:23.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c74806700 1 Processor -- start 2021-01-06T22:19:23.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c74806700 1 -- start start 2021-01-06T22:19:23.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c74806700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c70067340 0x7f5c70141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:23.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c74806700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c7014b4f0 0x7f5c7015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:23.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c74806700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c7014a480 0x7f5c7015e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:23.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c74806700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c70148df0 con 0x7f5c7014a480 2021-01-06T22:19:23.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c74806700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7015e770 con 0x7f5c70067340 2021-01-06T22:19:23.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c74806700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c701415c0 con 0x7f5c7014b4f0 2021-01-06T22:19:23.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c6ed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c70067340 0x7f5c70141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:23.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c6ed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c70067340 0x7f5c70141b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53012/0 (socket says 172.21.15.203:53012) 2021-01-06T22:19:23.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c6ed9d700 1 -- 172.21.15.203:0/2292038895 learned_addr learned my addr 172.21.15.203:0/2292038895 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:23.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c6e59c700 1 --2- 172.21.15.203:0/2292038895 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c7014b4f0 0x7f5c7015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:23.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c6f59e700 1 --2- 172.21.15.203:0/2292038895 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c7014a480 0x7f5c7015e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:23.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c6ed9d700 1 -- 172.21.15.203:0/2292038895 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c7014b4f0 msgr2=0x7f5c7015dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:23.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c6ed9d700 1 --2- 172.21.15.203:0/2292038895 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c7014b4f0 0x7f5c7015dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:23.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c6ed9d700 1 -- 172.21.15.203:0/2292038895 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c7014a480 msgr2=0x7f5c7015e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:23.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c6ed9d700 1 --2- 172.21.15.203:0/2292038895 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c7014a480 0x7f5c7015e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:23.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c6ed9d700 1 -- 172.21.15.203:0/2292038895 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c70145fc0 con 0x7f5c70067340 2021-01-06T22:19:23.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c6e59c700 1 --2- 172.21.15.203:0/2292038895 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c7014b4f0 0x7f5c7015dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:19:23.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.859+0000 7f5c6ed9d700 1 --2- 172.21.15.203:0/2292038895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c70067340 0x7f5c70141b20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f5c6000efd0 tx=0x7f5c60009290).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:23.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.863+0000 7f5c66ffd700 1 -- 172.21.15.203:0/2292038895 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6000ce20 con 0x7f5c70067340 2021-01-06T22:19:23.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.863+0000 7f5c66ffd700 1 -- 172.21.15.203:0/2292038895 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6001a470 con 0x7f5c70067340 2021-01-06T22:19:23.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.863+0000 7f5c66ffd700 1 -- 172.21.15.203:0/2292038895 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6001a7f0 con 0x7f5c70067340 2021-01-06T22:19:23.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.863+0000 7f5c74806700 1 -- 172.21.15.203:0/2292038895 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c701488b0 con 0x7f5c70067340 2021-01-06T22:19:23.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.863+0000 7f5c74806700 1 -- 172.21.15.203:0/2292038895 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c70141080 con 0x7f5c70067340 2021-01-06T22:19:23.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.863+0000 7f5c66ffd700 1 -- 172.21.15.203:0/2292038895 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f5c6001ab60 con 0x7f5c70067340 2021-01-06T22:19:23.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.863+0000 7f5c74806700 1 -- 172.21.15.203:0/2292038895 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7015da80 con 0x7f5c70067340 2021-01-06T22:19:23.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.863+0000 7f5c66ffd700 1 --2- 172.21.15.203:0/2292038895 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5c540563d0 0x7f5c54058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:23.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.863+0000 7f5c66ffd700 1 -- 172.21.15.203:0/2292038895 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7f5c60072220 con 0x7f5c70067340 2021-01-06T22:19:23.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.863+0000 7f5c6e59c700 1 --2- 172.21.15.203:0/2292038895 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5c540563d0 0x7f5c54058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:23.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.863+0000 7f5c6e59c700 1 --2- 172.21.15.203:0/2292038895 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5c540563d0 0x7f5c54058890 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f5c70066780 tx=0x7f5c58016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:23.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:23.871+0000 7f5c66ffd700 1 -- 172.21.15.203:0/2292038895 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f5c60047740 con 0x7f5c70067340 2021-01-06T22:19:24.079 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.079+0000 7f5c74806700 1 -- 172.21.15.203:0/2292038895 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c700450a0 con 0x7f5c70067340 2021-01-06T22:19:24.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.079+0000 7f5c66ffd700 1 -- 172.21.15.203:0/2292038895 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+3229 (secure 0 0 0) 0x7f5c6001f070 con 0x7f5c70067340 2021-01-06T22:19:24.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.079+0000 7f5c74806700 1 -- 172.21.15.203:0/2292038895 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5c540563d0 msgr2=0x7f5c54058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:24.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.079+0000 7f5c74806700 1 --2- 172.21.15.203:0/2292038895 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5c540563d0 0x7f5c54058890 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f5c70066780 tx=0x7f5c58016000).stop 2021-01-06T22:19:24.240 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.079+0000 7f5c74806700 1 -- 172.21.15.203:0/2292038895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c70067340 msgr2=0x7f5c70141b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:24.240 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.079+0000 7f5c74806700 1 --2- 172.21.15.203:0/2292038895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c70067340 0x7f5c70141b20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f5c6000efd0 tx=0x7f5c60009290).stop 2021-01-06T22:19:24.240 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.079+0000 7f5c74806700 1 -- 172.21.15.203:0/2292038895 shutdown_connections 2021-01-06T22:19:24.240 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.079+0000 7f5c74806700 1 --2- 172.21.15.203:0/2292038895 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5c540563d0 0x7f5c54058890 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:24.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.079+0000 7f5c74806700 1 --2- 172.21.15.203:0/2292038895 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c7014a480 0x7f5c7015e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:24.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.079+0000 7f5c74806700 1 --2- 172.21.15.203:0/2292038895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c70067340 0x7f5c70141b20 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:24.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.079+0000 7f5c74806700 1 --2- 172.21.15.203:0/2292038895 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c7014b4f0 0x7f5c7015dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:24.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.079+0000 7f5c74806700 1 -- 172.21.15.203:0/2292038895 >> 172.21.15.203:0/2292038895 conn(0x7f5c700667b0 msgr2=0x7f5c70060860 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:24.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.079+0000 7f5c74806700 1 -- 172.21.15.203:0/2292038895 shutdown_connections 2021-01-06T22:19:24.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.079+0000 7f5c74806700 1 -- 172.21.15.203:0/2292038895 wait complete. 2021-01-06T22:19:24.242 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 80 2021-01-06T22:19:24.242 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:19:24.242 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:19:23.646378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4724,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:reconnect","state_seq":202,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:active","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:19:24.243 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2021-01-06T22:19:24.243 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2021-01-06T22:19:24.243 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:19:24.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533f59e700 1 Processor -- start 2021-01-06T22:19:24.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533f59e700 1 -- start start 2021-01-06T22:19:24.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533f59e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f53401478c0 0x7f53400bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:24.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533f59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53400c16b0 con 0x7f5340064380 2021-01-06T22:19:24.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533f59e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53400bb2d0 con 0x7f534014e310 2021-01-06T22:19:24.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533f59e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534014a560 con 0x7f53401478c0 2021-01-06T22:19:24.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533ed9d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f53401478c0 0x7f53400bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:24.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533ed9d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f53401478c0 0x7f53400bb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53024/0 (socket says 172.21.15.203:53024) 2021-01-06T22:19:24.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533ed9d700 1 -- 172.21.15.203:0/3244762144 learned_addr learned my addr 172.21.15.203:0/3244762144 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:24.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533dd9b700 1 --1- 172.21.15.203:0/3244762144 >> v1:172.21.15.203:6789/0 conn(0x7f534014e310 0x7f53401471b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51616/0 (socket says 172.21.15.203:51616) 2021-01-06T22:19:24.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533d59a700 1 -- 172.21.15.203:0/3244762144 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131329056 0 0) 0x7f53300010d0 con 0x7f5340064380 2021-01-06T22:19:24.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533ed9d700 1 -- 172.21.15.203:0/3244762144 >> v1:172.21.15.203:6789/0 conn(0x7f534014e310 legacy=0x7f53401471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:24.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533ed9d700 1 -- 172.21.15.203:0/3244762144 >> v1:172.21.15.91:6789/0 conn(0x7f5340064380 legacy=0x7f53400c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:24.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533ed9d700 1 -- 172.21.15.203:0/3244762144 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534014a800 con 0x7f53401478c0 2021-01-06T22:19:24.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533ed9d700 1 --2- 172.21.15.203:0/3244762144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53401478c0 0x7f53400bb730 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f5338009d50 tx=0x7f5338009290).ready entity=mon.1 client_cookie=7811adbe410677f9 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:24.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533d59a700 1 -- 172.21.15.203:0/3244762144 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533800bd10 con 0x7f53401478c0 2021-01-06T22:19:24.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533d59a700 1 -- 172.21.15.203:0/3244762144 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533800be70 con 0x7f53401478c0 2021-01-06T22:19:24.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533d59a700 1 -- 172.21.15.203:0/3244762144 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5338011610 con 0x7f53401478c0 2021-01-06T22:19:24.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533f59e700 1 -- 172.21.15.203:0/3244762144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53401478c0 msgr2=0x7f53400bb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:24.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533f59e700 1 --2- 172.21.15.203:0/3244762144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53401478c0 0x7f53400bb730 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f5338009d50 tx=0x7f5338009290).stop 2021-01-06T22:19:24.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533f59e700 1 -- 172.21.15.203:0/3244762144 shutdown_connections 2021-01-06T22:19:24.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533f59e700 1 --2- 172.21.15.203:0/3244762144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53401478c0 0x7f53400bb730 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:24.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533f59e700 1 -- 172.21.15.203:0/3244762144 >> 172.21.15.203:0/3244762144 conn(0x7f53400667b0 msgr2=0x7f53400c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:24.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533f59e700 1 -- 172.21.15.203:0/3244762144 shutdown_connections 2021-01-06T22:19:24.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.379+0000 7f533f59e700 1 -- 172.21.15.203:0/3244762144 wait complete. 2021-01-06T22:19:24.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533f59e700 1 Processor -- start 2021-01-06T22:19:24.388 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533f59e700 1 -- start start 2021-01-06T22:19:24.390 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533f59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5340064380 0x7f534013fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:24.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533f59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53401478c0 0x7f5340148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:24.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533f59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f534014e310 0x7f53400b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:24.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533f59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5340141770 con 0x7f53401478c0 2021-01-06T22:19:24.391 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533f59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5340149940 con 0x7f534014e310 2021-01-06T22:19:24.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533f59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534013eed0 con 0x7f5340064380 2021-01-06T22:19:24.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533ed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f534014e310 0x7f53400b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:24.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533ed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f534014e310 0x7f53400b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53030/0 (socket says 172.21.15.203:53030) 2021-01-06T22:19:24.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533ed9d700 1 -- 172.21.15.203:0/1567093699 learned_addr learned my addr 172.21.15.203:0/1567093699 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:24.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533dd9b700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53401478c0 0x7f5340148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:24.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533e59c700 1 --2- 172.21.15.203:0/1567093699 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5340064380 0x7f534013fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:24.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533ed9d700 1 -- 172.21.15.203:0/1567093699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f534014e310 msgr2=0x7f53400b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2021-01-06T22:19:24.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533ed9d700 1 -- 172.21.15.203:0/1567093699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f534014e310 msgr2=0x7f53400b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2021-01-06T22:19:24.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533ed9d700 1 --2- 172.21.15.203:0/1567093699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f534014e310 0x7f53400b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:19:24.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.383+0000 7f533ed9d700 1 --2- 172.21.15.203:0/1567093699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f534014e310 0x7f53400b4c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-01-06T22:19:24.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.387+0000 7f533dd9b700 1 -- 172.21.15.203:0/1567093699 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5340064380 msgr2=0x7f534013fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:24.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.387+0000 7f533dd9b700 1 --2- 172.21.15.203:0/1567093699 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5340064380 0x7f534013fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:24.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.387+0000 7f533dd9b700 1 -- 172.21.15.203:0/1567093699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f534014e310 msgr2=0x7f53400b4c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2021-01-06T22:19:24.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.387+0000 7f533dd9b700 1 --2- 172.21.15.203:0/1567093699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f534014e310 0x7f53400b4c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:24.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.387+0000 7f533dd9b700 1 -- 172.21.15.203:0/1567093699 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53401419a0 con 0x7f53401478c0 2021-01-06T22:19:24.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.387+0000 7f533dd9b700 1 --2- 172.21.15.203:0/1567093699 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53401478c0 0x7f5340148fc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f532800c6e0 tx=0x7f5328007b70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:24.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.387+0000 7f53367fc700 1 -- 172.21.15.203:0/1567093699 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53280072e0 con 0x7f53401478c0 2021-01-06T22:19:24.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.387+0000 7f533f59e700 1 -- 172.21.15.203:0/1567093699 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5340141c40 con 0x7f53401478c0 2021-01-06T22:19:24.395 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.387+0000 7f53367fc700 1 -- 172.21.15.203:0/1567093699 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5328005520 con 0x7f53401478c0 2021-01-06T22:19:24.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.387+0000 7f53367fc700 1 -- 172.21.15.203:0/1567093699 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53280058a0 con 0x7f53401478c0 2021-01-06T22:19:24.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.387+0000 7f533f59e700 1 -- 172.21.15.203:0/1567093699 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53400b47f0 con 0x7f53401478c0 2021-01-06T22:19:24.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.387+0000 7f533f59e700 1 -- 172.21.15.203:0/1567093699 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53400450a0 con 0x7f53401478c0 2021-01-06T22:19:24.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.391+0000 7f53367fc700 1 -- 172.21.15.203:0/1567093699 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f5328010070 con 0x7f53401478c0 2021-01-06T22:19:24.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.391+0000 7f53367fc700 1 --2- 172.21.15.203:0/1567093699 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5324056400 0x7f53240588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:24.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.391+0000 7f53367fc700 1 -- 172.21.15.203:0/1567093699 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7f532806f270 con 0x7f53401478c0 2021-01-06T22:19:24.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.391+0000 7f533e59c700 1 --2- 172.21.15.203:0/1567093699 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5324056400 0x7f53240588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:24.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.391+0000 7f533e59c700 1 --2- 172.21.15.203:0/1567093699 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5324056400 0x7f53240588c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f53300076a0 tx=0x7f5330001710).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:24.400 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.399+0000 7f53367fc700 1 -- 172.21.15.203:0/1567093699 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f5328044790 con 0x7f53401478c0 2021-01-06T22:19:24.607 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.603+0000 7f533f59e700 1 -- 172.21.15.203:0/1567093699 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f534005ca60 con 0x7f53401478c0 2021-01-06T22:19:24.618 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.607+0000 7f53367fc700 1 -- 172.21.15.203:0/1567093699 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+3229 (secure 0 0 0) 0x7f5328007cb0 con 0x7f53401478c0 2021-01-06T22:19:24.618 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.607+0000 7f533f59e700 1 -- 172.21.15.203:0/1567093699 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5324056400 msgr2=0x7f53240588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:24.618 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.607+0000 7f533f59e700 1 --2- 172.21.15.203:0/1567093699 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5324056400 0x7f53240588c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f53300076a0 tx=0x7f5330001710).stop 2021-01-06T22:19:24.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.607+0000 7f533f59e700 1 -- 172.21.15.203:0/1567093699 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53401478c0 msgr2=0x7f5340148fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:24.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.607+0000 7f533f59e700 1 --2- 172.21.15.203:0/1567093699 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53401478c0 0x7f5340148fc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f532800c6e0 tx=0x7f5328007b70).stop 2021-01-06T22:19:24.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.607+0000 7f533f59e700 1 -- 172.21.15.203:0/1567093699 shutdown_connections 2021-01-06T22:19:24.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.607+0000 7f533f59e700 1 --2- 172.21.15.203:0/1567093699 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f5324056400 0x7f53240588c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:24.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.607+0000 7f533f59e700 1 --2- 172.21.15.203:0/1567093699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f534014e310 0x7f53400b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:24.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.607+0000 7f533f59e700 1 --2- 172.21.15.203:0/1567093699 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5340064380 0x7f534013fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:24.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.607+0000 7f533f59e700 1 --2- 172.21.15.203:0/1567093699 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53401478c0 0x7f5340148fc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:24.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.607+0000 7f533f59e700 1 -- 172.21.15.203:0/1567093699 >> 172.21.15.203:0/1567093699 conn(0x7f53400667b0 msgr2=0x7f53400671a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:24.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.607+0000 7f533f59e700 1 -- 172.21.15.203:0/1567093699 shutdown_connections 2021-01-06T22:19:24.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:24.607+0000 7f533f59e700 1 -- 172.21.15.203:0/1567093699 wait complete. 2021-01-06T22:19:24.622 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 80 2021-01-06T22:19:24.622 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:19:24.622 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:19:23.646378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4724,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:reconnect","state_seq":202,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:active","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:19:24.625 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 80, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 80, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4724": { "addr": "172.21.15.91:6836/2428882934", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6834", "nonce": 2428882934, "type": "v2" }, { "addr": "172.21.15.91:6836", "nonce": 2428882934, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4724, "incarnation": 78, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:reconnect", "state_seq": 202 }, "gid_5261": { "addr": "172.21.15.91:6837/2136238890", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6835", "nonce": 2136238890, "type": "v2" }, { "addr": "172.21.15.91:6837", "nonce": 2136238890, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5261, "incarnation": 76, "join_fscid": -1, "name": "f", "rank": 2, "state": "up:active", "state_seq": 19 }, "gid_5632": { "addr": "172.21.15.203:6837/2424499644", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6836", "nonce": 2424499644, "type": "v2" }, { "addr": "172.21.15.203:6837", "nonce": 2424499644, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5632, "incarnation": 69, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 16 } }, "last_failure": 0, "last_failure_osd_epoch": 33, "max_file_size": 1099511627776, "max_mds": 3, "metadata_pool": 3, "modified": "2021-01-06T22:19:23.646378+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4724, "mds_1": 5632, "mds_2": 5261 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2021-01-06T22:19:26.627 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:19:26.763 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.759+0000 7fdd0d2cd700 1 Processor -- start 2021-01-06T22:19:26.763 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.759+0000 7fdd0d2cd700 1 -- start start 2021-01-06T22:19:26.766 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd0d2cd700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd080c44f0 0x7fdd080c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:26.766 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd0d2cd700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdd080c48d0 0x7fdd080c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:26.766 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd0d2cd700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd080c0f50 con 0x7fdd08060fc0 2021-01-06T22:19:26.766 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd0d2cd700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd080c1d60 con 0x7fdd080c48d0 2021-01-06T22:19:26.767 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd0d2cd700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd08066cf0 con 0x7fdd080c44f0 2021-01-06T22:19:26.767 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd0cacc700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fdd08060fc0 0x7fdd08150730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51634/0 (socket says 172.21.15.203:51634) 2021-01-06T22:19:26.767 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd0cacc700 1 -- 172.21.15.203:0/3448899637 learned_addr learned my addr 172.21.15.203:0/3448899637 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:19:26.767 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd07fff700 1 --2- 172.21.15.203:0/3448899637 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd080c44f0 0x7fdd080c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:26.767 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdcfffff700 1 --2- 172.21.15.203:0/3448899637 >> v2:172.21.15.91:3300/0 conn(0x7fdd080c48d0 0x7fdd080c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:26.768 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 -- 172.21.15.203:0/3448899637 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009217451 0 0) 0x7fdd080c0f50 con 0x7fdd08060fc0 2021-01-06T22:19:26.768 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 -- 172.21.15.203:0/3448899637 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce8003230 con 0x7fdd08060fc0 2021-01-06T22:19:26.768 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 -- 172.21.15.203:0/3448899637 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2164708471 0 0) 0x7fdce8003230 con 0x7fdd08060fc0 2021-01-06T22:19:26.768 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 -- 172.21.15.203:0/3448899637 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd080c44f0 msgr2=0x7fdd080c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:26.768 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 --2- 172.21.15.203:0/3448899637 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd080c44f0 0x7fdd080c1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:26.769 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 -- 172.21.15.203:0/3448899637 >> v2:172.21.15.91:3300/0 conn(0x7fdd080c48d0 msgr2=0x7fdd080c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:26.769 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 --2- 172.21.15.203:0/3448899637 >> v2:172.21.15.91:3300/0 conn(0x7fdd080c48d0 0x7fdd080c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:26.769 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 -- 172.21.15.203:0/3448899637 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd08060750 con 0x7fdd08060fc0 2021-01-06T22:19:26.769 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 -- 172.21.15.203:0/3448899637 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7fdd00001860 con 0x7fdd08060fc0 2021-01-06T22:19:26.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 -- 172.21.15.203:0/3448899637 >> v1:172.21.15.203:6789/0 conn(0x7fdd08060fc0 legacy=0x7fdd08150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:26.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 --2- 172.21.15.203:0/3448899637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdce8003720 0x7fdce8003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:26.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 --2- 172.21.15.203:0/3448899637 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd080c44f0 0x7fdce80044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:26.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 --2- 172.21.15.203:0/3448899637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdce80049c0 0x7fdce8004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:26.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 -- 172.21.15.203:0/3448899637 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd080c1d60 con 0x7fdce80049c0 2021-01-06T22:19:26.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 -- 172.21.15.203:0/3448899637 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd08066cf0 con 0x7fdce8003720 2021-01-06T22:19:26.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 -- 172.21.15.203:0/3448899637 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce80041c0 con 0x7fdd080c44f0 2021-01-06T22:19:26.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd07fff700 1 --2- 172.21.15.203:0/3448899637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdce8003720 0x7fdce8003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:26.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd0cacc700 1 --2- 172.21.15.203:0/3448899637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdce80049c0 0x7fdce8004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:26.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdcfffff700 1 --2- 172.21.15.203:0/3448899637 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd080c44f0 0x7fdce80044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:26.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd07fff700 1 -- 172.21.15.203:0/3448899637 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd080c44f0 msgr2=0x7fdce80044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:26.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd07fff700 1 --2- 172.21.15.203:0/3448899637 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd080c44f0 0x7fdce80044d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:26.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd07fff700 1 -- 172.21.15.203:0/3448899637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdce80049c0 msgr2=0x7fdce8004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:26.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd07fff700 1 --2- 172.21.15.203:0/3448899637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdce80049c0 0x7fdce8004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:26.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd07fff700 1 -- 172.21.15.203:0/3448899637 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce8003230 con 0x7fdce8003720 2021-01-06T22:19:26.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd0cacc700 1 --2- 172.21.15.203:0/3448899637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdce80049c0 0x7fdce8004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:19:26.774 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd07fff700 1 --2- 172.21.15.203:0/3448899637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdce8003720 0x7fdce8003b40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fdcf800bb90 tx=0x7fdcf8008040).ready entity=mon.1 client_cookie=a8de8461ae12e317 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:26.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 -- 172.21.15.203:0/3448899637 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf800b2b0 con 0x7fdce8003720 2021-01-06T22:19:26.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 -- 172.21.15.203:0/3448899637 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf800ea90 con 0x7fdce8003720 2021-01-06T22:19:26.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd077fe700 1 -- 172.21.15.203:0/3448899637 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf800ee30 con 0x7fdce8003720 2021-01-06T22:19:26.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd0d2cd700 1 -- 172.21.15.203:0/3448899637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdce8003720 msgr2=0x7fdce8003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:26.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd0d2cd700 1 --2- 172.21.15.203:0/3448899637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdce8003720 0x7fdce8003b40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fdcf800bb90 tx=0x7fdcf8008040).stop 2021-01-06T22:19:26.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd0cacc700 1 -- 172.21.15.203:0/3448899637 reap_dead start 2021-01-06T22:19:26.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd0d2cd700 1 -- 172.21.15.203:0/3448899637 shutdown_connections 2021-01-06T22:19:26.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd0d2cd700 1 -- 172.21.15.203:0/3448899637 >> 172.21.15.203:0/3448899637 conn(0x7fdd08056cd0 msgr2=0x7fdd08061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:26.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd0d2cd700 1 -- 172.21.15.203:0/3448899637 shutdown_connections 2021-01-06T22:19:26.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.763+0000 7fdd0d2cd700 1 -- 172.21.15.203:0/3448899637 wait complete. 2021-01-06T22:19:26.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.767+0000 7fdd0d2cd700 1 Processor -- start 2021-01-06T22:19:26.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.767+0000 7fdd0d2cd700 1 -- start start 2021-01-06T22:19:26.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.767+0000 7fdd0d2cd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd080c3620 0x7fdd081618f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:26.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.767+0000 7fdd0d2cd700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd08151970 0x7fdd08153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:26.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.767+0000 7fdd0d2cd700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd080c3a00 0x7fdd0815d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:26.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd0d2cd700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd080bb450 con 0x7fdd08151970 2021-01-06T22:19:26.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd0d2cd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0815dda0 con 0x7fdd080c3620 2021-01-06T22:19:26.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd0d2cd700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd081537f0 con 0x7fdd080c3a00 2021-01-06T22:19:26.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd07fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd080c3620 0x7fdd081618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:26.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd07fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd080c3620 0x7fdd081618f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53048/0 (socket says 172.21.15.203:53048) 2021-01-06T22:19:26.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd07fff700 1 -- 172.21.15.203:0/262120992 learned_addr learned my addr 172.21.15.203:0/262120992 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:26.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd0cacc700 1 --2- 172.21.15.203:0/262120992 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd080c3a00 0x7fdd0815d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:26.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdcfffff700 1 --2- 172.21.15.203:0/262120992 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd08151970 0x7fdd08153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:26.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd07fff700 1 -- 172.21.15.203:0/262120992 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd080c3a00 msgr2=0x7fdd0815d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:26.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd07fff700 1 --2- 172.21.15.203:0/262120992 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd080c3a00 0x7fdd0815d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:26.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd07fff700 1 -- 172.21.15.203:0/262120992 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd08151970 msgr2=0x7fdd08153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:26.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd07fff700 1 --2- 172.21.15.203:0/262120992 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd08151970 0x7fdd08153a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:26.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd07fff700 1 -- 172.21.15.203:0/262120992 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd08153010 con 0x7fdd080c3620 2021-01-06T22:19:26.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdcfffff700 1 --2- 172.21.15.203:0/262120992 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd08151970 0x7fdd08153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:19:26.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd07fff700 1 --2- 172.21.15.203:0/262120992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd080c3620 0x7fdd081618f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fdcf8014000 tx=0x7fdcf801a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:26.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd04ff9700 1 -- 172.21.15.203:0/262120992 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8025440 con 0x7fdd080c3620 2021-01-06T22:19:26.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd04ff9700 1 -- 172.21.15.203:0/262120992 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf80255a0 con 0x7fdd080c3620 2021-01-06T22:19:26.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd0d2cd700 1 -- 172.21.15.203:0/262120992 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd081532b0 con 0x7fdd080c3620 2021-01-06T22:19:26.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd04ff9700 1 -- 172.21.15.203:0/262120992 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8025940 con 0x7fdd080c3620 2021-01-06T22:19:26.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd0d2cd700 1 -- 172.21.15.203:0/262120992 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd08160f10 con 0x7fdd080c3620 2021-01-06T22:19:26.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd0d2cd700 1 -- 172.21.15.203:0/262120992 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd080450a0 con 0x7fdd080c3620 2021-01-06T22:19:26.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd04ff9700 1 -- 172.21.15.203:0/262120992 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fdcf801b420 con 0x7fdd080c3620 2021-01-06T22:19:26.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd04ff9700 1 --2- 172.21.15.203:0/262120992 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdcec0563d0 0x7fdcec058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:26.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdd04ff9700 1 -- 172.21.15.203:0/262120992 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7fdcf8071bb0 con 0x7fdd080c3620 2021-01-06T22:19:26.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.771+0000 7fdcfffff700 1 --2- 172.21.15.203:0/262120992 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdcec0563d0 0x7fdcec058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:26.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.775+0000 7fdcfffff700 1 --2- 172.21.15.203:0/262120992 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdcec0563d0 0x7fdcec058890 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fdd080575d0 tx=0x7fdcf0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:26.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:26.775+0000 7fdd04ff9700 1 -- 172.21.15.203:0/262120992 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fdcf8078080 con 0x7fdd080c3620 2021-01-06T22:19:27.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:27.003+0000 7fdd0d2cd700 1 -- 172.21.15.203:0/262120992 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd0805ca60 con 0x7fdd080c3620 2021-01-06T22:19:27.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:27.003+0000 7fdd04ff9700 1 -- 172.21.15.203:0/262120992 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+3226 (secure 0 0 0) 0x7fdcf8016070 con 0x7fdd080c3620 2021-01-06T22:19:27.006 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:19:27.007 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:19:25.667364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4724,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":204,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:active","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:19:27.008 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:27.007+0000 7fdd0d2cd700 1 -- 172.21.15.203:0/262120992 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdcec0563d0 msgr2=0x7fdcec058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:27.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:27.007+0000 7fdd0d2cd700 1 --2- 172.21.15.203:0/262120992 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdcec0563d0 0x7fdcec058890 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fdd080575d0 tx=0x7fdcf0008040).stop 2021-01-06T22:19:27.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:27.007+0000 7fdd0d2cd700 1 -- 172.21.15.203:0/262120992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd080c3620 msgr2=0x7fdd081618f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:27.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:27.007+0000 7fdd0d2cd700 1 --2- 172.21.15.203:0/262120992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd080c3620 0x7fdd081618f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fdcf8014000 tx=0x7fdcf801a000).stop 2021-01-06T22:19:27.009 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:27.007+0000 7fdd0d2cd700 1 -- 172.21.15.203:0/262120992 shutdown_connections 2021-01-06T22:19:27.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:27.007+0000 7fdd0d2cd700 1 --2- 172.21.15.203:0/262120992 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fdcec0563d0 0x7fdcec058890 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:27.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:27.007+0000 7fdd0d2cd700 1 --2- 172.21.15.203:0/262120992 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd080c3a00 0x7fdd0815d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:27.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:27.007+0000 7fdd0d2cd700 1 --2- 172.21.15.203:0/262120992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd080c3620 0x7fdd081618f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:27.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:27.007+0000 7fdd0d2cd700 1 --2- 172.21.15.203:0/262120992 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd08151970 0x7fdd08153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:27.010 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:27.007+0000 7fdd0d2cd700 1 -- 172.21.15.203:0/262120992 >> 172.21.15.203:0/262120992 conn(0x7fdd08056cd0 msgr2=0x7fdd080bc800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:27.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:27.007+0000 7fdd0d2cd700 1 -- 172.21.15.203:0/262120992 shutdown_connections 2021-01-06T22:19:27.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:27.007+0000 7fdd0d2cd700 1 -- 172.21.15.203:0/262120992 wait complete. 2021-01-06T22:19:27.011 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 82 2021-01-06T22:19:27.028 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=0, replacing gid=5539 2021-01-06T22:19:27.028 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.c 2021-01-06T22:19:30.545 INFO:tasks.ceph.mds.b.smithi091.stderr:2021-01-06T22:19:30.539+0000 7fe0dc57c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2021-01-06T22:19:30.545 INFO:tasks.ceph.mds.b.smithi091.stderr:2021-01-06T22:19:30.539+0000 7fe0dc57c700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2021-01-06T22:19:30.546 INFO:tasks.ceph.mds.b.smithi091.stderr:2021-01-06T22:19:30.539+0000 7fe0dc57c700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2021-01-06T22:19:32.028 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2021-01-06T22:19:32.029 INFO:tasks.ceph.mds.c:Restarting daemon 2021-01-06T22:19:32.029 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-01-06T22:19:32.032 INFO:tasks.ceph.mds.c:Started 2021-01-06T22:19:32.032 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:19:32.126 INFO:tasks.ceph.mds.c.smithi203.stdout:starting mds.c at 2021-01-06T22:19:32.167 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f9588700 1 Processor -- start 2021-01-06T22:19:32.168 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f9588700 1 -- start start 2021-01-06T22:19:32.169 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f9588700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc3f40be730 0x7fc3f40657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:32.170 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f9588700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3f414ba00 0x7fc3f414be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:32.170 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f9588700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f414ee80 con 0x7fc3f40be350 2021-01-06T22:19:32.170 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f9588700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f414f120 con 0x7fc3f40be730 2021-01-06T22:19:32.170 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f9588700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f414f3c0 con 0x7fc3f414ba00 2021-01-06T22:19:32.171 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3ebfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc3f40be730 0x7fc3f40657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:32.171 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3ebfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc3f40be730 0x7fc3f40657c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53084/0 (socket says 172.21.15.203:53084) 2021-01-06T22:19:32.171 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3ebfff700 1 -- 172.21.15.203:0/3342677569 learned_addr learned my addr 172.21.15.203:0/3342677569 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:32.171 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f8d87700 1 --2- 172.21.15.203:0/3342677569 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3f414ba00 0x7fc3f414be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:32.171 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3ebfff700 1 -- 172.21.15.203:0/3342677569 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3f414ba00 msgr2=0x7fc3f414be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:32.172 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3ebfff700 1 --2- 172.21.15.203:0/3342677569 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3f414ba00 0x7fc3f414be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:32.172 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3ebfff700 1 -- 172.21.15.203:0/3342677569 >> v1:172.21.15.91:6789/0 conn(0x7fc3f40be350 legacy=0x7fc3f40ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:32.172 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3ebfff700 1 -- 172.21.15.203:0/3342677569 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f414f660 con 0x7fc3f40be730 2021-01-06T22:19:32.172 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f8d87700 1 --2- 172.21.15.203:0/3342677569 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3f414ba00 0x7fc3f414be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:19:32.172 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3ebfff700 1 --2- 172.21.15.203:0/3342677569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f40be730 0x7fc3f40657c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7fc3dc00b9e0 tx=0x7fc3dc006ea0).ready entity=mon.1 client_cookie=afcd7d18f86e3fb server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:32.173 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f37fe700 1 -- 172.21.15.203:0/3342677569 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0098d0 con 0x7fc3f40be730 2021-01-06T22:19:32.173 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f37fe700 1 -- 172.21.15.203:0/3342677569 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc009a30 con 0x7fc3f40be730 2021-01-06T22:19:32.173 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f37fe700 1 -- 172.21.15.203:0/3342677569 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc009db0 con 0x7fc3f40be730 2021-01-06T22:19:32.175 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f9588700 1 -- 172.21.15.203:0/3342677569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f40be730 msgr2=0x7fc3f40657c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:32.176 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f9588700 1 --2- 172.21.15.203:0/3342677569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f40be730 0x7fc3f40657c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7fc3dc00b9e0 tx=0x7fc3dc006ea0).stop 2021-01-06T22:19:32.176 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f9588700 1 -- 172.21.15.203:0/3342677569 shutdown_connections 2021-01-06T22:19:32.176 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f9588700 1 --2- 172.21.15.203:0/3342677569 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3f414ba00 0x7fc3f414be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:32.176 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f9588700 1 --2- 172.21.15.203:0/3342677569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f40be730 0x7fc3f40657c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:32.177 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f9588700 1 -- 172.21.15.203:0/3342677569 >> 172.21.15.203:0/3342677569 conn(0x7fc3f4067e20 msgr2=0x7fc3f4068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:32.177 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f9588700 1 -- 172.21.15.203:0/3342677569 shutdown_connections 2021-01-06T22:19:32.177 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.167+0000 7fc3f9588700 1 -- 172.21.15.203:0/3342677569 wait complete. 2021-01-06T22:19:32.177 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f9588700 1 Processor -- start 2021-01-06T22:19:32.177 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f9588700 1 -- start start 2021-01-06T22:19:32.178 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f9588700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f40be350 0x7fc3f40b8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:32.178 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f9588700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3f40be730 0x7fc3f40b8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:32.178 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f9588700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3f414ba00 0x7fc3f414ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:32.178 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f9588700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40b6db0 con 0x7fc3f414ba00 2021-01-06T22:19:32.178 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f9588700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40b8c60 con 0x7fc3f40be350 2021-01-06T22:19:32.179 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f9588700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40b96c0 con 0x7fc3f40be730 2021-01-06T22:19:32.179 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f40be350 0x7fc3f40b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:32.179 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f40be350 0x7fc3f40b8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53088/0 (socket says 172.21.15.203:53088) 2021-01-06T22:19:32.179 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f3fff700 1 -- 172.21.15.203:0/2156604232 learned_addr learned my addr 172.21.15.203:0/2156604232 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:32.179 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3ebfff700 1 --2- 172.21.15.203:0/2156604232 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3f40be730 0x7fc3f40b8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:32.180 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f8d87700 1 --2- 172.21.15.203:0/2156604232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3f414ba00 0x7fc3f414ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:32.180 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f3fff700 1 -- 172.21.15.203:0/2156604232 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3f40be730 msgr2=0x7fc3f40b8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:32.180 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f3fff700 1 --2- 172.21.15.203:0/2156604232 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3f40be730 0x7fc3f40b8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:32.180 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f3fff700 1 -- 172.21.15.203:0/2156604232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3f414ba00 msgr2=0x7fc3f414ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:32.180 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f3fff700 1 --2- 172.21.15.203:0/2156604232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3f414ba00 0x7fc3f414ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:32.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f3fff700 1 -- 172.21.15.203:0/2156604232 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f40b9960 con 0x7fc3f40be350 2021-01-06T22:19:32.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f3fff700 1 --2- 172.21.15.203:0/2156604232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f40be350 0x7fc3f40b8280 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7fc3e400c740 tx=0x7fc3e4007c00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:32.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f0ff9700 1 -- 172.21.15.203:0/2156604232 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e400e560 con 0x7fc3f40be350 2021-01-06T22:19:32.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f0ff9700 1 -- 172.21.15.203:0/2156604232 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e400e6c0 con 0x7fc3f40be350 2021-01-06T22:19:32.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f0ff9700 1 -- 172.21.15.203:0/2156604232 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e400ea60 con 0x7fc3f40be350 2021-01-06T22:19:32.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f9588700 1 -- 172.21.15.203:0/2156604232 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f40b9c00 con 0x7fc3f40be350 2021-01-06T22:19:32.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.171+0000 7fc3f9588700 1 -- 172.21.15.203:0/2156604232 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f40ba140 con 0x7fc3f40be350 2021-01-06T22:19:32.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.175+0000 7fc3f9588700 1 -- 172.21.15.203:0/2156604232 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f40b80f0 con 0x7fc3f40be350 2021-01-06T22:19:32.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.175+0000 7fc3f0ff9700 1 -- 172.21.15.203:0/2156604232 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fc3e4010030 con 0x7fc3f40be350 2021-01-06T22:19:32.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.175+0000 7fc3f0ff9700 1 --2- 172.21.15.203:0/2156604232 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc3d80563d0 0x7fc3d8058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:32.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.175+0000 7fc3f0ff9700 1 -- 172.21.15.203:0/2156604232 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7fc3e406f750 con 0x7fc3f40be350 2021-01-06T22:19:32.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.175+0000 7fc3ebfff700 1 --2- 172.21.15.203:0/2156604232 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc3d80563d0 0x7fc3d8058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:32.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.175+0000 7fc3ebfff700 1 --2- 172.21.15.203:0/2156604232 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc3d80563d0 0x7fc3d8058890 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fc3f4157730 tx=0x7fc3dc016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:32.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.179+0000 7fc3f0ff9700 1 -- 172.21.15.203:0/2156604232 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fc3e4044c70 con 0x7fc3f40be350 2021-01-06T22:19:32.407 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.407+0000 7fc3f9588700 1 -- 172.21.15.203:0/2156604232 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3f40450a0 con 0x7fc3f40be350 2021-01-06T22:19:32.408 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.407+0000 7fc3f0ff9700 1 -- 172.21.15.203:0/2156604232 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+3226 (secure 0 0 0) 0x7fc3e4048290 con 0x7fc3f40be350 2021-01-06T22:19:32.409 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:19:32.409 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:19:25.667364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4724,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":204,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:active","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:19:32.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.407+0000 7fc3f9588700 1 -- 172.21.15.203:0/2156604232 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc3d80563d0 msgr2=0x7fc3d8058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:32.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.407+0000 7fc3f9588700 1 --2- 172.21.15.203:0/2156604232 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc3d80563d0 0x7fc3d8058890 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fc3f4157730 tx=0x7fc3dc016000).stop 2021-01-06T22:19:32.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.407+0000 7fc3f9588700 1 -- 172.21.15.203:0/2156604232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f40be350 msgr2=0x7fc3f40b8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:32.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.407+0000 7fc3f9588700 1 --2- 172.21.15.203:0/2156604232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f40be350 0x7fc3f40b8280 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7fc3e400c740 tx=0x7fc3e4007c00).stop 2021-01-06T22:19:32.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.407+0000 7fc3f9588700 1 -- 172.21.15.203:0/2156604232 shutdown_connections 2021-01-06T22:19:32.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.407+0000 7fc3f9588700 1 --2- 172.21.15.203:0/2156604232 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc3d80563d0 0x7fc3d8058890 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:32.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.407+0000 7fc3f9588700 1 --2- 172.21.15.203:0/2156604232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3f414ba00 0x7fc3f414ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:32.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.407+0000 7fc3f9588700 1 --2- 172.21.15.203:0/2156604232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f40be350 0x7fc3f40b8280 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:32.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.407+0000 7fc3f9588700 1 --2- 172.21.15.203:0/2156604232 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3f40be730 0x7fc3f40b8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:32.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.407+0000 7fc3f9588700 1 -- 172.21.15.203:0/2156604232 >> 172.21.15.203:0/2156604232 conn(0x7fc3f4067e20 msgr2=0x7fc3f40602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:32.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.407+0000 7fc3f9588700 1 -- 172.21.15.203:0/2156604232 shutdown_connections 2021-01-06T22:19:32.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:32.407+0000 7fc3f9588700 1 -- 172.21.15.203:0/2156604232 wait complete. 2021-01-06T22:19:32.416 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 82 2021-01-06T22:19:32.423 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2021-01-06T22:19:34.424 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:19:34.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2bca84700 1 Processor -- start 2021-01-06T22:19:34.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2bca84700 1 -- start start 2021-01-06T22:19:34.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2bca84700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2b80ba820 0x7fc2b80615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:34.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2bca84700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc2b8149130 0x7fc2b81495c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:34.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2bca84700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b80bd450 con 0x7fc2b80ba440 2021-01-06T22:19:34.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2bca84700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b8149ab0 con 0x7fc2b8149130 2021-01-06T22:19:34.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2bca84700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b8057140 con 0x7fc2b80ba820 2021-01-06T22:19:34.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2b659c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fc2b80ba440 0x7fc2b80b41a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51690/0 (socket says 172.21.15.203:51690) 2021-01-06T22:19:34.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2b659c700 1 -- 172.21.15.203:0/361925678 learned_addr learned my addr 172.21.15.203:0/361925678 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:19:34.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2b6d9d700 1 --2- 172.21.15.203:0/361925678 >> v2:172.21.15.203:3300/0 conn(0x7fc2b8149130 0x7fc2b81495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:34.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2b5d9b700 1 --2- 172.21.15.203:0/361925678 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2b80ba820 0x7fc2b80615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:34.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2b559a700 1 -- 172.21.15.203:0/361925678 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203712504 0 0) 0x7fc2b80bd450 con 0x7fc2b80ba440 2021-01-06T22:19:34.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2b559a700 1 -- 172.21.15.203:0/361925678 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc29c003230 con 0x7fc2b80ba440 2021-01-06T22:19:34.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2b6d9d700 1 -- 172.21.15.203:0/361925678 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2b80ba820 msgr2=0x7fc2b80615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:34.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2b6d9d700 1 --2- 172.21.15.203:0/361925678 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2b80ba820 0x7fc2b80615d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:34.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2b6d9d700 1 -- 172.21.15.203:0/361925678 >> v1:172.21.15.203:6789/0 conn(0x7fc2b80ba440 legacy=0x7fc2b80b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:34.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2b6d9d700 1 -- 172.21.15.203:0/361925678 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b80bc730 con 0x7fc2b8149130 2021-01-06T22:19:34.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2b6d9d700 1 --2- 172.21.15.203:0/361925678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b8149130 0x7fc2b81495c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7fc2b0009c90 tx=0x7fc2b00091d0).ready entity=mon.1 client_cookie=a0cb663110194f4a server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:34.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2b559a700 1 -- 172.21.15.203:0/361925678 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b000ba70 con 0x7fc2b8149130 2021-01-06T22:19:34.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2b559a700 1 -- 172.21.15.203:0/361925678 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b000bbd0 con 0x7fc2b8149130 2021-01-06T22:19:34.566 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2b559a700 1 -- 172.21.15.203:0/361925678 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0011400 con 0x7fc2b8149130 2021-01-06T22:19:34.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2bca84700 1 -- 172.21.15.203:0/361925678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b8149130 msgr2=0x7fc2b81495c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:34.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2bca84700 1 --2- 172.21.15.203:0/361925678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b8149130 0x7fc2b81495c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7fc2b0009c90 tx=0x7fc2b00091d0).stop 2021-01-06T22:19:34.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2bca84700 1 -- 172.21.15.203:0/361925678 shutdown_connections 2021-01-06T22:19:34.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2bca84700 1 --2- 172.21.15.203:0/361925678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b8149130 0x7fc2b81495c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:34.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2bca84700 1 --2- 172.21.15.203:0/361925678 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2b80ba820 0x7fc2b80615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:34.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2bca84700 1 -- 172.21.15.203:0/361925678 >> 172.21.15.203:0/361925678 conn(0x7fc2b80c0c50 msgr2=0x7fc2b80c1030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:34.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2bca84700 1 -- 172.21.15.203:0/361925678 shutdown_connections 2021-01-06T22:19:34.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.559+0000 7fc2bca84700 1 -- 172.21.15.203:0/361925678 wait complete. 2021-01-06T22:19:34.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2bca84700 1 Processor -- start 2021-01-06T22:19:34.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2bca84700 1 -- start start 2021-01-06T22:19:34.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2bca84700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2b80ba440 0x7fc2b8063630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:34.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2bca84700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2b80ba820 0x7fc2b815f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:34.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2bca84700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b8149130 0x7fc2b8144de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:34.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2bca84700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b8145e80 con 0x7fc2b80ba440 2021-01-06T22:19:34.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2bca84700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b81459b0 con 0x7fc2b8149130 2021-01-06T22:19:34.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2bca84700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b8067e70 con 0x7fc2b80ba820 2021-01-06T22:19:34.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2b6d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b8149130 0x7fc2b8144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:34.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2b6d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b8149130 0x7fc2b8144de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53106/0 (socket says 172.21.15.203:53106) 2021-01-06T22:19:34.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2b6d9d700 1 -- 172.21.15.203:0/3330858456 learned_addr learned my addr 172.21.15.203:0/3330858456 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:34.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2b5d9b700 1 --2- 172.21.15.203:0/3330858456 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2b80ba820 0x7fc2b815f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:34.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2b659c700 1 --2- 172.21.15.203:0/3330858456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2b80ba440 0x7fc2b8063630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:34.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2b6d9d700 1 -- 172.21.15.203:0/3330858456 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2b80ba820 msgr2=0x7fc2b815f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:34.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2b6d9d700 1 --2- 172.21.15.203:0/3330858456 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2b80ba820 0x7fc2b815f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:34.574 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2b6d9d700 1 -- 172.21.15.203:0/3330858456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2b80ba440 msgr2=0x7fc2b8063630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:34.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2b6d9d700 1 --2- 172.21.15.203:0/3330858456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2b80ba440 0x7fc2b8063630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:34.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2b6d9d700 1 -- 172.21.15.203:0/3330858456 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b815ed80 con 0x7fc2b8149130 2021-01-06T22:19:34.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2b6d9d700 1 --2- 172.21.15.203:0/3330858456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b8149130 0x7fc2b8144de0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7fc2b000c010 tx=0x7fc2b0017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:34.575 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2a67fc700 1 -- 172.21.15.203:0/3330858456 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b001c700 con 0x7fc2b8149130 2021-01-06T22:19:34.576 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2a67fc700 1 -- 172.21.15.203:0/3330858456 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b001c860 con 0x7fc2b8149130 2021-01-06T22:19:34.576 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2bca84700 1 -- 172.21.15.203:0/3330858456 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b8143430 con 0x7fc2b8149130 2021-01-06T22:19:34.576 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2a67fc700 1 -- 172.21.15.203:0/3330858456 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b001cbe0 con 0x7fc2b8149130 2021-01-06T22:19:34.576 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.563+0000 7fc2bca84700 1 -- 172.21.15.203:0/3330858456 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b8145be0 con 0x7fc2b8149130 2021-01-06T22:19:34.576 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.567+0000 7fc2bca84700 1 -- 172.21.15.203:0/3330858456 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b805d600 con 0x7fc2b8149130 2021-01-06T22:19:34.577 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.567+0000 7fc2a67fc700 1 -- 172.21.15.203:0/3330858456 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fc2b0024030 con 0x7fc2b8149130 2021-01-06T22:19:34.577 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.567+0000 7fc2a67fc700 1 --2- 172.21.15.203:0/3330858456 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc2940563d0 0x7fc294058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:34.577 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.567+0000 7fc2a67fc700 1 -- 172.21.15.203:0/3330858456 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7fc2b0070ed0 con 0x7fc2b8149130 2021-01-06T22:19:34.577 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.567+0000 7fc2b659c700 1 --2- 172.21.15.203:0/3330858456 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc2940563d0 0x7fc294058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:34.578 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.567+0000 7fc2b659c700 1 --2- 172.21.15.203:0/3330858456 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc2940563d0 0x7fc294058890 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fc2a8005590 tx=0x7fc2a8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:34.578 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.571+0000 7fc2a67fc700 1 -- 172.21.15.203:0/3330858456 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fc2b00464a0 con 0x7fc2b8149130 2021-01-06T22:19:34.797 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.795+0000 7fc2bca84700 1 -- 172.21.15.203:0/3330858456 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2b80646f0 con 0x7fc2b8149130 2021-01-06T22:19:34.798 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.795+0000 7fc2a67fc700 1 -- 172.21.15.203:0/3330858456 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+3226 (secure 0 0 0) 0x7fc2b0022070 con 0x7fc2b8149130 2021-01-06T22:19:34.799 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:19:34.799 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:19:25.667364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4724,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":204,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:active","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:19:34.801 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.799+0000 7fc2bca84700 1 -- 172.21.15.203:0/3330858456 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc2940563d0 msgr2=0x7fc294058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:34.801 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.799+0000 7fc2bca84700 1 --2- 172.21.15.203:0/3330858456 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc2940563d0 0x7fc294058890 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fc2a8005590 tx=0x7fc2a8009040).stop 2021-01-06T22:19:34.801 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.799+0000 7fc2bca84700 1 -- 172.21.15.203:0/3330858456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b8149130 msgr2=0x7fc2b8144de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:34.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.799+0000 7fc2bca84700 1 --2- 172.21.15.203:0/3330858456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b8149130 0x7fc2b8144de0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7fc2b000c010 tx=0x7fc2b0017000).stop 2021-01-06T22:19:34.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.799+0000 7fc2bca84700 1 -- 172.21.15.203:0/3330858456 shutdown_connections 2021-01-06T22:19:34.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.799+0000 7fc2bca84700 1 --2- 172.21.15.203:0/3330858456 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc2940563d0 0x7fc294058890 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:34.802 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.799+0000 7fc2bca84700 1 --2- 172.21.15.203:0/3330858456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b8149130 0x7fc2b8144de0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:34.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.799+0000 7fc2bca84700 1 --2- 172.21.15.203:0/3330858456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2b80ba440 0x7fc2b8063630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:34.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.799+0000 7fc2bca84700 1 --2- 172.21.15.203:0/3330858456 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2b80ba820 0x7fc2b815f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:34.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.799+0000 7fc2bca84700 1 -- 172.21.15.203:0/3330858456 >> 172.21.15.203:0/3330858456 conn(0x7fc2b80c0c50 msgr2=0x7fc2b8067530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:34.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.799+0000 7fc2bca84700 1 -- 172.21.15.203:0/3330858456 shutdown_connections 2021-01-06T22:19:34.803 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:34.799+0000 7fc2bca84700 1 -- 172.21.15.203:0/3330858456 wait complete. 2021-01-06T22:19:34.804 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 82 2021-01-06T22:19:34.819 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2021-01-06T22:19:36.820 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:19:36.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.943+0000 7fb1a3002700 1 Processor -- start 2021-01-06T22:19:36.946 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.943+0000 7fb1a3002700 1 -- start start 2021-01-06T22:19:36.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.943+0000 7fb1a3002700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb19c063010 0x7fb19c14e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:36.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.943+0000 7fb1a3002700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb19c0be1b0 0x7fb19c147790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:36.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.943+0000 7fb1a3002700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb19c0be990 con 0x7fb19c0633f0 2021-01-06T22:19:36.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.943+0000 7fb1a3002700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb19c14ec60 con 0x7fb19c063010 2021-01-06T22:19:36.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.943+0000 7fb1a3002700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb19c0be6f0 con 0x7fb19c0be1b0 2021-01-06T22:19:36.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.943+0000 7fb1a2801700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb19c0be1b0 0x7fb19c147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:36.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.943+0000 7fb1a2801700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb19c0be1b0 0x7fb19c147790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53118/0 (socket says 172.21.15.203:53118) 2021-01-06T22:19:36.949 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.943+0000 7fb1a2801700 1 -- 172.21.15.203:0/4144992182 learned_addr learned my addr 172.21.15.203:0/4144992182 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:36.950 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.943+0000 7fb1a2000700 1 --2- 172.21.15.203:0/4144992182 >> v2:172.21.15.91:3300/0 conn(0x7fb19c063010 0x7fb19c14e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:36.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a2801700 1 -- 172.21.15.203:0/4144992182 >> v2:172.21.15.91:3300/0 conn(0x7fb19c063010 msgr2=0x7fb19c14e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:36.951 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a2801700 1 --2- 172.21.15.203:0/4144992182 >> v2:172.21.15.91:3300/0 conn(0x7fb19c063010 0x7fb19c14e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:36.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a2801700 1 -- 172.21.15.203:0/4144992182 >> v1:172.21.15.91:6789/0 conn(0x7fb19c0633f0 legacy=0x7fb19c064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:19:36.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a2801700 1 -- 172.21.15.203:0/4144992182 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c0c1050 con 0x7fb19c0be1b0 2021-01-06T22:19:36.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a2801700 1 --2- 172.21.15.203:0/4144992182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb19c0be1b0 0x7fb19c147790 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fb19c0bdb10 tx=0x7fb194009290).ready entity=mon.1 client_cookie=6af9914aa17ed502 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:36.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a0ffe700 1 -- 172.21.15.203:0/4144992182 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19400bc10 con 0x7fb19c0be1b0 2021-01-06T22:19:36.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a0ffe700 1 -- 172.21.15.203:0/4144992182 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19400bd70 con 0x7fb19c0be1b0 2021-01-06T22:19:36.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a0ffe700 1 -- 172.21.15.203:0/4144992182 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194011610 con 0x7fb19c0be1b0 2021-01-06T22:19:36.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a3002700 1 -- 172.21.15.203:0/4144992182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb19c0be1b0 msgr2=0x7fb19c147790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:36.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a3002700 1 --2- 172.21.15.203:0/4144992182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb19c0be1b0 0x7fb19c147790 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fb19c0bdb10 tx=0x7fb194009290).stop 2021-01-06T22:19:36.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a3002700 1 -- 172.21.15.203:0/4144992182 shutdown_connections 2021-01-06T22:19:36.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a3002700 1 --2- 172.21.15.203:0/4144992182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb19c0be1b0 0x7fb19c147790 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:36.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a3002700 1 --2- 172.21.15.203:0/4144992182 >> v2:172.21.15.91:3300/0 conn(0x7fb19c063010 0x7fb19c14e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:36.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a3002700 1 -- 172.21.15.203:0/4144992182 >> 172.21.15.203:0/4144992182 conn(0x7fb19c067e20 msgr2=0x7fb19c068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:36.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a3002700 1 -- 172.21.15.203:0/4144992182 shutdown_connections 2021-01-06T22:19:36.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a3002700 1 -- 172.21.15.203:0/4144992182 wait complete. 2021-01-06T22:19:36.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a3002700 1 Processor -- start 2021-01-06T22:19:36.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.947+0000 7fb1a3002700 1 -- start start 2021-01-06T22:19:36.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a3002700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb19c063010 0x7fb19c14bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:36.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a3002700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb19c0633f0 0x7fb19c14d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:36.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a3002700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb19c0be1b0 0x7fb19c145160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:36.959 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a3002700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c1445a0 con 0x7fb19c0be1b0 2021-01-06T22:19:36.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a3002700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c144840 con 0x7fb19c063010 2021-01-06T22:19:36.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a3002700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c15d530 con 0x7fb19c0633f0 2021-01-06T22:19:36.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a2000700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb19c063010 0x7fb19c14bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:36.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a2000700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb19c063010 0x7fb19c14bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53122/0 (socket says 172.21.15.203:53122) 2021-01-06T22:19:36.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a2000700 1 -- 172.21.15.203:0/81343467 learned_addr learned my addr 172.21.15.203:0/81343467 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:19:36.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a2801700 1 --2- 172.21.15.203:0/81343467 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb19c0be1b0 0x7fb19c145160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:36.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a17ff700 1 --2- 172.21.15.203:0/81343467 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb19c0633f0 0x7fb19c14d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:36.961 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a2000700 1 -- 172.21.15.203:0/81343467 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb19c0633f0 msgr2=0x7fb19c14d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:36.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a2000700 1 --2- 172.21.15.203:0/81343467 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb19c0633f0 0x7fb19c14d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:36.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a2000700 1 -- 172.21.15.203:0/81343467 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb19c0be1b0 msgr2=0x7fb19c145160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:36.962 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a2000700 1 --2- 172.21.15.203:0/81343467 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb19c0be1b0 0x7fb19c145160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:36.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a2000700 1 -- 172.21.15.203:0/81343467 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c15d7d0 con 0x7fb19c063010 2021-01-06T22:19:36.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a2000700 1 --2- 172.21.15.203:0/81343467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb19c063010 0x7fb19c14bb90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7fb19800ebd0 tx=0x7fb198006de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:36.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb189ffb700 1 -- 172.21.15.203:0/81343467 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19800b8a0 con 0x7fb19c063010 2021-01-06T22:19:36.963 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb189ffb700 1 -- 172.21.15.203:0/81343467 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19800ba00 con 0x7fb19c063010 2021-01-06T22:19:36.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a3002700 1 -- 172.21.15.203:0/81343467 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19c15da70 con 0x7fb19c063010 2021-01-06T22:19:36.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb189ffb700 1 -- 172.21.15.203:0/81343467 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19800bd80 con 0x7fb19c063010 2021-01-06T22:19:36.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.951+0000 7fb1a3002700 1 -- 172.21.15.203:0/81343467 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19c14c080 con 0x7fb19c063010 2021-01-06T22:19:36.964 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.955+0000 7fb1a3002700 1 -- 172.21.15.203:0/81343467 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19c0450a0 con 0x7fb19c063010 2021-01-06T22:19:36.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.955+0000 7fb189ffb700 1 -- 172.21.15.203:0/81343467 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fb198012600 con 0x7fb19c063010 2021-01-06T22:19:36.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.955+0000 7fb189ffb700 1 --2- 172.21.15.203:0/81343467 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb17c0563d0 0x7fb17c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:19:36.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.955+0000 7fb189ffb700 1 -- 172.21.15.203:0/81343467 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7fb19806fbb0 con 0x7fb19c063010 2021-01-06T22:19:36.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.955+0000 7fb1a17ff700 1 --2- 172.21.15.203:0/81343467 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb17c0563d0 0x7fb17c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:19:36.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.955+0000 7fb1a17ff700 1 --2- 172.21.15.203:0/81343467 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb17c0563d0 0x7fb17c058890 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fb19c063840 tx=0x7fb18c007b60).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:19:36.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:36.959+0000 7fb189ffb700 1 -- 172.21.15.203:0/81343467 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fb198076080 con 0x7fb19c063010 2021-01-06T22:19:37.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:37.207+0000 7fb1a3002700 1 -- 172.21.15.203:0/81343467 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb19c061e70 con 0x7fb19c063010 2021-01-06T22:19:37.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:37.207+0000 7fb189ffb700 1 -- 172.21.15.203:0/81343467 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+3582 (secure 0 0 0) 0x7fb1980450d0 con 0x7fb19c063010 2021-01-06T22:19:37.210 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:19:37.211 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":83}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:19:25.667364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4724,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":204,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:active","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:19:37.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:37.211+0000 7fb1a3002700 1 -- 172.21.15.203:0/81343467 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb17c0563d0 msgr2=0x7fb17c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:37.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:37.211+0000 7fb1a3002700 1 --2- 172.21.15.203:0/81343467 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb17c0563d0 0x7fb17c058890 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fb19c063840 tx=0x7fb18c007b60).stop 2021-01-06T22:19:37.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:37.211+0000 7fb1a3002700 1 -- 172.21.15.203:0/81343467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb19c063010 msgr2=0x7fb19c14bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:19:37.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:37.211+0000 7fb1a3002700 1 --2- 172.21.15.203:0/81343467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb19c063010 0x7fb19c14bb90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7fb19800ebd0 tx=0x7fb198006de0).stop 2021-01-06T22:19:37.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:37.211+0000 7fb1a3002700 1 -- 172.21.15.203:0/81343467 shutdown_connections 2021-01-06T22:19:37.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:37.211+0000 7fb1a3002700 1 --2- 172.21.15.203:0/81343467 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb17c0563d0 0x7fb17c058890 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:37.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:37.211+0000 7fb1a3002700 1 --2- 172.21.15.203:0/81343467 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb19c0be1b0 0x7fb19c145160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:37.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:37.211+0000 7fb1a3002700 1 --2- 172.21.15.203:0/81343467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb19c063010 0x7fb19c14bb90 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:37.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:37.211+0000 7fb1a3002700 1 --2- 172.21.15.203:0/81343467 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb19c0633f0 0x7fb19c14d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:19:37.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:37.211+0000 7fb1a3002700 1 -- 172.21.15.203:0/81343467 >> 172.21.15.203:0/81343467 conn(0x7fb19c067e20 msgr2=0x7fb19c141e00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:19:37.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:37.211+0000 7fb1a3002700 1 -- 172.21.15.203:0/81343467 shutdown_connections 2021-01-06T22:19:37.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:19:37.211+0000 7fb1a3002700 1 -- 172.21.15.203:0/81343467 wait complete. 2021-01-06T22:19:37.216 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 83 2021-01-06T22:19:37.229 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2021-01-06T22:19:37.229 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 92 secs before thrashing 2021-01-06T22:19:48.514 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:19:48.516 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:20:18.556 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:20:18.559 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:20:50.504 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:20:50.507 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:21:09.231 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:09.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.356+0000 7fb252575700 1 Processor -- start 2021-01-06T22:21:09.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb252575700 1 -- start start 2021-01-06T22:21:09.365 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb252575700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb24c063010 0x7fb24c14e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:09.365 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb252575700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb24c0be990 con 0x7fb24c0633f0 2021-01-06T22:21:09.365 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb252575700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb24c0be6f0 con 0x7fb24c0be1b0 2021-01-06T22:21:09.366 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb252575700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c14ec60 con 0x7fb24c063010 2021-01-06T22:21:09.366 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb251d74700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fb24c0be1b0 0x7fb24c147790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51814/0 (socket says 172.21.15.203:51814) 2021-01-06T22:21:09.366 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb251d74700 1 -- 172.21.15.203:0/220111904 learned_addr learned my addr 172.21.15.203:0/220111904 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:21:09.366 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb251573700 1 --2- 172.21.15.203:0/220111904 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb24c063010 0x7fb24c14e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:09.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 -- 172.21.15.203:0/220111904 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196034600 0 0) 0x7fb24c0be6f0 con 0x7fb24c0be1b0 2021-01-06T22:21:09.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 -- 172.21.15.203:0/220111904 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb234003230 con 0x7fb24c0be1b0 2021-01-06T22:21:09.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 -- 172.21.15.203:0/220111904 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074797713 0 0) 0x7fb24c0be990 con 0x7fb24c0633f0 2021-01-06T22:21:09.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 -- 172.21.15.203:0/220111904 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb24c0be6f0 con 0x7fb24c0633f0 2021-01-06T22:21:09.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 -- 172.21.15.203:0/220111904 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3007625207 0 0) 0x7fb234003230 con 0x7fb24c0be1b0 2021-01-06T22:21:09.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 -- 172.21.15.203:0/220111904 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb24c063010 msgr2=0x7fb24c14e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:09.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 --2- 172.21.15.203:0/220111904 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb24c063010 0x7fb24c14e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:09.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 -- 172.21.15.203:0/220111904 >> v1:172.21.15.91:6789/0 conn(0x7fb24c0633f0 legacy=0x7fb24c064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:09.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 -- 172.21.15.203:0/220111904 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c0c1050 con 0x7fb24c0be1b0 2021-01-06T22:21:09.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 -- 172.21.15.203:0/220111904 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7fb2440014e0 con 0x7fb24c0be1b0 2021-01-06T22:21:09.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 -- 172.21.15.203:0/220111904 >> v1:172.21.15.203:6789/0 conn(0x7fb24c0be1b0 legacy=0x7fb24c147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:09.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 --2- 172.21.15.203:0/220111904 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2340038b0 0x7fb234003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:09.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 --2- 172.21.15.203:0/220111904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2340044d0 0x7fb234004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:09.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 --2- 172.21.15.203:0/220111904 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb24c063010 0x7fb234004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:09.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 -- 172.21.15.203:0/220111904 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c14ec60 con 0x7fb2340038b0 2021-01-06T22:21:09.373 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 -- 172.21.15.203:0/220111904 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb234005330 con 0x7fb2340044d0 2021-01-06T22:21:09.373 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb23bfff700 1 -- 172.21.15.203:0/220111904 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23400b720 con 0x7fb24c063010 2021-01-06T22:21:09.373 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb250d72700 1 --2- 172.21.15.203:0/220111904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2340044d0 0x7fb234004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:09.374 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb251573700 1 --2- 172.21.15.203:0/220111904 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2340038b0 0x7fb234003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:09.374 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb251d74700 1 --2- 172.21.15.203:0/220111904 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb24c063010 0x7fb234004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:09.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb250d72700 1 -- 172.21.15.203:0/220111904 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb24c063010 msgr2=0x7fb234004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:09.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb250d72700 1 --2- 172.21.15.203:0/220111904 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb24c063010 0x7fb234004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:09.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb250d72700 1 -- 172.21.15.203:0/220111904 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2340038b0 msgr2=0x7fb234003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:09.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb250d72700 1 --2- 172.21.15.203:0/220111904 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2340038b0 0x7fb234003cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:09.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb250d72700 1 -- 172.21.15.203:0/220111904 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb234003230 con 0x7fb2340044d0 2021-01-06T22:21:09.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.360+0000 7fb251d74700 1 --2- 172.21.15.203:0/220111904 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb24c063010 0x7fb234004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:21:09.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.364+0000 7fb250d72700 1 --2- 172.21.15.203:0/220111904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2340044d0 0x7fb234004950 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fb23c001a80 tx=0x7fb23c007b80).ready entity=mon.1 client_cookie=3dfab6f1361a0b6 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:09.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.364+0000 7fb23bfff700 1 -- 172.21.15.203:0/220111904 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c007020 con 0x7fb2340044d0 2021-01-06T22:21:09.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.364+0000 7fb23bfff700 1 -- 172.21.15.203:0/220111904 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23c007180 con 0x7fb2340044d0 2021-01-06T22:21:09.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.364+0000 7fb23bfff700 1 -- 172.21.15.203:0/220111904 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c00c850 con 0x7fb2340044d0 2021-01-06T22:21:09.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.364+0000 7fb252575700 1 -- 172.21.15.203:0/220111904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2340044d0 msgr2=0x7fb234004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:09.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.364+0000 7fb252575700 1 --2- 172.21.15.203:0/220111904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2340044d0 0x7fb234004950 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fb23c001a80 tx=0x7fb23c007b80).stop 2021-01-06T22:21:09.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.364+0000 7fb251d74700 1 -- 172.21.15.203:0/220111904 reap_dead start 2021-01-06T22:21:09.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.364+0000 7fb252575700 1 -- 172.21.15.203:0/220111904 shutdown_connections 2021-01-06T22:21:09.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.364+0000 7fb252575700 1 -- 172.21.15.203:0/220111904 >> 172.21.15.203:0/220111904 conn(0x7fb24c067e20 msgr2=0x7fb24c068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:09.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.364+0000 7fb252575700 1 -- 172.21.15.203:0/220111904 shutdown_connections 2021-01-06T22:21:09.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.364+0000 7fb252575700 1 -- 172.21.15.203:0/220111904 wait complete. 2021-01-06T22:21:09.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.364+0000 7fb252575700 1 Processor -- start 2021-01-06T22:21:09.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.364+0000 7fb252575700 1 -- start start 2021-01-06T22:21:09.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb252575700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2340044d0 0x7fb24c146e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:09.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb252575700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb24c147ff0 0x7fb24c1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:09.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb252575700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb24c159ed0 0x7fb24c140170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:09.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb252575700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c15a350 con 0x7fb24c147ff0 2021-01-06T22:21:09.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb252575700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c143d40 con 0x7fb2340044d0 2021-01-06T22:21:09.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb252575700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c142d20 con 0x7fb24c159ed0 2021-01-06T22:21:09.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb251573700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2340044d0 0x7fb24c146e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:09.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb251573700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2340044d0 0x7fb24c146e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53228/0 (socket says 172.21.15.203:53228) 2021-01-06T22:21:09.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb251573700 1 -- 172.21.15.203:0/1758808818 learned_addr learned my addr 172.21.15.203:0/1758808818 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:09.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb251d74700 1 --2- 172.21.15.203:0/1758808818 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb24c159ed0 0x7fb24c140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:09.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb250d72700 1 --2- 172.21.15.203:0/1758808818 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb24c147ff0 0x7fb24c1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:09.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb251573700 1 -- 172.21.15.203:0/1758808818 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb24c159ed0 msgr2=0x7fb24c140170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:09.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb251573700 1 --2- 172.21.15.203:0/1758808818 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb24c159ed0 0x7fb24c140170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:09.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb251573700 1 -- 172.21.15.203:0/1758808818 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb24c147ff0 msgr2=0x7fb24c1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:09.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb251573700 1 --2- 172.21.15.203:0/1758808818 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb24c147ff0 0x7fb24c1599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:09.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb251573700 1 -- 172.21.15.203:0/1758808818 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c143f70 con 0x7fb2340044d0 2021-01-06T22:21:09.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb251573700 1 --2- 172.21.15.203:0/1758808818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2340044d0 0x7fb24c146e00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fb248009ef0 tx=0x7fb2480091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:09.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb2397fa700 1 -- 172.21.15.203:0/1758808818 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24800fe30 con 0x7fb2340044d0 2021-01-06T22:21:09.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb2397fa700 1 -- 172.21.15.203:0/1758808818 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb248014440 con 0x7fb2340044d0 2021-01-06T22:21:09.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb252575700 1 -- 172.21.15.203:0/1758808818 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb24c142f80 con 0x7fb2340044d0 2021-01-06T22:21:09.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb2397fa700 1 -- 172.21.15.203:0/1758808818 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2480147c0 con 0x7fb2340044d0 2021-01-06T22:21:09.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb252575700 1 -- 172.21.15.203:0/1758808818 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24c1434c0 con 0x7fb2340044d0 2021-01-06T22:21:09.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb2397fa700 1 -- 172.21.15.203:0/1758808818 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fb248024070 con 0x7fb2340044d0 2021-01-06T22:21:09.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.368+0000 7fb252575700 1 -- 172.21.15.203:0/1758808818 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb24c0450a0 con 0x7fb2340044d0 2021-01-06T22:21:09.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.372+0000 7fb2397fa700 1 --2- 172.21.15.203:0/1758808818 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb22c056360 0x7fb22c058820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:09.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.372+0000 7fb2397fa700 1 -- 172.21.15.203:0/1758808818 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7fb2480704a0 con 0x7fb2340044d0 2021-01-06T22:21:09.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.372+0000 7fb250d72700 1 --2- 172.21.15.203:0/1758808818 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb22c056360 0x7fb22c058820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:09.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.372+0000 7fb250d72700 1 --2- 172.21.15.203:0/1758808818 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb22c056360 0x7fb22c058820 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7fb24c0651e0 tx=0x7fb23c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:09.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.376+0000 7fb2397fa700 1 -- 172.21.15.203:0/1758808818 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fb2480459c0 con 0x7fb2340044d0 2021-01-06T22:21:09.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.612+0000 7fb252575700 1 -- 172.21.15.203:0/1758808818 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb24c0beed0 con 0x7fb2340044d0 2021-01-06T22:21:09.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.612+0000 7fb2397fa700 1 -- 172.21.15.203:0/1758808818 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+3582 (secure 0 0 0) 0x7fb248048fe0 con 0x7fb2340044d0 2021-01-06T22:21:09.620 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:09.620 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":83}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:19:25.667364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4724,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":204,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:active","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:09.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.616+0000 7fb252575700 1 -- 172.21.15.203:0/1758808818 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb22c056360 msgr2=0x7fb22c058820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:09.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.616+0000 7fb252575700 1 --2- 172.21.15.203:0/1758808818 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb22c056360 0x7fb22c058820 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7fb24c0651e0 tx=0x7fb23c016000).stop 2021-01-06T22:21:09.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.616+0000 7fb252575700 1 -- 172.21.15.203:0/1758808818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2340044d0 msgr2=0x7fb24c146e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:09.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.616+0000 7fb252575700 1 --2- 172.21.15.203:0/1758808818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2340044d0 0x7fb24c146e00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fb248009ef0 tx=0x7fb2480091d0).stop 2021-01-06T22:21:09.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.616+0000 7fb252575700 1 -- 172.21.15.203:0/1758808818 shutdown_connections 2021-01-06T22:21:09.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.616+0000 7fb252575700 1 --2- 172.21.15.203:0/1758808818 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb22c056360 0x7fb22c058820 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:09.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.616+0000 7fb252575700 1 --2- 172.21.15.203:0/1758808818 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb24c159ed0 0x7fb24c140170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:09.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.616+0000 7fb252575700 1 --2- 172.21.15.203:0/1758808818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2340044d0 0x7fb24c146e00 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:09.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.616+0000 7fb252575700 1 --2- 172.21.15.203:0/1758808818 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb24c147ff0 0x7fb24c1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:09.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.616+0000 7fb252575700 1 -- 172.21.15.203:0/1758808818 >> 172.21.15.203:0/1758808818 conn(0x7fb24c067e20 msgr2=0x7fb24c0652e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:09.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.616+0000 7fb252575700 1 -- 172.21.15.203:0/1758808818 shutdown_connections 2021-01-06T22:21:09.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.616+0000 7fb252575700 1 -- 172.21.15.203:0/1758808818 wait complete. 2021-01-06T22:21:09.624 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 83 2021-01-06T22:21:09.638 INFO:tasks.mds_thrash.fs.[cephfs]:thrashing max_mds: 3 -> 2 2021-01-06T22:21:09.638 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2021-01-06T22:21:09.765 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.760+0000 7fe076ae0700 1 Processor -- start 2021-01-06T22:21:09.765 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.760+0000 7fe076ae0700 1 -- start start 2021-01-06T22:21:09.767 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.760+0000 7fe076ae0700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe070148d70 0x7fe0700c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:09.767 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.760+0000 7fe076ae0700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe0700602b0 0x7fe070149150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:09.767 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.760+0000 7fe076ae0700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe07014f0a0 con 0x7fe07014fa10 2021-01-06T22:21:09.768 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.760+0000 7fe076ae0700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe07014c3c0 con 0x7fe070148d70 2021-01-06T22:21:09.768 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.760+0000 7fe076ae0700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe07014c660 con 0x7fe0700602b0 2021-01-06T22:21:09.768 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.760+0000 7fe0752dd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe0700602b0 0x7fe070149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:09.768 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.760+0000 7fe0752dd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe0700602b0 0x7fe070149150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53238/0 (socket says 172.21.15.203:53238) 2021-01-06T22:21:09.768 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.760+0000 7fe0752dd700 1 -- 172.21.15.203:0/2221008797 learned_addr learned my addr 172.21.15.203:0/2221008797 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:09.769 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.760+0000 7fe075ade700 1 --2- 172.21.15.203:0/2221008797 >> v2:172.21.15.91:3300/0 conn(0x7fe070148d70 0x7fe0700c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:09.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe0752dd700 1 -- 172.21.15.203:0/2221008797 >> v2:172.21.15.91:3300/0 conn(0x7fe070148d70 msgr2=0x7fe0700c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:09.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe0752dd700 1 --2- 172.21.15.203:0/2221008797 >> v2:172.21.15.91:3300/0 conn(0x7fe070148d70 0x7fe0700c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:09.770 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe0752dd700 1 -- 172.21.15.203:0/2221008797 >> v1:172.21.15.203:6789/0 conn(0x7fe07014fa10 legacy=0x7fe07014bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:09.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe0752dd700 1 -- 172.21.15.203:0/2221008797 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07014c900 con 0x7fe0700602b0 2021-01-06T22:21:09.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe0752dd700 1 --2- 172.21.15.203:0/2221008797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0700602b0 0x7fe070149150 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7fe070065350 tx=0x7fe060006ea0).ready entity=mon.1 client_cookie=c39cc88d45f578e5 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:09.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe074adc700 1 -- 172.21.15.203:0/2221008797 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0600098a0 con 0x7fe0700602b0 2021-01-06T22:21:09.771 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe074adc700 1 -- 172.21.15.203:0/2221008797 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe060009a00 con 0x7fe0700602b0 2021-01-06T22:21:09.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe074adc700 1 -- 172.21.15.203:0/2221008797 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060009d80 con 0x7fe0700602b0 2021-01-06T22:21:09.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe076ae0700 1 -- 172.21.15.203:0/2221008797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0700602b0 msgr2=0x7fe070149150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:09.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe076ae0700 1 --2- 172.21.15.203:0/2221008797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0700602b0 0x7fe070149150 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7fe070065350 tx=0x7fe060006ea0).stop 2021-01-06T22:21:09.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe076ae0700 1 -- 172.21.15.203:0/2221008797 shutdown_connections 2021-01-06T22:21:09.772 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe076ae0700 1 --2- 172.21.15.203:0/2221008797 >> v2:172.21.15.91:3300/0 conn(0x7fe070148d70 0x7fe0700c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:09.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe076ae0700 1 --2- 172.21.15.203:0/2221008797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0700602b0 0x7fe070149150 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:09.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe076ae0700 1 -- 172.21.15.203:0/2221008797 >> 172.21.15.203:0/2221008797 conn(0x7fe0700667b0 msgr2=0x7fe0700c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:09.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe076ae0700 1 -- 172.21.15.203:0/2221008797 shutdown_connections 2021-01-06T22:21:09.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe076ae0700 1 -- 172.21.15.203:0/2221008797 wait complete. 2021-01-06T22:21:09.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.764+0000 7fe076ae0700 1 Processor -- start 2021-01-06T22:21:09.775 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe076ae0700 1 -- start start 2021-01-06T22:21:09.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe076ae0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0700602b0 0x7fe07015c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:09.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe076ae0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe070148d70 0x7fe0700bf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:09.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe076ae0700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe07014fa10 0x7fe07014ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:09.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe076ae0700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07015c780 con 0x7fe07014fa10 2021-01-06T22:21:09.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe076ae0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0701510a0 con 0x7fe0700602b0 2021-01-06T22:21:09.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe076ae0700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0701539d0 con 0x7fe070148d70 2021-01-06T22:21:09.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe075ade700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0700602b0 0x7fe07015c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:09.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe075ade700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0700602b0 0x7fe07015c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53242/0 (socket says 172.21.15.203:53242) 2021-01-06T22:21:09.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe075ade700 1 -- 172.21.15.203:0/2837058512 learned_addr learned my addr 172.21.15.203:0/2837058512 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:09.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe0762df700 1 --2- 172.21.15.203:0/2837058512 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe07014fa10 0x7fe07014ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:09.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe0752dd700 1 --2- 172.21.15.203:0/2837058512 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe070148d70 0x7fe0700bf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:09.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe075ade700 1 -- 172.21.15.203:0/2837058512 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe070148d70 msgr2=0x7fe0700bf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:09.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe075ade700 1 --2- 172.21.15.203:0/2837058512 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe070148d70 0x7fe0700bf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:09.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe075ade700 1 -- 172.21.15.203:0/2837058512 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe07014fa10 msgr2=0x7fe07014ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:09.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe075ade700 1 --2- 172.21.15.203:0/2837058512 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe07014fa10 0x7fe07014ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:09.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe075ade700 1 -- 172.21.15.203:0/2837058512 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe070153c70 con 0x7fe0700602b0 2021-01-06T22:21:09.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe075ade700 1 --2- 172.21.15.203:0/2837058512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0700602b0 0x7fe07015c290 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7fe06c00eb50 tx=0x7fe06c006de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:09.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe05dffb700 1 -- 172.21.15.203:0/2837058512 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c00ca40 con 0x7fe0700602b0 2021-01-06T22:21:09.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe05dffb700 1 -- 172.21.15.203:0/2837058512 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06c00cba0 con 0x7fe0700602b0 2021-01-06T22:21:09.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe076ae0700 1 -- 172.21.15.203:0/2837058512 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0701508c0 con 0x7fe0700602b0 2021-01-06T22:21:09.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe05dffb700 1 -- 172.21.15.203:0/2837058512 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c0123f0 con 0x7fe0700602b0 2021-01-06T22:21:09.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.768+0000 7fe076ae0700 1 -- 172.21.15.203:0/2837058512 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe070150e00 con 0x7fe0700602b0 2021-01-06T22:21:09.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.772+0000 7fe076ae0700 1 -- 172.21.15.203:0/2837058512 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0700450a0 con 0x7fe0700602b0 2021-01-06T22:21:09.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.772+0000 7fe05dffb700 1 -- 172.21.15.203:0/2837058512 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fe06c012760 con 0x7fe0700602b0 2021-01-06T22:21:09.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.772+0000 7fe05dffb700 1 --2- 172.21.15.203:0/2837058512 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe050056400 0x7fe0500588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:09.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.772+0000 7fe0752dd700 1 --2- 172.21.15.203:0/2837058512 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe050056400 0x7fe0500588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:09.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.772+0000 7fe05dffb700 1 -- 172.21.15.203:0/2837058512 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7fe06c06fc60 con 0x7fe0700602b0 2021-01-06T22:21:09.782 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.772+0000 7fe0752dd700 1 --2- 172.21.15.203:0/2837058512 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe050056400 0x7fe0500588c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fe060006ee0 tx=0x7fe060016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:09.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:09.780+0000 7fe05dffb700 1 -- 172.21.15.203:0/2837058512 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fe06c045180 con 0x7fe0700602b0 2021-01-06T22:21:10.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.020+0000 7fe076ae0700 1 -- 172.21.15.203:0/2837058512 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7fe070057340 con 0x7fe0700602b0 2021-01-06T22:21:10.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.464+0000 7fe05dffb700 1 -- 172.21.15.203:0/2837058512 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v84) v1 ==== 105+0+0 (secure 0 0 0) 0x7fe06c0487a0 con 0x7fe0700602b0 2021-01-06T22:21:10.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.464+0000 7fe076ae0700 1 -- 172.21.15.203:0/2837058512 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe050056400 msgr2=0x7fe0500588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:10.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.464+0000 7fe076ae0700 1 --2- 172.21.15.203:0/2837058512 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe050056400 0x7fe0500588c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fe060006ee0 tx=0x7fe060016000).stop 2021-01-06T22:21:10.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.464+0000 7fe076ae0700 1 -- 172.21.15.203:0/2837058512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0700602b0 msgr2=0x7fe07015c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:10.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.464+0000 7fe076ae0700 1 --2- 172.21.15.203:0/2837058512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0700602b0 0x7fe07015c290 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7fe06c00eb50 tx=0x7fe06c006de0).stop 2021-01-06T22:21:10.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.464+0000 7fe076ae0700 1 -- 172.21.15.203:0/2837058512 shutdown_connections 2021-01-06T22:21:10.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.464+0000 7fe076ae0700 1 --2- 172.21.15.203:0/2837058512 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe050056400 0x7fe0500588c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:10.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.464+0000 7fe076ae0700 1 --2- 172.21.15.203:0/2837058512 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe07014fa10 0x7fe07014ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:10.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.464+0000 7fe076ae0700 1 --2- 172.21.15.203:0/2837058512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0700602b0 0x7fe07015c290 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:10.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.464+0000 7fe076ae0700 1 --2- 172.21.15.203:0/2837058512 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe070148d70 0x7fe0700bf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:10.473 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.464+0000 7fe076ae0700 1 -- 172.21.15.203:0/2837058512 >> 172.21.15.203:0/2837058512 conn(0x7fe0700667b0 msgr2=0x7fe0700bdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:10.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.468+0000 7fe076ae0700 1 -- 172.21.15.203:0/2837058512 shutdown_connections 2021-01-06T22:21:10.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.468+0000 7fe076ae0700 1 -- 172.21.15.203:0/2837058512 wait complete. 2021-01-06T22:21:10.489 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2021-01-06T22:21:10.490 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:10.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03bd678700 1 Processor -- start 2021-01-06T22:21:10.621 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03bd678700 1 -- start start 2021-01-06T22:21:10.627 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03bd678700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03b80be730 0x7f03b80657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:10.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03bd678700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b814ee80 con 0x7f03b814ba00 2021-01-06T22:21:10.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03bd678700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b814f120 con 0x7f03b80be350 2021-01-06T22:21:10.628 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03bd678700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b814f3c0 con 0x7f03b80be730 2021-01-06T22:21:10.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b7fff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f03b80be350 0x7f03b80ba350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51844/0 (socket says 172.21.15.203:51844) 2021-01-06T22:21:10.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b7fff700 1 -- 172.21.15.203:0/3589055660 learned_addr learned my addr 172.21.15.203:0/3589055660 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:21:10.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b77fe700 1 --2- 172.21.15.203:0/3589055660 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03b80be730 0x7f03b80657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:10.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b6ffd700 1 -- 172.21.15.203:0/3589055660 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067841114 0 0) 0x7f03b814f120 con 0x7f03b80be350 2021-01-06T22:21:10.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b6ffd700 1 -- 172.21.15.203:0/3589055660 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0398003230 con 0x7f03b80be350 2021-01-06T22:21:10.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b6ffd700 1 -- 172.21.15.203:0/3589055660 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2578736711 0 0) 0x7f0398003230 con 0x7f03b80be350 2021-01-06T22:21:10.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b6ffd700 1 -- 172.21.15.203:0/3589055660 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03b80be730 msgr2=0x7f03b80657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:10.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b6ffd700 1 --2- 172.21.15.203:0/3589055660 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03b80be730 0x7f03b80657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:10.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b6ffd700 1 -- 172.21.15.203:0/3589055660 >> v1:172.21.15.91:6789/0 conn(0x7f03b814ba00 legacy=0x7f03b8148740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:10.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b6ffd700 1 -- 172.21.15.203:0/3589055660 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b814f660 con 0x7f03b80be350 2021-01-06T22:21:10.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b6ffd700 1 -- 172.21.15.203:0/3589055660 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f03a8001860 con 0x7f03b80be350 2021-01-06T22:21:10.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b6ffd700 1 -- 172.21.15.203:0/3589055660 >> v1:172.21.15.203:6789/0 conn(0x7f03b80be350 legacy=0x7f03b80ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:10.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b6ffd700 1 --2- 172.21.15.203:0/3589055660 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0398003720 0x7f0398003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:10.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b6ffd700 1 --2- 172.21.15.203:0/3589055660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03980044d0 0x7f0398004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:10.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b6ffd700 1 --2- 172.21.15.203:0/3589055660 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03b80be730 0x7f0398004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:10.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b6ffd700 1 -- 172.21.15.203:0/3589055660 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b814f3c0 con 0x7f0398003720 2021-01-06T22:21:10.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b6ffd700 1 -- 172.21.15.203:0/3589055660 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0398005380 con 0x7f03980044d0 2021-01-06T22:21:10.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b6ffd700 1 -- 172.21.15.203:0/3589055660 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0398004310 con 0x7f03b80be730 2021-01-06T22:21:10.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.616+0000 7f03b77fe700 1 --2- 172.21.15.203:0/3589055660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03980044d0 0x7f0398004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:10.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03b7fff700 1 --2- 172.21.15.203:0/3589055660 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0398003720 0x7f0398003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:10.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03bce77700 1 --2- 172.21.15.203:0/3589055660 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03b80be730 0x7f0398004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:10.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03b77fe700 1 -- 172.21.15.203:0/3589055660 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03b80be730 msgr2=0x7f0398004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:10.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03b77fe700 1 --2- 172.21.15.203:0/3589055660 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03b80be730 0x7f0398004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:10.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03b77fe700 1 -- 172.21.15.203:0/3589055660 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0398003720 msgr2=0x7f0398003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:10.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03b77fe700 1 --2- 172.21.15.203:0/3589055660 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0398003720 0x7f0398003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:10.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03b77fe700 1 -- 172.21.15.203:0/3589055660 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0398003230 con 0x7f03980044d0 2021-01-06T22:21:10.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03b77fe700 1 --2- 172.21.15.203:0/3589055660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03980044d0 0x7f0398004950 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f03a000bb90 tx=0x7f03a0008040).ready entity=mon.1 client_cookie=d8fae0ac198064c6 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:10.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03b6ffd700 1 -- 172.21.15.203:0/3589055660 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a000b2b0 con 0x7f03980044d0 2021-01-06T22:21:10.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03b6ffd700 1 -- 172.21.15.203:0/3589055660 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a000ea90 con 0x7f03980044d0 2021-01-06T22:21:10.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03b6ffd700 1 -- 172.21.15.203:0/3589055660 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a000ee30 con 0x7f03980044d0 2021-01-06T22:21:10.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03bd678700 1 -- 172.21.15.203:0/3589055660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03980044d0 msgr2=0x7f0398004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:10.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03bd678700 1 --2- 172.21.15.203:0/3589055660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03980044d0 0x7f0398004950 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f03a000bb90 tx=0x7f03a0008040).stop 2021-01-06T22:21:10.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03bce77700 1 -- 172.21.15.203:0/3589055660 reap_dead start 2021-01-06T22:21:10.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03bd678700 1 -- 172.21.15.203:0/3589055660 shutdown_connections 2021-01-06T22:21:10.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03bd678700 1 -- 172.21.15.203:0/3589055660 >> 172.21.15.203:0/3589055660 conn(0x7f03b8067e20 msgr2=0x7f03b8068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:10.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03bd678700 1 -- 172.21.15.203:0/3589055660 shutdown_connections 2021-01-06T22:21:10.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03bd678700 1 -- 172.21.15.203:0/3589055660 wait complete. 2021-01-06T22:21:10.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03bd678700 1 Processor -- start 2021-01-06T22:21:10.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.620+0000 7f03bd678700 1 -- start start 2021-01-06T22:21:10.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bd678700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03b813cb80 0x7f03b8139b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:10.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bd678700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03b813cf60 0x7f03b813a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:10.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bd678700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b813bd30 0x7f03b813c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:10.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bd678700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b81450f0 con 0x7f03b813cf60 2021-01-06T22:21:10.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bd678700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b813a4f0 con 0x7f03b813bd30 2021-01-06T22:21:10.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bd678700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b814b230 con 0x7f03b813cb80 2021-01-06T22:21:10.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bce77700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b813bd30 0x7f03b813c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:10.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bce77700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b813bd30 0x7f03b813c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53266/0 (socket says 172.21.15.203:53266) 2021-01-06T22:21:10.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bce77700 1 -- 172.21.15.203:0/640826276 learned_addr learned my addr 172.21.15.203:0/640826276 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:10.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03b77fe700 1 --2- 172.21.15.203:0/640826276 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03b813cf60 0x7f03b813a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:10.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03b7fff700 1 --2- 172.21.15.203:0/640826276 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03b813cb80 0x7f03b8139b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:10.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bce77700 1 -- 172.21.15.203:0/640826276 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03b813cb80 msgr2=0x7f03b8139b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:10.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bce77700 1 --2- 172.21.15.203:0/640826276 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03b813cb80 0x7f03b8139b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:10.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bce77700 1 -- 172.21.15.203:0/640826276 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03b813cf60 msgr2=0x7f03b813a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:10.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bce77700 1 --2- 172.21.15.203:0/640826276 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03b813cf60 0x7f03b813a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:10.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bce77700 1 -- 172.21.15.203:0/640826276 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b813c6a0 con 0x7f03b813bd30 2021-01-06T22:21:10.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bce77700 1 --2- 172.21.15.203:0/640826276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b813bd30 0x7f03b813c1b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f03b8065ce0 tx=0x7f03b0009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:10.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03affff700 1 -- 172.21.15.203:0/640826276 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0003d40 con 0x7f03b813bd30 2021-01-06T22:21:10.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03affff700 1 -- 172.21.15.203:0/640826276 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b0010450 con 0x7f03b813bd30 2021-01-06T22:21:10.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bd678700 1 -- 172.21.15.203:0/640826276 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b814b490 con 0x7f03b813bd30 2021-01-06T22:21:10.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03affff700 1 -- 172.21.15.203:0/640826276 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0010810 con 0x7f03b813bd30 2021-01-06T22:21:10.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.624+0000 7f03bd678700 1 -- 172.21.15.203:0/640826276 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b814a960 con 0x7f03b813bd30 2021-01-06T22:21:10.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.628+0000 7f03adffb700 1 -- 172.21.15.203:0/640826276 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b80450a0 con 0x7f03b813bd30 2021-01-06T22:21:10.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.628+0000 7f03affff700 1 -- 172.21.15.203:0/640826276 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f03b0010b80 con 0x7f03b813bd30 2021-01-06T22:21:10.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.628+0000 7f03affff700 1 --2- 172.21.15.203:0/640826276 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f039c0563d0 0x7f039c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:10.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.628+0000 7f03affff700 1 -- 172.21.15.203:0/640826276 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7f03b006f090 con 0x7f03b813bd30 2021-01-06T22:21:10.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.628+0000 7f03b7fff700 1 --2- 172.21.15.203:0/640826276 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f039c0563d0 0x7f039c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:10.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.628+0000 7f03b7fff700 1 --2- 172.21.15.203:0/640826276 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f039c0563d0 0x7f039c058890 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f03a8001150 tx=0x7f03a8007040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:10.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.636+0000 7f03affff700 1 -- 172.21.15.203:0/640826276 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f03b00445b0 con 0x7f03b813bd30 2021-01-06T22:21:10.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.872+0000 7f03adffb700 1 -- 172.21.15.203:0/640826276 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03b81491c0 con 0x7f03b813bd30 2021-01-06T22:21:10.877 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.872+0000 7f03affff700 1 -- 172.21.15.203:0/640826276 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+3584 (secure 0 0 0) 0x7f03b0007b50 con 0x7f03b813bd30 2021-01-06T22:21:10.878 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:10.879 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":83}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:10.465189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4724,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":204,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:stopping","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:10.880 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.876+0000 7f03adffb700 1 -- 172.21.15.203:0/640826276 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f039c0563d0 msgr2=0x7f039c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:10.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.876+0000 7f03adffb700 1 --2- 172.21.15.203:0/640826276 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f039c0563d0 0x7f039c058890 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f03a8001150 tx=0x7f03a8007040).stop 2021-01-06T22:21:10.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.876+0000 7f03adffb700 1 -- 172.21.15.203:0/640826276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b813bd30 msgr2=0x7f03b813c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:10.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.876+0000 7f03adffb700 1 --2- 172.21.15.203:0/640826276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b813bd30 0x7f03b813c1b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f03b8065ce0 tx=0x7f03b0009040).stop 2021-01-06T22:21:10.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.876+0000 7f03adffb700 1 -- 172.21.15.203:0/640826276 shutdown_connections 2021-01-06T22:21:10.881 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.876+0000 7f03adffb700 1 --2- 172.21.15.203:0/640826276 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f039c0563d0 0x7f039c058890 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:10.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.876+0000 7f03adffb700 1 --2- 172.21.15.203:0/640826276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b813bd30 0x7f03b813c1b0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:10.882 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.876+0000 7f03adffb700 1 --2- 172.21.15.203:0/640826276 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03b813cb80 0x7f03b8139b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:10.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.876+0000 7f03adffb700 1 --2- 172.21.15.203:0/640826276 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03b813cf60 0x7f03b813a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:10.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.876+0000 7f03adffb700 1 -- 172.21.15.203:0/640826276 >> 172.21.15.203:0/640826276 conn(0x7f03b8067e20 msgr2=0x7f03b80654d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:10.883 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.876+0000 7f03adffb700 1 -- 172.21.15.203:0/640826276 shutdown_connections 2021-01-06T22:21:10.884 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:10.876+0000 7f03adffb700 1 -- 172.21.15.203:0/640826276 wait complete. 2021-01-06T22:21:10.884 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 85 2021-01-06T22:21:10.895 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2021-01-06T22:21:10.896 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 85, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 85, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4724": { "addr": "172.21.15.91:6836/2428882934", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6834", "nonce": 2428882934, "type": "v2" }, { "addr": "172.21.15.91:6836", "nonce": 2428882934, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4724, "incarnation": 78, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 204 }, "gid_5261": { "addr": "172.21.15.91:6837/2136238890", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6835", "nonce": 2136238890, "type": "v2" }, { "addr": "172.21.15.91:6837", "nonce": 2136238890, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5261, "incarnation": 76, "join_fscid": -1, "name": "f", "rank": 2, "state": "up:stopping", "state_seq": 19 }, "gid_5632": { "addr": "172.21.15.203:6837/2424499644", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6836", "nonce": 2424499644, "type": "v2" }, { "addr": "172.21.15.203:6837", "nonce": 2424499644, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5632, "incarnation": 69, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 16 } }, "last_failure": 0, "last_failure_osd_epoch": 33, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2021-01-06T22:21:10.465189+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4724, "mds_1": 5632, "mds_2": 5261 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6835/3960786803", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6834", "nonce": 3960786803, "type": "v2" }, { "addr": "172.21.15.203:6835", "nonce": 3960786803, "type": "v1" } ] }, "epoch": 83, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5836, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 2 } ] } 2021-01-06T22:21:12.898 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:13.034 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f2037377700 1 Processor -- start 2021-01-06T22:21:13.034 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f2037377700 1 -- start start 2021-01-06T22:21:13.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f2037377700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2030064bc0 0x7f20300bf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:13.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f2037377700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2030151560 0x7f2030147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:13.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f2037377700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2030152460 con 0x7f20300bf100 2021-01-06T22:21:13.038 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f2037377700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f203014e740 con 0x7f2030064bc0 2021-01-06T22:21:13.038 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f2037377700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203014e9e0 con 0x7f2030151560 2021-01-06T22:21:13.038 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f2034912700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2030064bc0 0x7f20300bf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:13.038 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f2035914700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2030151560 0x7f2030147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:13.039 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f2034912700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2030064bc0 0x7f20300bf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:32888/0 (socket says 172.21.15.203:32888) 2021-01-06T22:21:13.039 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f2034912700 1 -- 172.21.15.203:0/3632006820 learned_addr learned my addr 172.21.15.203:0/3632006820 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:13.039 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f2035914700 1 -- 172.21.15.203:0/3632006820 >> v2:172.21.15.91:3300/0 conn(0x7f2030064bc0 msgr2=0x7f20300bf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:13.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f2035914700 1 --2- 172.21.15.203:0/3632006820 >> v2:172.21.15.91:3300/0 conn(0x7f2030064bc0 0x7f20300bf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:13.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f2035914700 1 -- 172.21.15.203:0/3632006820 >> v1:172.21.15.91:6789/0 conn(0x7f20300bf100 legacy=0x7f203014df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:13.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f2035914700 1 -- 172.21.15.203:0/3632006820 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203014ec80 con 0x7f2030151560 2021-01-06T22:21:13.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f2035914700 1 --2- 172.21.15.203:0/3632006820 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2030151560 0x7f2030147bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f2028009c90 tx=0x7f20280091d0).ready entity=mon.2 client_cookie=17dbad104f0fd265 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:13.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f201ffff700 1 -- 172.21.15.203:0/3632006820 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202800ba70 con 0x7f2030151560 2021-01-06T22:21:13.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f201ffff700 1 -- 172.21.15.203:0/3632006820 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202800bbd0 con 0x7f2030151560 2021-01-06T22:21:13.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.028+0000 7f201ffff700 1 -- 172.21.15.203:0/3632006820 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028011400 con 0x7f2030151560 2021-01-06T22:21:13.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.032+0000 7f2037377700 1 -- 172.21.15.203:0/3632006820 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2030151560 msgr2=0x7f2030147bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:13.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.032+0000 7f2037377700 1 --2- 172.21.15.203:0/3632006820 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2030151560 0x7f2030147bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f2028009c90 tx=0x7f20280091d0).stop 2021-01-06T22:21:13.042 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.032+0000 7f2037377700 1 -- 172.21.15.203:0/3632006820 shutdown_connections 2021-01-06T22:21:13.042 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.032+0000 7f2037377700 1 --2- 172.21.15.203:0/3632006820 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2030151560 0x7f2030147bf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:13.042 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.032+0000 7f2037377700 1 --2- 172.21.15.203:0/3632006820 >> v2:172.21.15.91:3300/0 conn(0x7f2030064bc0 0x7f20300bf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:13.042 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.032+0000 7f2037377700 1 -- 172.21.15.203:0/3632006820 >> 172.21.15.203:0/3632006820 conn(0x7f20300602b0 msgr2=0x7f20300606c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:13.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.032+0000 7f2037377700 1 -- 172.21.15.203:0/3632006820 shutdown_connections 2021-01-06T22:21:13.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.032+0000 7f2037377700 1 -- 172.21.15.203:0/3632006820 wait complete. 2021-01-06T22:21:13.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.032+0000 7f2037377700 1 Processor -- start 2021-01-06T22:21:13.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.032+0000 7f2037377700 1 -- start start 2021-01-06T22:21:13.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.032+0000 7f2037377700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2030064bc0 0x7f20300c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:13.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.032+0000 7f2037377700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f20300bf100 0x7f203014a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:13.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.032+0000 7f2037377700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2030151560 0x7f203014ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:13.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2037377700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20300babe0 con 0x7f2030151560 2021-01-06T22:21:13.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2037377700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2030150af0 con 0x7f2030064bc0 2021-01-06T22:21:13.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2037377700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203014b240 con 0x7f20300bf100 2021-01-06T22:21:13.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2035113700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2030064bc0 0x7f20300c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:13.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2035113700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2030064bc0 0x7f20300c38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53276/0 (socket says 172.21.15.203:53276) 2021-01-06T22:21:13.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2035113700 1 -- 172.21.15.203:0/2949341568 learned_addr learned my addr 172.21.15.203:0/2949341568 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:13.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2034912700 1 --2- 172.21.15.203:0/2949341568 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f20300bf100 0x7f203014a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:13.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2035914700 1 --2- 172.21.15.203:0/2949341568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2030151560 0x7f203014ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:13.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2035113700 1 -- 172.21.15.203:0/2949341568 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f20300bf100 msgr2=0x7f203014a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:13.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2035113700 1 --2- 172.21.15.203:0/2949341568 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f20300bf100 0x7f203014a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:13.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2035113700 1 -- 172.21.15.203:0/2949341568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2030151560 msgr2=0x7f203014ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:13.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2035113700 1 --2- 172.21.15.203:0/2949341568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2030151560 0x7f203014ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:13.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2035113700 1 -- 172.21.15.203:0/2949341568 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20300b1330 con 0x7f2030064bc0 2021-01-06T22:21:13.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2035113700 1 --2- 172.21.15.203:0/2949341568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2030064bc0 0x7f20300c38a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f202c00c740 tx=0x7f202c007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:13.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f201d7fa700 1 -- 172.21.15.203:0/2949341568 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c007020 con 0x7f2030064bc0 2021-01-06T22:21:13.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f201d7fa700 1 -- 172.21.15.203:0/2949341568 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c007180 con 0x7f2030064bc0 2021-01-06T22:21:13.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f201d7fa700 1 -- 172.21.15.203:0/2949341568 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c0064e0 con 0x7f2030064bc0 2021-01-06T22:21:13.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2037377700 1 -- 172.21.15.203:0/2949341568 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203015e400 con 0x7f2030064bc0 2021-01-06T22:21:13.051 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2037377700 1 -- 172.21.15.203:0/2949341568 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20300b1560 con 0x7f2030064bc0 2021-01-06T22:21:13.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f201d7fa700 1 -- 172.21.15.203:0/2949341568 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f202c010070 con 0x7f2030064bc0 2021-01-06T22:21:13.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2037377700 1 -- 172.21.15.203:0/2949341568 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203005d600 con 0x7f2030064bc0 2021-01-06T22:21:13.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f201d7fa700 1 --2- 172.21.15.203:0/2949341568 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2010056400 0x7f20100588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:13.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f201d7fa700 1 -- 172.21.15.203:0/2949341568 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7f202c06ea80 con 0x7f2030064bc0 2021-01-06T22:21:13.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.036+0000 7f2034912700 1 --2- 172.21.15.203:0/2949341568 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2010056400 0x7f20100588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:13.053 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.040+0000 7f2034912700 1 --2- 172.21.15.203:0/2949341568 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2010056400 0x7f20100588c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f2030147bc0 tx=0x7f2020006ea0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:13.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.044+0000 7f201d7fa700 1 -- 172.21.15.203:0/2949341568 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f202c043670 con 0x7f2030064bc0 2021-01-06T22:21:13.274 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.268+0000 7f2037377700 1 -- 172.21.15.203:0/2949341568 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20300612c0 con 0x7f2030064bc0 2021-01-06T22:21:13.275 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.268+0000 7f201d7fa700 1 -- 172.21.15.203:0/2949341568 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+3584 (secure 0 0 0) 0x7f202c043fa0 con 0x7f2030064bc0 2021-01-06T22:21:13.275 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:13.275 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":83}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:10.465189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4724,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":204,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:stopping","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:13.277 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.272+0000 7f2037377700 1 -- 172.21.15.203:0/2949341568 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2010056400 msgr2=0x7f20100588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:13.277 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.272+0000 7f2037377700 1 --2- 172.21.15.203:0/2949341568 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2010056400 0x7f20100588c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f2030147bc0 tx=0x7f2020006ea0).stop 2021-01-06T22:21:13.277 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.272+0000 7f2037377700 1 -- 172.21.15.203:0/2949341568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2030064bc0 msgr2=0x7f20300c38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:13.277 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.272+0000 7f2037377700 1 --2- 172.21.15.203:0/2949341568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2030064bc0 0x7f20300c38a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f202c00c740 tx=0x7f202c007c40).stop 2021-01-06T22:21:13.278 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.272+0000 7f2037377700 1 -- 172.21.15.203:0/2949341568 shutdown_connections 2021-01-06T22:21:13.278 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.272+0000 7f2037377700 1 --2- 172.21.15.203:0/2949341568 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2010056400 0x7f20100588c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:13.278 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.272+0000 7f2037377700 1 --2- 172.21.15.203:0/2949341568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2030151560 0x7f203014ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:13.278 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.272+0000 7f2037377700 1 --2- 172.21.15.203:0/2949341568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2030064bc0 0x7f20300c38a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:13.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.272+0000 7f2037377700 1 --2- 172.21.15.203:0/2949341568 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f20300bf100 0x7f203014a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:13.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.272+0000 7f2037377700 1 -- 172.21.15.203:0/2949341568 >> 172.21.15.203:0/2949341568 conn(0x7f20300602b0 msgr2=0x7f2030066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:13.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.272+0000 7f2037377700 1 -- 172.21.15.203:0/2949341568 shutdown_connections 2021-01-06T22:21:13.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:13.272+0000 7f2037377700 1 -- 172.21.15.203:0/2949341568 wait complete. 2021-01-06T22:21:13.280 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 85 2021-01-06T22:21:13.293 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:21:15.294 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:15.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.436+0000 7ff186cac700 1 Processor -- start 2021-01-06T22:21:15.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.436+0000 7ff186cac700 1 -- start start 2021-01-06T22:21:15.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff186cac700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff180064380 0x7ff1800c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:15.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff186cac700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff18014e310 0x7ff1801471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:15.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff186cac700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff180064760 0x7ff1801476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:15.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff186cac700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff180147b90 con 0x7ff180064380 2021-01-06T22:21:15.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff186cac700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1800bb2d0 con 0x7ff18014e310 2021-01-06T22:21:15.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff186cac700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18014a560 con 0x7ff180064760 2021-01-06T22:21:15.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff1854a9700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff18014e310 0x7ff1801471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:15.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff1854a9700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff18014e310 0x7ff1801471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53290/0 (socket says 172.21.15.203:53290) 2021-01-06T22:21:15.451 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff1854a9700 1 -- 172.21.15.203:0/1072321856 learned_addr learned my addr 172.21.15.203:0/1072321856 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:15.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff185caa700 1 --2- 172.21.15.203:0/1072321856 >> v2:172.21.15.91:3300/0 conn(0x7ff180064380 0x7ff1800c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:15.452 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff1864ab700 1 --2- 172.21.15.203:0/1072321856 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff180064760 0x7ff1801476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:15.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff1854a9700 1 -- 172.21.15.203:0/1072321856 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff180064760 msgr2=0x7ff1801476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:15.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff1854a9700 1 --2- 172.21.15.203:0/1072321856 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff180064760 0x7ff1801476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:15.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff1854a9700 1 -- 172.21.15.203:0/1072321856 >> v2:172.21.15.91:3300/0 conn(0x7ff180064380 msgr2=0x7ff1800c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:15.454 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff1854a9700 1 --2- 172.21.15.203:0/1072321856 >> v2:172.21.15.91:3300/0 conn(0x7ff180064380 0x7ff1800c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:15.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff1854a9700 1 -- 172.21.15.203:0/1072321856 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18014a800 con 0x7ff18014e310 2021-01-06T22:21:15.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff1864ab700 1 --2- 172.21.15.203:0/1072321856 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff180064760 0x7ff1801476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:21:15.455 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff1854a9700 1 --2- 172.21.15.203:0/1072321856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff18014e310 0x7ff1801471b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7ff180061c90 tx=0x7ff170006f30).ready entity=mon.1 client_cookie=c3a33d5415950f72 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:15.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff184ca8700 1 -- 172.21.15.203:0/1072321856 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17000cb60 con 0x7ff18014e310 2021-01-06T22:21:15.456 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff184ca8700 1 -- 172.21.15.203:0/1072321856 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17000ccc0 con 0x7ff18014e310 2021-01-06T22:21:15.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff184ca8700 1 -- 172.21.15.203:0/1072321856 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170010620 con 0x7ff18014e310 2021-01-06T22:21:15.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff186cac700 1 -- 172.21.15.203:0/1072321856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff18014e310 msgr2=0x7ff1801471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:15.457 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff186cac700 1 --2- 172.21.15.203:0/1072321856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff18014e310 0x7ff1801471b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7ff180061c90 tx=0x7ff170006f30).stop 2021-01-06T22:21:15.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff186cac700 1 -- 172.21.15.203:0/1072321856 shutdown_connections 2021-01-06T22:21:15.458 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff186cac700 1 --2- 172.21.15.203:0/1072321856 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff180064760 0x7ff1801476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:15.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff186cac700 1 --2- 172.21.15.203:0/1072321856 >> v2:172.21.15.91:3300/0 conn(0x7ff180064380 0x7ff1800c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:15.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff186cac700 1 --2- 172.21.15.203:0/1072321856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff18014e310 0x7ff1801471b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:15.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff186cac700 1 -- 172.21.15.203:0/1072321856 >> 172.21.15.203:0/1072321856 conn(0x7ff1800667b0 msgr2=0x7ff1800c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:15.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff186cac700 1 -- 172.21.15.203:0/1072321856 shutdown_connections 2021-01-06T22:21:15.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.440+0000 7ff186cac700 1 -- 172.21.15.203:0/1072321856 wait complete. 2021-01-06T22:21:15.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff186cac700 1 Processor -- start 2021-01-06T22:21:15.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff186cac700 1 -- start start 2021-01-06T22:21:15.465 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff186cac700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff180064380 0x7ff1800b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:15.466 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff186cac700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff180064760 0x7ff1800b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:15.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff186cac700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff18014e310 0x7ff180145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:15.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff186cac700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1800b46a0 con 0x7ff18014e310 2021-01-06T22:21:15.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff186cac700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1800bfbb0 con 0x7ff180064760 2021-01-06T22:21:15.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff186cac700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1800bf910 con 0x7ff180064380 2021-01-06T22:21:15.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff1854a9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff180064760 0x7ff1800b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:15.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff1854a9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff180064760 0x7ff1800b8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53296/0 (socket says 172.21.15.203:53296) 2021-01-06T22:21:15.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff1854a9700 1 -- 172.21.15.203:0/3600872324 learned_addr learned my addr 172.21.15.203:0/3600872324 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:15.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff185caa700 1 --2- 172.21.15.203:0/3600872324 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff180064380 0x7ff1800b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:15.468 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff1864ab700 1 --2- 172.21.15.203:0/3600872324 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff18014e310 0x7ff180145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:15.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff1854a9700 1 -- 172.21.15.203:0/3600872324 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff180064380 msgr2=0x7ff1800b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:15.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff1854a9700 1 --2- 172.21.15.203:0/3600872324 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff180064380 0x7ff1800b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:15.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff1854a9700 1 -- 172.21.15.203:0/3600872324 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff18014e310 msgr2=0x7ff180145580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:15.469 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff1854a9700 1 --2- 172.21.15.203:0/3600872324 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff18014e310 0x7ff180145580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:15.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff1854a9700 1 -- 172.21.15.203:0/3600872324 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff180144040 con 0x7ff180064760 2021-01-06T22:21:15.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff1854a9700 1 --2- 172.21.15.203:0/3600872324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff180064760 0x7ff1800b8ed0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7ff1800635e0 tx=0x7ff170006f30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:15.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff16dffb700 1 -- 172.21.15.203:0/3600872324 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1700105e0 con 0x7ff180064760 2021-01-06T22:21:15.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff16dffb700 1 -- 172.21.15.203:0/3600872324 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff170019630 con 0x7ff180064760 2021-01-06T22:21:15.470 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff186cac700 1 -- 172.21.15.203:0/3600872324 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1801442e0 con 0x7ff180064760 2021-01-06T22:21:15.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.444+0000 7ff16dffb700 1 -- 172.21.15.203:0/3600872324 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1700199f0 con 0x7ff180064760 2021-01-06T22:21:15.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.448+0000 7ff186cac700 1 -- 172.21.15.203:0/3600872324 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1800b5d30 con 0x7ff180064760 2021-01-06T22:21:15.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.448+0000 7ff186cac700 1 -- 172.21.15.203:0/3600872324 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1800450a0 con 0x7ff180064760 2021-01-06T22:21:15.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.448+0000 7ff16dffb700 1 -- 172.21.15.203:0/3600872324 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff17000e040 con 0x7ff180064760 2021-01-06T22:21:15.471 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.448+0000 7ff16dffb700 1 --2- 172.21.15.203:0/3600872324 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff1600563d0 0x7ff160058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:15.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.448+0000 7ff16dffb700 1 -- 172.21.15.203:0/3600872324 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7ff1700708b0 con 0x7ff180064760 2021-01-06T22:21:15.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.448+0000 7ff185caa700 1 --2- 172.21.15.203:0/3600872324 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff1600563d0 0x7ff160058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:15.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.448+0000 7ff185caa700 1 --2- 172.21.15.203:0/3600872324 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff1600563d0 0x7ff160058890 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7ff180146e00 tx=0x7ff17c006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:15.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.452+0000 7ff16dffb700 1 -- 172.21.15.203:0/3600872324 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff170045dd0 con 0x7ff180064760 2021-01-06T22:21:15.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.688+0000 7ff186cac700 1 -- 172.21.15.203:0/3600872324 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff180145e60 con 0x7ff180064760 2021-01-06T22:21:15.693 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.688+0000 7ff16dffb700 1 -- 172.21.15.203:0/3600872324 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+3584 (secure 0 0 0) 0x7ff170014020 con 0x7ff180064760 2021-01-06T22:21:15.694 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:15.694 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":83}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:10.465189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4724,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":204,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:stopping","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:15.696 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.688+0000 7ff186cac700 1 -- 172.21.15.203:0/3600872324 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff1600563d0 msgr2=0x7ff160058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:15.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.688+0000 7ff186cac700 1 --2- 172.21.15.203:0/3600872324 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff1600563d0 0x7ff160058890 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7ff180146e00 tx=0x7ff17c006de0).stop 2021-01-06T22:21:15.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.688+0000 7ff186cac700 1 -- 172.21.15.203:0/3600872324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff180064760 msgr2=0x7ff1800b8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:15.697 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.688+0000 7ff186cac700 1 --2- 172.21.15.203:0/3600872324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff180064760 0x7ff1800b8ed0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7ff1800635e0 tx=0x7ff170006f30).stop 2021-01-06T22:21:15.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.692+0000 7ff186cac700 1 -- 172.21.15.203:0/3600872324 shutdown_connections 2021-01-06T22:21:15.698 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.692+0000 7ff186cac700 1 --2- 172.21.15.203:0/3600872324 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff1600563d0 0x7ff160058890 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:15.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.692+0000 7ff186cac700 1 --2- 172.21.15.203:0/3600872324 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff18014e310 0x7ff180145580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:15.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.692+0000 7ff186cac700 1 --2- 172.21.15.203:0/3600872324 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff180064380 0x7ff1800b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:15.699 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.692+0000 7ff186cac700 1 --2- 172.21.15.203:0/3600872324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff180064760 0x7ff1800b8ed0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:15.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.692+0000 7ff186cac700 1 -- 172.21.15.203:0/3600872324 >> 172.21.15.203:0/3600872324 conn(0x7ff1800667b0 msgr2=0x7ff180146f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:15.700 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.692+0000 7ff186cac700 1 -- 172.21.15.203:0/3600872324 shutdown_connections 2021-01-06T22:21:15.701 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:15.692+0000 7ff186cac700 1 -- 172.21.15.203:0/3600872324 wait complete. 2021-01-06T22:21:15.701 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 85 2021-01-06T22:21:15.709 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:21:17.711 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:17.844 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb448b65700 1 Processor -- start 2021-01-06T22:21:17.845 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb448b65700 1 -- start start 2021-01-06T22:21:17.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb448b65700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb4440c44f0 0x7fb4440c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:17.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb448b65700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4440c0f50 con 0x7fb4440c48d0 2021-01-06T22:21:17.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb448b65700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb444066cf0 con 0x7fb444060fc0 2021-01-06T22:21:17.846 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb448b65700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb444060750 con 0x7fb4440c44f0 2021-01-06T22:21:17.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb4437fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb4440c44f0 0x7fb4440c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:17.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb4437fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb4440c44f0 0x7fb4440c1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53302/0 (socket says 172.21.15.203:53302) 2021-01-06T22:21:17.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb4437fe700 1 -- 172.21.15.203:0/2132422930 learned_addr learned my addr 172.21.15.203:0/2132422930 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:17.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb4437fe700 1 -- 172.21.15.203:0/2132422930 >> v1:172.21.15.203:6789/0 conn(0x7fb444060fc0 legacy=0x7fb444158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:17.847 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb4437fe700 1 -- 172.21.15.203:0/2132422930 >> v1:172.21.15.91:6789/0 conn(0x7fb4440c48d0 legacy=0x7fb444150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:17.848 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb4437fe700 1 -- 172.21.15.203:0/2132422930 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4440609f0 con 0x7fb4440c44f0 2021-01-06T22:21:17.848 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb4437fe700 1 --2- 172.21.15.203:0/2132422930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4440c44f0 0x7fb4440c1380 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7fb43400b9e0 tx=0x7fb434006ea0).ready entity=mon.1 client_cookie=df21a26c35d9ee3b server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:17.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb442ffd700 1 -- 172.21.15.203:0/2132422930 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4340098d0 con 0x7fb4440c44f0 2021-01-06T22:21:17.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb442ffd700 1 -- 172.21.15.203:0/2132422930 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb434009a30 con 0x7fb4440c44f0 2021-01-06T22:21:17.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb442ffd700 1 -- 172.21.15.203:0/2132422930 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb434009db0 con 0x7fb4440c44f0 2021-01-06T22:21:17.850 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb448b65700 1 -- 172.21.15.203:0/2132422930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4440c44f0 msgr2=0x7fb4440c1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:17.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.840+0000 7fb448b65700 1 --2- 172.21.15.203:0/2132422930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4440c44f0 0x7fb4440c1380 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7fb43400b9e0 tx=0x7fb434006ea0).stop 2021-01-06T22:21:17.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb448b65700 1 -- 172.21.15.203:0/2132422930 shutdown_connections 2021-01-06T22:21:17.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb448b65700 1 --2- 172.21.15.203:0/2132422930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4440c44f0 0x7fb4440c1380 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:17.851 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb448b65700 1 -- 172.21.15.203:0/2132422930 >> 172.21.15.203:0/2132422930 conn(0x7fb444056cd0 msgr2=0x7fb444061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:17.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb448b65700 1 -- 172.21.15.203:0/2132422930 shutdown_connections 2021-01-06T22:21:17.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb448b65700 1 -- 172.21.15.203:0/2132422930 wait complete. 2021-01-06T22:21:17.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb448b65700 1 Processor -- start 2021-01-06T22:21:17.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb448b65700 1 -- start start 2021-01-06T22:21:17.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb448b65700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb444060fc0 0x7fb4440a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:17.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb448b65700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4440c44f0 0x7fb4440c1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:17.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb448b65700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb4440c48d0 0x7fb4440a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:17.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb448b65700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440a3080 con 0x7fb4440c48d0 2021-01-06T22:21:17.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb448b65700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb444161180 con 0x7fb4440c44f0 2021-01-06T22:21:17.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb448b65700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb444161420 con 0x7fb444060fc0 2021-01-06T22:21:17.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb43bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4440c44f0 0x7fb4440c1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:17.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb43bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4440c44f0 0x7fb4440c1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53310/0 (socket says 172.21.15.203:53310) 2021-01-06T22:21:17.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb43bfff700 1 -- 172.21.15.203:0/3221767239 learned_addr learned my addr 172.21.15.203:0/3221767239 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:17.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb443fff700 1 --2- 172.21.15.203:0/3221767239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb4440c48d0 0x7fb4440a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:17.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.844+0000 7fb4437fe700 1 --2- 172.21.15.203:0/3221767239 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb444060fc0 0x7fb4440a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:17.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb43bfff700 1 -- 172.21.15.203:0/3221767239 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb444060fc0 msgr2=0x7fb4440a28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:17.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb43bfff700 1 --2- 172.21.15.203:0/3221767239 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb444060fc0 0x7fb4440a28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:17.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb43bfff700 1 -- 172.21.15.203:0/3221767239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb4440c48d0 msgr2=0x7fb4440a47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:17.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb43bfff700 1 --2- 172.21.15.203:0/3221767239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb4440c48d0 0x7fb4440a47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:17.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb43bfff700 1 -- 172.21.15.203:0/3221767239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4441616c0 con 0x7fb4440c44f0 2021-01-06T22:21:17.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb443fff700 1 --2- 172.21.15.203:0/3221767239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb4440c48d0 0x7fb4440a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:21:17.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb4437fe700 1 --2- 172.21.15.203:0/3221767239 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb444060fc0 0x7fb4440a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:21:17.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb43bfff700 1 --2- 172.21.15.203:0/3221767239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4440c44f0 0x7fb4440c1440 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7fb42c00c710 tx=0x7fb42c007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:17.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb43b7fe700 1 -- 172.21.15.203:0/3221767239 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c007020 con 0x7fb4440c44f0 2021-01-06T22:21:17.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb43b7fe700 1 -- 172.21.15.203:0/3221767239 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42c007180 con 0x7fb4440c44f0 2021-01-06T22:21:17.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb448b65700 1 -- 172.21.15.203:0/3221767239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb444161960 con 0x7fb4440c44f0 2021-01-06T22:21:17.859 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb43b7fe700 1 -- 172.21.15.203:0/3221767239 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c00e560 con 0x7fb4440c44f0 2021-01-06T22:21:17.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb448b65700 1 -- 172.21.15.203:0/3221767239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb444161e00 con 0x7fb4440c44f0 2021-01-06T22:21:17.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb448b65700 1 -- 172.21.15.203:0/3221767239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4440450a0 con 0x7fb4440c44f0 2021-01-06T22:21:17.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb43b7fe700 1 -- 172.21.15.203:0/3221767239 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fb42c010070 con 0x7fb4440c44f0 2021-01-06T22:21:17.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb43b7fe700 1 --2- 172.21.15.203:0/3221767239 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb4280563d0 0x7fb428058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:17.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb43b7fe700 1 -- 172.21.15.203:0/3221767239 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7fb42c06eeb0 con 0x7fb4440c44f0 2021-01-06T22:21:17.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb4437fe700 1 --2- 172.21.15.203:0/3221767239 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb4280563d0 0x7fb428058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:17.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.848+0000 7fb4437fe700 1 --2- 172.21.15.203:0/3221767239 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb4280563d0 0x7fb428058890 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fb43400b190 tx=0x7fb434016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:17.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:17.852+0000 7fb43b7fe700 1 -- 172.21.15.203:0/3221767239 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fb42c0443d0 con 0x7fb4440c44f0 2021-01-06T22:21:18.094 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:18.088+0000 7fb448b65700 1 -- 172.21.15.203:0/3221767239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4440aa090 con 0x7fb4440c44f0 2021-01-06T22:21:18.095 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:18.088+0000 7fb43b7fe700 1 -- 172.21.15.203:0/3221767239 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+3584 (secure 0 0 0) 0x7fb42c016020 con 0x7fb4440c44f0 2021-01-06T22:21:18.096 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:18.096 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":83}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:10.465189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4724,"mds_1":5632,"mds_2":5261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":204,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5261":{"gid":5261,"name":"f","rank":2,"incarnation":76,"state":"up:stopping","state_seq":19,"addr":"172.21.15.91:6837/2136238890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2136238890},{"type":"v1","addr":"172.21.15.91:6837","nonce":2136238890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:18.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:18.092+0000 7fb448b65700 1 -- 172.21.15.203:0/3221767239 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb4280563d0 msgr2=0x7fb428058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:18.097 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:18.092+0000 7fb448b65700 1 --2- 172.21.15.203:0/3221767239 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb4280563d0 0x7fb428058890 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fb43400b190 tx=0x7fb434016000).stop 2021-01-06T22:21:18.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:18.092+0000 7fb448b65700 1 -- 172.21.15.203:0/3221767239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4440c44f0 msgr2=0x7fb4440c1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:18.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:18.092+0000 7fb448b65700 1 --2- 172.21.15.203:0/3221767239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4440c44f0 0x7fb4440c1440 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7fb42c00c710 tx=0x7fb42c007c40).stop 2021-01-06T22:21:18.098 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:18.092+0000 7fb448b65700 1 -- 172.21.15.203:0/3221767239 shutdown_connections 2021-01-06T22:21:18.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:18.092+0000 7fb448b65700 1 --2- 172.21.15.203:0/3221767239 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb4280563d0 0x7fb428058890 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:18.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:18.092+0000 7fb448b65700 1 --2- 172.21.15.203:0/3221767239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb4440c48d0 0x7fb4440a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:18.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:18.092+0000 7fb448b65700 1 --2- 172.21.15.203:0/3221767239 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb444060fc0 0x7fb4440a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:18.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:18.092+0000 7fb448b65700 1 --2- 172.21.15.203:0/3221767239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4440c44f0 0x7fb4440c1440 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:18.099 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:18.092+0000 7fb448b65700 1 -- 172.21.15.203:0/3221767239 >> 172.21.15.203:0/3221767239 conn(0x7fb444056cd0 msgr2=0x7fb4440bcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:18.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:18.092+0000 7fb448b65700 1 -- 172.21.15.203:0/3221767239 shutdown_connections 2021-01-06T22:21:18.100 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:18.092+0000 7fb448b65700 1 -- 172.21.15.203:0/3221767239 wait complete. 2021-01-06T22:21:18.100 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 85 2021-01-06T22:21:18.111 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:21:19.742 INFO:tasks.ceph.mds.f.smithi091.stdout:starting mds.f at 2021-01-06T22:21:20.113 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:20.237 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.232+0000 7fd9c6dc7700 1 Processor -- start 2021-01-06T22:21:20.238 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.232+0000 7fd9c6dc7700 1 -- start start 2021-01-06T22:21:20.240 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.232+0000 7fd9c6dc7700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd9c00c0fa0 0x7fd9c00641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:20.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.232+0000 7fd9c6dc7700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c00bccb0 con 0x7fd9c00ba9e0 2021-01-06T22:21:20.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.232+0000 7fd9c6dc7700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c00bd180 con 0x7fd9c014df70 2021-01-06T22:21:20.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.232+0000 7fd9c6dc7700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0067ac0 con 0x7fd9c00c0fa0 2021-01-06T22:21:20.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.232+0000 7fd9c5dc5700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fd9c014df70 0x7fd9c014e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51910/0 (socket says 172.21.15.203:51910) 2021-01-06T22:21:20.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.232+0000 7fd9c5dc5700 1 -- 172.21.15.203:0/1695743310 learned_addr learned my addr 172.21.15.203:0/1695743310 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:21:20.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.232+0000 7fd9c55c4700 1 --2- 172.21.15.203:0/1695743310 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd9c00c0fa0 0x7fd9c00641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:20.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.232+0000 7fd9c4dc3700 1 -- 172.21.15.203:0/1695743310 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416593868 0 0) 0x7fd9c00bd180 con 0x7fd9c014df70 2021-01-06T22:21:20.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 -- 172.21.15.203:0/1695743310 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9a8003230 con 0x7fd9c014df70 2021-01-06T22:21:20.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 -- 172.21.15.203:0/1695743310 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4045913993 0 0) 0x7fd9a8003230 con 0x7fd9c014df70 2021-01-06T22:21:20.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 -- 172.21.15.203:0/1695743310 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd9c00c0fa0 msgr2=0x7fd9c00641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:20.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 --2- 172.21.15.203:0/1695743310 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd9c00c0fa0 0x7fd9c00641e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 -- 172.21.15.203:0/1695743310 >> v1:172.21.15.91:6789/0 conn(0x7fd9c00ba9e0 legacy=0x7fd9c014ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:20.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 -- 172.21.15.203:0/1695743310 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c00bcee0 con 0x7fd9c014df70 2021-01-06T22:21:20.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 -- 172.21.15.203:0/1695743310 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7fd9bc0014e0 con 0x7fd9c014df70 2021-01-06T22:21:20.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 -- 172.21.15.203:0/1695743310 >> v1:172.21.15.203:6789/0 conn(0x7fd9c014df70 legacy=0x7fd9c014e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:20.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 --2- 172.21.15.203:0/1695743310 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd9c00c0fa0 0x7fd9a8003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:20.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 --2- 172.21.15.203:0/1695743310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a8003c10 0x7fd9a80044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:20.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 --2- 172.21.15.203:0/1695743310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9a80049c0 0x7fd9a8004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:20.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 -- 172.21.15.203:0/1695743310 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0067ac0 con 0x7fd9a80049c0 2021-01-06T22:21:20.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 -- 172.21.15.203:0/1695743310 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a8004130 con 0x7fd9a8003c10 2021-01-06T22:21:20.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 -- 172.21.15.203:0/1695743310 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80042f0 con 0x7fd9c00c0fa0 2021-01-06T22:21:20.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c55c4700 1 --2- 172.21.15.203:0/1695743310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a8003c10 0x7fd9a80044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:20.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c65c6700 1 --2- 172.21.15.203:0/1695743310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9a80049c0 0x7fd9a8004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:20.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c55c4700 1 -- 172.21.15.203:0/1695743310 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd9c00c0fa0 msgr2=0x7fd9a8003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:20.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c55c4700 1 --2- 172.21.15.203:0/1695743310 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd9c00c0fa0 0x7fd9a8003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:21:20.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c55c4700 1 -- 172.21.15.203:0/1695743310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9a80049c0 msgr2=0x7fd9a8004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:20.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c55c4700 1 --2- 172.21.15.203:0/1695743310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9a80049c0 0x7fd9a8004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c55c4700 1 -- 172.21.15.203:0/1695743310 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a8003230 con 0x7fd9a8003c10 2021-01-06T22:21:20.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c65c6700 1 --2- 172.21.15.203:0/1695743310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9a80049c0 0x7fd9a8004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:21:20.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c55c4700 1 --2- 172.21.15.203:0/1695743310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a8003c10 0x7fd9a80044d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7fd9b000bb60 tx=0x7fd9b0008040).ready entity=mon.1 client_cookie=9c11db0ac0425dd0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:20.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 -- 172.21.15.203:0/1695743310 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b000b2b0 con 0x7fd9a8003c10 2021-01-06T22:21:20.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 -- 172.21.15.203:0/1695743310 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b000ea90 con 0x7fd9a8003c10 2021-01-06T22:21:20.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c4dc3700 1 -- 172.21.15.203:0/1695743310 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b000ee30 con 0x7fd9a8003c10 2021-01-06T22:21:20.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c6dc7700 1 -- 172.21.15.203:0/1695743310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a8003c10 msgr2=0x7fd9a80044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:20.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c6dc7700 1 --2- 172.21.15.203:0/1695743310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a8003c10 0x7fd9a80044d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7fd9b000bb60 tx=0x7fd9b0008040).stop 2021-01-06T22:21:20.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c65c6700 1 -- 172.21.15.203:0/1695743310 reap_dead start 2021-01-06T22:21:20.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c6dc7700 1 -- 172.21.15.203:0/1695743310 shutdown_connections 2021-01-06T22:21:20.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c6dc7700 1 -- 172.21.15.203:0/1695743310 >> 172.21.15.203:0/1695743310 conn(0x7fd9c00667b0 msgr2=0x7fd9c00c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:20.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c6dc7700 1 -- 172.21.15.203:0/1695743310 shutdown_connections 2021-01-06T22:21:20.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.236+0000 7fd9c6dc7700 1 -- 172.21.15.203:0/1695743310 wait complete. 2021-01-06T22:21:20.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c6dc7700 1 Processor -- start 2021-01-06T22:21:20.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c6dc7700 1 -- start start 2021-01-06T22:21:20.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c6dc7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a8003c10 0x7fd9c0160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:20.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c6dc7700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9c00bfb90 0x7fd9c00b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:20.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c6dc7700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd9c00bfff0 0x7fd9c00b80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:20.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c6dc7700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0150850 con 0x7fd9c00bfb90 2021-01-06T22:21:20.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c6dc7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c014f690 con 0x7fd9a8003c10 2021-01-06T22:21:20.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c6dc7700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c014f930 con 0x7fd9c00bfff0 2021-01-06T22:21:20.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c5dc5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a8003c10 0x7fd9c0160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:20.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c5dc5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a8003c10 0x7fd9c0160830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53328/0 (socket says 172.21.15.203:53328) 2021-01-06T22:21:20.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c5dc5700 1 -- 172.21.15.203:0/1650186688 learned_addr learned my addr 172.21.15.203:0/1650186688 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:20.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c65c6700 1 --2- 172.21.15.203:0/1650186688 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd9c00bfff0 0x7fd9c00b80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:20.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c55c4700 1 --2- 172.21.15.203:0/1650186688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9c00bfb90 0x7fd9c00b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:20.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c5dc5700 1 -- 172.21.15.203:0/1650186688 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd9c00bfff0 msgr2=0x7fd9c00b80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:20.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c5dc5700 1 --2- 172.21.15.203:0/1650186688 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd9c00bfff0 0x7fd9c00b80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c5dc5700 1 -- 172.21.15.203:0/1650186688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9c00bfb90 msgr2=0x7fd9c00b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:20.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c5dc5700 1 --2- 172.21.15.203:0/1650186688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9c00bfb90 0x7fd9c00b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c5dc5700 1 -- 172.21.15.203:0/1650186688 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c014fbd0 con 0x7fd9a8003c10 2021-01-06T22:21:20.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c5dc5700 1 --2- 172.21.15.203:0/1650186688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a8003c10 0x7fd9c0160830 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fd9bc00cac0 tx=0x7fd9c00bd070).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:20.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9adffb700 1 -- 172.21.15.203:0/1650186688 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc007170 con 0x7fd9a8003c10 2021-01-06T22:21:20.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9adffb700 1 -- 172.21.15.203:0/1650186688 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9bc0072d0 con 0x7fd9a8003c10 2021-01-06T22:21:20.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c6dc7700 1 -- 172.21.15.203:0/1650186688 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c014fe70 con 0x7fd9a8003c10 2021-01-06T22:21:20.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9adffb700 1 -- 172.21.15.203:0/1650186688 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc00a980 con 0x7fd9a8003c10 2021-01-06T22:21:20.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.240+0000 7fd9c6dc7700 1 -- 172.21.15.203:0/1650186688 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c01503b0 con 0x7fd9a8003c10 2021-01-06T22:21:20.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.244+0000 7fd9c6dc7700 1 -- 172.21.15.203:0/1650186688 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c0150570 con 0x7fd9a8003c10 2021-01-06T22:21:20.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.244+0000 7fd9adffb700 1 -- 172.21.15.203:0/1650186688 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fd9bc010070 con 0x7fd9a8003c10 2021-01-06T22:21:20.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.244+0000 7fd9adffb700 1 --2- 172.21.15.203:0/1650186688 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9a00563d0 0x7fd9a0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:20.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.244+0000 7fd9adffb700 1 -- 172.21.15.203:0/1650186688 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7fd9bc06f490 con 0x7fd9a8003c10 2021-01-06T22:21:20.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.244+0000 7fd9c55c4700 1 --2- 172.21.15.203:0/1650186688 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9a00563d0 0x7fd9a0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:20.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.244+0000 7fd9c55c4700 1 --2- 172.21.15.203:0/1650186688 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9a00563d0 0x7fd9a0058890 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fd9c00bc590 tx=0x7fd9b001a000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:20.259 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.248+0000 7fd9adffb700 1 -- 172.21.15.203:0/1650186688 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fd9bc0449b0 con 0x7fd9a8003c10 2021-01-06T22:21:20.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.476+0000 7fd9c6dc7700 1 -- 172.21.15.203:0/1650186688 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9c00450a0 con 0x7fd9a8003c10 2021-01-06T22:21:20.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.476+0000 7fd9adffb700 1 -- 172.21.15.203:0/1650186688 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+3215 (secure 0 0 0) 0x7fd9bc047fd0 con 0x7fd9a8003c10 2021-01-06T22:21:20.482 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:20.483 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":83}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:19.698147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4724,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":204,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:20.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.480+0000 7fd9c6dc7700 1 -- 172.21.15.203:0/1650186688 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9a00563d0 msgr2=0x7fd9a0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:20.485 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.480+0000 7fd9c6dc7700 1 --2- 172.21.15.203:0/1650186688 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9a00563d0 0x7fd9a0058890 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fd9c00bc590 tx=0x7fd9b001a000).stop 2021-01-06T22:21:20.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.480+0000 7fd9c6dc7700 1 -- 172.21.15.203:0/1650186688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a8003c10 msgr2=0x7fd9c0160830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:20.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.480+0000 7fd9c6dc7700 1 --2- 172.21.15.203:0/1650186688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a8003c10 0x7fd9c0160830 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fd9bc00cac0 tx=0x7fd9c00bd070).stop 2021-01-06T22:21:20.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.480+0000 7fd9c6dc7700 1 -- 172.21.15.203:0/1650186688 shutdown_connections 2021-01-06T22:21:20.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.480+0000 7fd9c6dc7700 1 --2- 172.21.15.203:0/1650186688 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd9a00563d0 0x7fd9a0058890 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.480+0000 7fd9c6dc7700 1 --2- 172.21.15.203:0/1650186688 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd9c00bfff0 0x7fd9c00b80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.480+0000 7fd9c6dc7700 1 --2- 172.21.15.203:0/1650186688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a8003c10 0x7fd9c0160830 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.480+0000 7fd9c6dc7700 1 --2- 172.21.15.203:0/1650186688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9c00bfb90 0x7fd9c00b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.480+0000 7fd9c6dc7700 1 -- 172.21.15.203:0/1650186688 >> 172.21.15.203:0/1650186688 conn(0x7fd9c00667b0 msgr2=0x7fd9c00602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:20.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.480+0000 7fd9c6dc7700 1 -- 172.21.15.203:0/1650186688 shutdown_connections 2021-01-06T22:21:20.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.480+0000 7fd9c6dc7700 1 -- 172.21.15.203:0/1650186688 wait complete. 2021-01-06T22:21:20.488 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 86 2021-01-06T22:21:20.502 INFO:tasks.mds_thrash.fs.[cephfs]:mds cluster has 2 alive and active, now stable! 2021-01-06T22:21:20.502 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=0) 2021-01-06T22:21:20.503 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-01-06T22:21:20.503 INFO:teuthology.orchestra.run:waiting for 300 2021-01-06T22:21:20.521 INFO:tasks.ceph.mds.b:Stopped 2021-01-06T22:21:20.521 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:20.629 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f0732d40700 1 Processor -- start 2021-01-06T22:21:20.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f0732d40700 1 -- start start 2021-01-06T22:21:20.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f0732d40700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f072c060d20 0x7f072c0bf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:20.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f0732d40700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f072c0606b0 0x7f072c14e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:20.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f0732d40700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072c062ec0 con 0x7f072c0bf420 2021-01-06T22:21:20.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f0732d40700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f072c14ebb0 con 0x7f072c0606b0 2021-01-06T22:21:20.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f0732d40700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0611f0 con 0x7f072c060d20 2021-01-06T22:21:20.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f073253f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f072c0606b0 0x7f072c14e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:20.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f073253f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f072c0606b0 0x7f072c14e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53340/0 (socket says 172.21.15.203:53340) 2021-01-06T22:21:20.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f073253f700 1 -- 172.21.15.203:0/1959834906 learned_addr learned my addr 172.21.15.203:0/1959834906 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:20.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f073253f700 1 -- 172.21.15.203:0/1959834906 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f072c060d20 msgr2=0x7f072c0bf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:20.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f073153d700 1 --2- 172.21.15.203:0/1959834906 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f072c060d20 0x7f072c0bf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:20.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f073253f700 1 --2- 172.21.15.203:0/1959834906 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f072c060d20 0x7f072c0bf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f073253f700 1 -- 172.21.15.203:0/1959834906 >> v1:172.21.15.91:6789/0 conn(0x7f072c0bf420 legacy=0x7f072c0c21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:20.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f073253f700 1 -- 172.21.15.203:0/1959834906 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c0626e0 con 0x7f072c0606b0 2021-01-06T22:21:20.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f073153d700 1 --2- 172.21.15.203:0/1959834906 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f072c060d20 0x7f072c0bf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-01-06T22:21:20.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.624+0000 7f073253f700 1 --2- 172.21.15.203:0/1959834906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f072c0606b0 0x7f072c14e6c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f0724009d50 tx=0x7f0724009290).ready entity=mon.1 client_cookie=5d53b8d8d1cbe00e server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:20.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.628+0000 7f0730d3c700 1 -- 172.21.15.203:0/1959834906 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072400bcd0 con 0x7f072c0606b0 2021-01-06T22:21:20.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.628+0000 7f0730d3c700 1 -- 172.21.15.203:0/1959834906 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072400be30 con 0x7f072c0606b0 2021-01-06T22:21:20.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.628+0000 7f0730d3c700 1 -- 172.21.15.203:0/1959834906 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0724011610 con 0x7f072c0606b0 2021-01-06T22:21:20.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.628+0000 7f0732d40700 1 -- 172.21.15.203:0/1959834906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f072c0606b0 msgr2=0x7f072c14e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:20.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.628+0000 7f0732d40700 1 --2- 172.21.15.203:0/1959834906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f072c0606b0 0x7f072c14e6c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f0724009d50 tx=0x7f0724009290).stop 2021-01-06T22:21:20.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.628+0000 7f0732d40700 1 -- 172.21.15.203:0/1959834906 shutdown_connections 2021-01-06T22:21:20.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.628+0000 7f0732d40700 1 --2- 172.21.15.203:0/1959834906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f072c0606b0 0x7f072c14e6c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.628+0000 7f0732d40700 1 --2- 172.21.15.203:0/1959834906 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f072c060d20 0x7f072c0bf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.628+0000 7f0732d40700 1 -- 172.21.15.203:0/1959834906 >> 172.21.15.203:0/1959834906 conn(0x7f072c0680f0 msgr2=0x7f072c056cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:20.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.628+0000 7f0732d40700 1 -- 172.21.15.203:0/1959834906 shutdown_connections 2021-01-06T22:21:20.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.628+0000 7f0732d40700 1 -- 172.21.15.203:0/1959834906 wait complete. 2021-01-06T22:21:20.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.628+0000 7f0732d40700 1 Processor -- start 2021-01-06T22:21:20.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.628+0000 7f0732d40700 1 -- start start 2021-01-06T22:21:20.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0732d40700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f072c0606b0 0x7f072c15dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:20.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0732d40700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f072c060d20 0x7f072c0a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:20.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0732d40700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f072c0bf420 0x7f072c160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:20.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0732d40700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c14a260 con 0x7f072c060d20 2021-01-06T22:21:20.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0732d40700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1607f0 con 0x7f072c0606b0 2021-01-06T22:21:20.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0732d40700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0b6150 con 0x7f072c0bf420 2021-01-06T22:21:20.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0731d3e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f072c0606b0 0x7f072c15dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:20.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0731d3e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f072c0606b0 0x7f072c15dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53344/0 (socket says 172.21.15.203:53344) 2021-01-06T22:21:20.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0731d3e700 1 -- 172.21.15.203:0/2235851396 learned_addr learned my addr 172.21.15.203:0/2235851396 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:20.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f073153d700 1 --2- 172.21.15.203:0/2235851396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f072c060d20 0x7f072c0a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:20.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f073253f700 1 --2- 172.21.15.203:0/2235851396 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f072c0bf420 0x7f072c160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:20.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0731d3e700 1 -- 172.21.15.203:0/2235851396 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f072c0bf420 msgr2=0x7f072c160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:20.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0731d3e700 1 --2- 172.21.15.203:0/2235851396 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f072c0bf420 0x7f072c160ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0731d3e700 1 -- 172.21.15.203:0/2235851396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f072c060d20 msgr2=0x7f072c0a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:20.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0731d3e700 1 --2- 172.21.15.203:0/2235851396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f072c060d20 0x7f072c0a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0731d3e700 1 -- 172.21.15.203:0/2235851396 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c15d220 con 0x7f072c0606b0 2021-01-06T22:21:20.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0731d3e700 1 --2- 172.21.15.203:0/2235851396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f072c0606b0 0x7f072c15dc00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f07280096e0 tx=0x7f0728006d60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:20.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0719ffb700 1 -- 172.21.15.203:0/2235851396 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072800d560 con 0x7f072c0606b0 2021-01-06T22:21:20.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0719ffb700 1 -- 172.21.15.203:0/2235851396 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0728000e90 con 0x7f072c0606b0 2021-01-06T22:21:20.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0732d40700 1 -- 172.21.15.203:0/2235851396 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072c0b7250 con 0x7f072c0606b0 2021-01-06T22:21:20.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0719ffb700 1 -- 172.21.15.203:0/2235851396 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072800d910 con 0x7f072c0606b0 2021-01-06T22:21:20.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0732d40700 1 -- 172.21.15.203:0/2235851396 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072c0b7790 con 0x7f072c0606b0 2021-01-06T22:21:20.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0732d40700 1 -- 172.21.15.203:0/2235851396 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f072c0450a0 con 0x7f072c0606b0 2021-01-06T22:21:20.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0719ffb700 1 -- 172.21.15.203:0/2235851396 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f072800f030 con 0x7f072c0606b0 2021-01-06T22:21:20.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0719ffb700 1 --2- 172.21.15.203:0/2235851396 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f070c0563d0 0x7f070c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:20.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f0719ffb700 1 -- 172.21.15.203:0/2235851396 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7f072806f600 con 0x7f072c0606b0 2021-01-06T22:21:20.649 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.632+0000 7f073153d700 1 --2- 172.21.15.203:0/2235851396 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f070c0563d0 0x7f070c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:20.649 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.636+0000 7f073153d700 1 --2- 172.21.15.203:0/2235851396 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f070c0563d0 0x7f070c058890 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f071c006fd0 tx=0x7f071c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:20.649 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.636+0000 7f0719ffb700 1 -- 172.21.15.203:0/2235851396 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f0728044b20 con 0x7f072c0606b0 2021-01-06T22:21:20.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.864+0000 7f0732d40700 1 -- 172.21.15.203:0/2235851396 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f072c15e0f0 con 0x7f072c0606b0 2021-01-06T22:21:20.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.864+0000 7f0719ffb700 1 -- 172.21.15.203:0/2235851396 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+3215 (secure 0 0 0) 0x7f0728048140 con 0x7f072c0606b0 2021-01-06T22:21:20.869 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:20.870 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":83}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:19.698147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4724,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":204,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:20.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.868+0000 7f0732d40700 1 -- 172.21.15.203:0/2235851396 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f070c0563d0 msgr2=0x7f070c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:20.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.868+0000 7f0732d40700 1 --2- 172.21.15.203:0/2235851396 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f070c0563d0 0x7f070c058890 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f071c006fd0 tx=0x7f071c008040).stop 2021-01-06T22:21:20.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.868+0000 7f0732d40700 1 -- 172.21.15.203:0/2235851396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f072c0606b0 msgr2=0x7f072c15dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:20.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.868+0000 7f0732d40700 1 --2- 172.21.15.203:0/2235851396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f072c0606b0 0x7f072c15dc00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f07280096e0 tx=0x7f0728006d60).stop 2021-01-06T22:21:20.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.868+0000 7f0732d40700 1 -- 172.21.15.203:0/2235851396 shutdown_connections 2021-01-06T22:21:20.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.868+0000 7f0732d40700 1 --2- 172.21.15.203:0/2235851396 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f070c0563d0 0x7f070c058890 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.868+0000 7f0732d40700 1 --2- 172.21.15.203:0/2235851396 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f072c0bf420 0x7f072c160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.868+0000 7f0732d40700 1 --2- 172.21.15.203:0/2235851396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f072c0606b0 0x7f072c15dc00 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.868+0000 7f0732d40700 1 --2- 172.21.15.203:0/2235851396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f072c060d20 0x7f072c0a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:20.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.868+0000 7f0732d40700 1 -- 172.21.15.203:0/2235851396 >> 172.21.15.203:0/2235851396 conn(0x7f072c0680f0 msgr2=0x7f072c0be1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:20.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.868+0000 7f0732d40700 1 -- 172.21.15.203:0/2235851396 shutdown_connections 2021-01-06T22:21:20.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:20.868+0000 7f0732d40700 1 -- 172.21.15.203:0/2235851396 wait complete. 2021-01-06T22:21:20.875 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 86 2021-01-06T22:21:20.887 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2021-01-06T22:21:21.572 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:21:21.575 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:21:22.888 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:23.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.016+0000 7fe23c8e1700 1 Processor -- start 2021-01-06T22:21:23.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.016+0000 7fe23c8e1700 1 -- start start 2021-01-06T22:21:23.026 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe23c8e1700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe238064380 0x7fe2380c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:23.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe23c8e1700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe23814e310 0x7fe2381471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:23.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe23c8e1700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2380bb2d0 con 0x7fe238064760 2021-01-06T22:21:23.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe23c8e1700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23814a560 con 0x7fe238064380 2021-01-06T22:21:23.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe23c8e1700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23814a800 con 0x7fe23814e310 2021-01-06T22:21:23.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe237fff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fe238064760 0x7fe2380bb730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51968/0 (socket says 172.21.15.203:51968) 2021-01-06T22:21:23.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe237fff700 1 -- 172.21.15.203:0/1860069149 learned_addr learned my addr 172.21.15.203:0/1860069149 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:21:23.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe2377fe700 1 --2- 172.21.15.203:0/1860069149 >> v2:172.21.15.91:3300/0 conn(0x7fe238064380 0x7fe2380c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:23.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe236ffd700 1 --2- 172.21.15.203:0/1860069149 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe23814e310 0x7fe2381471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:23.029 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe2367fc700 1 -- 172.21.15.203:0/1860069149 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205040039 0 0) 0x7fe2380bb2d0 con 0x7fe238064760 2021-01-06T22:21:23.029 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe2367fc700 1 -- 172.21.15.203:0/1860069149 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe224003230 con 0x7fe238064760 2021-01-06T22:21:23.029 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe2377fe700 1 -- 172.21.15.203:0/1860069149 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe23814e310 msgr2=0x7fe2381471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:23.029 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe2377fe700 1 --2- 172.21.15.203:0/1860069149 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe23814e310 0x7fe2381471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:23.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe2377fe700 1 -- 172.21.15.203:0/1860069149 >> v1:172.21.15.203:6789/0 conn(0x7fe238064760 legacy=0x7fe2380bb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:23.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe2377fe700 1 -- 172.21.15.203:0/1860069149 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23814aaa0 con 0x7fe238064380 2021-01-06T22:21:23.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe2377fe700 1 --2- 172.21.15.203:0/1860069149 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe238064380 0x7fe2380c0fa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fe22800b980 tx=0x7fe228006de0).ready entity=mon.0 client_cookie=3b9eb656732fb8f5 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:23.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe2367fc700 1 -- 172.21.15.203:0/1860069149 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2280096d0 con 0x7fe238064380 2021-01-06T22:21:23.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe2367fc700 1 -- 172.21.15.203:0/1860069149 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe228009830 con 0x7fe238064380 2021-01-06T22:21:23.031 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe2367fc700 1 -- 172.21.15.203:0/1860069149 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228009bb0 con 0x7fe238064380 2021-01-06T22:21:23.031 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe23c8e1700 1 -- 172.21.15.203:0/1860069149 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe238064380 msgr2=0x7fe2380c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:23.031 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe23c8e1700 1 --2- 172.21.15.203:0/1860069149 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe238064380 0x7fe2380c0fa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fe22800b980 tx=0x7fe228006de0).stop 2021-01-06T22:21:23.031 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe23c8e1700 1 -- 172.21.15.203:0/1860069149 shutdown_connections 2021-01-06T22:21:23.031 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe23c8e1700 1 --2- 172.21.15.203:0/1860069149 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe238064380 0x7fe2380c0fa0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:23.032 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe23c8e1700 1 --2- 172.21.15.203:0/1860069149 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe23814e310 0x7fe2381471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:23.032 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe23c8e1700 1 -- 172.21.15.203:0/1860069149 >> 172.21.15.203:0/1860069149 conn(0x7fe2380667b0 msgr2=0x7fe2380c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:23.032 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe23c8e1700 1 -- 172.21.15.203:0/1860069149 shutdown_connections 2021-01-06T22:21:23.032 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.020+0000 7fe23c8e1700 1 -- 172.21.15.203:0/1860069149 wait complete. 2021-01-06T22:21:23.034 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe23c8e1700 1 Processor -- start 2021-01-06T22:21:23.035 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe23c8e1700 1 -- start start 2021-01-06T22:21:23.035 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe23c8e1700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe238064380 0x7fe238146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:23.035 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe23c8e1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe238064760 0x7fe238152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:23.035 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe23c8e1700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe23814e310 0x7fe2380bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:23.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe23c8e1700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe238147800 con 0x7fe23814e310 2021-01-06T22:21:23.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe23c8e1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe238147560 con 0x7fe238064760 2021-01-06T22:21:23.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe23c8e1700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe238160b70 con 0x7fe238064380 2021-01-06T22:21:23.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe236ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe238064760 0x7fe238152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:23.036 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe236ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe238064760 0x7fe238152bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53380/0 (socket says 172.21.15.203:53380) 2021-01-06T22:21:23.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe236ffd700 1 -- 172.21.15.203:0/694770496 learned_addr learned my addr 172.21.15.203:0/694770496 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:23.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe237fff700 1 --2- 172.21.15.203:0/694770496 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe23814e310 0x7fe2380bfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:23.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe2377fe700 1 --2- 172.21.15.203:0/694770496 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe238064380 0x7fe238146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:23.037 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe236ffd700 1 -- 172.21.15.203:0/694770496 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe238064380 msgr2=0x7fe238146120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:23.038 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe236ffd700 1 --2- 172.21.15.203:0/694770496 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe238064380 0x7fe238146120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:23.038 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe236ffd700 1 -- 172.21.15.203:0/694770496 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe23814e310 msgr2=0x7fe2380bfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:23.038 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe236ffd700 1 --2- 172.21.15.203:0/694770496 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe23814e310 0x7fe2380bfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:23.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe236ffd700 1 -- 172.21.15.203:0/694770496 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe238160e10 con 0x7fe238064760 2021-01-06T22:21:23.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe237fff700 1 --2- 172.21.15.203:0/694770496 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe23814e310 0x7fe2380bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:21:23.040 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe236ffd700 1 --2- 172.21.15.203:0/694770496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe238064760 0x7fe238152bb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7fe22c00eba0 tx=0x7fe22c006de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:23.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe21f7fe700 1 -- 172.21.15.203:0/694770496 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c00b960 con 0x7fe238064760 2021-01-06T22:21:23.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe21f7fe700 1 -- 172.21.15.203:0/694770496 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c00bac0 con 0x7fe238064760 2021-01-06T22:21:23.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe23c8e1700 1 -- 172.21.15.203:0/694770496 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2381610b0 con 0x7fe238064760 2021-01-06T22:21:23.041 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe21f7fe700 1 -- 172.21.15.203:0/694770496 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c00be40 con 0x7fe238064760 2021-01-06T22:21:23.042 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.024+0000 7fe23c8e1700 1 -- 172.21.15.203:0/694770496 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2381617f0 con 0x7fe238064760 2021-01-06T22:21:23.042 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.028+0000 7fe23c8e1700 1 -- 172.21.15.203:0/694770496 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2380450a0 con 0x7fe238064760 2021-01-06T22:21:23.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.028+0000 7fe21f7fe700 1 -- 172.21.15.203:0/694770496 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fe22c012600 con 0x7fe238064760 2021-01-06T22:21:23.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.028+0000 7fe21f7fe700 1 --2- 172.21.15.203:0/694770496 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe2180563d0 0x7fe218058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:23.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.028+0000 7fe2377fe700 1 --2- 172.21.15.203:0/694770496 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe2180563d0 0x7fe218058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:23.043 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.028+0000 7fe21f7fe700 1 -- 172.21.15.203:0/694770496 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7fe22c06fd70 con 0x7fe238064760 2021-01-06T22:21:23.044 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.028+0000 7fe2377fe700 1 --2- 172.21.15.203:0/694770496 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe2180563d0 0x7fe218058890 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fe22800b130 tx=0x7fe228016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:23.044 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.032+0000 7fe21f7fe700 1 -- 172.21.15.203:0/694770496 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fe22c045290 con 0x7fe238064760 2021-01-06T22:21:23.241 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.236+0000 7fe23c8e1700 1 -- 172.21.15.203:0/694770496 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe23805ca60 con 0x7fe238064760 2021-01-06T22:21:23.242 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.236+0000 7fe21f7fe700 1 -- 172.21.15.203:0/694770496 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+3215 (secure 0 0 0) 0x7fe22c0488b0 con 0x7fe238064760 2021-01-06T22:21:23.242 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:23.243 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":83}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:19.698147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4724,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":204,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:23.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.240+0000 7fe23c8e1700 1 -- 172.21.15.203:0/694770496 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe2180563d0 msgr2=0x7fe218058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:23.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.240+0000 7fe23c8e1700 1 --2- 172.21.15.203:0/694770496 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe2180563d0 0x7fe218058890 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fe22800b130 tx=0x7fe228016000).stop 2021-01-06T22:21:23.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.240+0000 7fe23c8e1700 1 -- 172.21.15.203:0/694770496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe238064760 msgr2=0x7fe238152bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:23.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.240+0000 7fe23c8e1700 1 --2- 172.21.15.203:0/694770496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe238064760 0x7fe238152bb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7fe22c00eba0 tx=0x7fe22c006de0).stop 2021-01-06T22:21:23.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.240+0000 7fe23c8e1700 1 -- 172.21.15.203:0/694770496 shutdown_connections 2021-01-06T22:21:23.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.240+0000 7fe23c8e1700 1 --2- 172.21.15.203:0/694770496 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe2180563d0 0x7fe218058890 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:23.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.240+0000 7fe23c8e1700 1 --2- 172.21.15.203:0/694770496 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe23814e310 0x7fe2380bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:23.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.240+0000 7fe23c8e1700 1 --2- 172.21.15.203:0/694770496 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe238064380 0x7fe238146120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:23.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.240+0000 7fe23c8e1700 1 --2- 172.21.15.203:0/694770496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe238064760 0x7fe238152bb0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:23.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.240+0000 7fe23c8e1700 1 -- 172.21.15.203:0/694770496 >> 172.21.15.203:0/694770496 conn(0x7fe2380667b0 msgr2=0x7fe2380604e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:23.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.240+0000 7fe23c8e1700 1 -- 172.21.15.203:0/694770496 shutdown_connections 2021-01-06T22:21:23.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:23.240+0000 7fe23c8e1700 1 -- 172.21.15.203:0/694770496 wait complete. 2021-01-06T22:21:23.247 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 86 2021-01-06T22:21:23.257 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2021-01-06T22:21:25.258 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:25.390 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.384+0000 7f3d6a5cb700 1 Processor -- start 2021-01-06T22:21:25.390 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.384+0000 7f3d6a5cb700 1 -- start start 2021-01-06T22:21:25.392 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d6a5cb700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3d64067660 0x7f3d6414dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:25.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d6a5cb700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d64067a40 0x7f3d64068330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:25.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d6a5cb700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d640b9080 con 0x7f3d64061830 2021-01-06T22:21:25.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d6a5cb700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d641481f0 con 0x7f3d64067660 2021-01-06T22:21:25.393 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d6a5cb700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d640b8300 con 0x7f3d64067a40 2021-01-06T22:21:25.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d637fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d64067a40 0x7f3d64068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:25.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d637fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d64067a40 0x7f3d64068330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53394/0 (socket says 172.21.15.203:53394) 2021-01-06T22:21:25.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d637fe700 1 -- 172.21.15.203:0/3423787351 learned_addr learned my addr 172.21.15.203:0/3423787351 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:25.394 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d63fff700 1 --2- 172.21.15.203:0/3423787351 >> v2:172.21.15.91:3300/0 conn(0x7f3d64067660 0x7f3d6414dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:25.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d637fe700 1 -- 172.21.15.203:0/3423787351 >> v2:172.21.15.91:3300/0 conn(0x7f3d64067660 msgr2=0x7f3d6414dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:25.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d637fe700 1 --2- 172.21.15.203:0/3423787351 >> v2:172.21.15.91:3300/0 conn(0x7f3d64067660 0x7f3d6414dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:25.396 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d637fe700 1 -- 172.21.15.203:0/3423787351 >> v1:172.21.15.91:6789/0 conn(0x7f3d64061830 legacy=0x7f3d640ba4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:25.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d637fe700 1 -- 172.21.15.203:0/3423787351 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6414e170 con 0x7f3d64067a40 2021-01-06T22:21:25.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d637fe700 1 --2- 172.21.15.203:0/3423787351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d64067a40 0x7f3d64068330 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f3d640c0b50 tx=0x7f3d4c006ea0).ready entity=mon.1 client_cookie=bbc739ed3a416a7 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:25.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d62ffd700 1 -- 172.21.15.203:0/3423787351 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0098a0 con 0x7f3d64067a40 2021-01-06T22:21:25.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d62ffd700 1 -- 172.21.15.203:0/3423787351 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c009a00 con 0x7f3d64067a40 2021-01-06T22:21:25.397 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.388+0000 7f3d62ffd700 1 -- 172.21.15.203:0/3423787351 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c009d80 con 0x7f3d64067a40 2021-01-06T22:21:25.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.392+0000 7f3d6a5cb700 1 -- 172.21.15.203:0/3423787351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d64067a40 msgr2=0x7f3d64068330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:25.398 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.392+0000 7f3d6a5cb700 1 --2- 172.21.15.203:0/3423787351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d64067a40 0x7f3d64068330 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f3d640c0b50 tx=0x7f3d4c006ea0).stop 2021-01-06T22:21:25.402 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.392+0000 7f3d6a5cb700 1 -- 172.21.15.203:0/3423787351 shutdown_connections 2021-01-06T22:21:25.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.392+0000 7f3d6a5cb700 1 --2- 172.21.15.203:0/3423787351 >> v2:172.21.15.91:3300/0 conn(0x7f3d64067660 0x7f3d6414dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:25.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.392+0000 7f3d6a5cb700 1 --2- 172.21.15.203:0/3423787351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d64067a40 0x7f3d64068330 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:25.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.392+0000 7f3d6a5cb700 1 -- 172.21.15.203:0/3423787351 >> 172.21.15.203:0/3423787351 conn(0x7f3d640c0bf0 msgr2=0x7f3d640c0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:25.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.392+0000 7f3d6a5cb700 1 -- 172.21.15.203:0/3423787351 shutdown_connections 2021-01-06T22:21:25.403 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.392+0000 7f3d6a5cb700 1 -- 172.21.15.203:0/3423787351 wait complete. 2021-01-06T22:21:25.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.392+0000 7f3d6a5cb700 1 Processor -- start 2021-01-06T22:21:25.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.392+0000 7f3d6a5cb700 1 -- start start 2021-01-06T22:21:25.410 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d6a5cb700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d64061830 0x7f3d64062880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:25.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d6a5cb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d64067660 0x7f3d640b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:25.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d6a5cb700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d64067a40 0x7f3d640b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:25.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d6a5cb700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640634d0 con 0x7f3d64061830 2021-01-06T22:21:25.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d6a5cb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d64063770 con 0x7f3d64067660 2021-01-06T22:21:25.411 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d6a5cb700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640b5b10 con 0x7f3d64067a40 2021-01-06T22:21:25.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d637fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d64067660 0x7f3d640b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:25.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d637fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d64067660 0x7f3d640b6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53400/0 (socket says 172.21.15.203:53400) 2021-01-06T22:21:25.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d63fff700 1 --2- 172.21.15.203:0/2031524662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d64061830 0x7f3d64062880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:25.412 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d637fe700 1 -- 172.21.15.203:0/2031524662 learned_addr learned my addr 172.21.15.203:0/2031524662 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:25.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d68b68700 1 --2- 172.21.15.203:0/2031524662 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d64067a40 0x7f3d640b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:25.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d637fe700 1 -- 172.21.15.203:0/2031524662 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d64067a40 msgr2=0x7f3d640b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:25.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d637fe700 1 --2- 172.21.15.203:0/2031524662 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d64067a40 0x7f3d640b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:25.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d637fe700 1 -- 172.21.15.203:0/2031524662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d64061830 msgr2=0x7f3d64062880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:25.413 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d637fe700 1 --2- 172.21.15.203:0/2031524662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d64061830 0x7f3d64062880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:25.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d637fe700 1 -- 172.21.15.203:0/2031524662 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d640b5db0 con 0x7f3d64067660 2021-01-06T22:21:25.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d63fff700 1 --2- 172.21.15.203:0/2031524662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d64061830 0x7f3d64062880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:21:25.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d637fe700 1 --2- 172.21.15.203:0/2031524662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d64067660 0x7f3d640b6f20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f3d640b8440 tx=0x7f3d4c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:25.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d5bfff700 1 -- 172.21.15.203:0/2031524662 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c011cd0 con 0x7f3d64067660 2021-01-06T22:21:25.414 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d6a5cb700 1 -- 172.21.15.203:0/2031524662 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d640b6050 con 0x7f3d64067660 2021-01-06T22:21:25.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d5bfff700 1 -- 172.21.15.203:0/2031524662 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c011e30 con 0x7f3d64067660 2021-01-06T22:21:25.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d5bfff700 1 -- 172.21.15.203:0/2031524662 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c01d660 con 0x7f3d64067660 2021-01-06T22:21:25.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.396+0000 7f3d6a5cb700 1 -- 172.21.15.203:0/2031524662 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6414b820 con 0x7f3d64067660 2021-01-06T22:21:25.415 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.400+0000 7f3d6a5cb700 1 -- 172.21.15.203:0/2031524662 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6405d600 con 0x7f3d64067660 2021-01-06T22:21:25.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.400+0000 7f3d5bfff700 1 -- 172.21.15.203:0/2031524662 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f3d4c00f070 con 0x7f3d64067660 2021-01-06T22:21:25.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.400+0000 7f3d5bfff700 1 --2- 172.21.15.203:0/2031524662 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3d480563d0 0x7f3d48058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:25.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.400+0000 7f3d5bfff700 1 -- 172.21.15.203:0/2031524662 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7f3d4c0712f0 con 0x7f3d64067660 2021-01-06T22:21:25.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.400+0000 7f3d63fff700 1 --2- 172.21.15.203:0/2031524662 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3d480563d0 0x7f3d48058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:25.416 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.400+0000 7f3d63fff700 1 --2- 172.21.15.203:0/2031524662 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3d480563d0 0x7f3d48058890 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f3d54009fd0 tx=0x7f3d540091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:25.417 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.408+0000 7f3d5bfff700 1 -- 172.21.15.203:0/2031524662 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f3d4c046700 con 0x7f3d64067660 2021-01-06T22:21:25.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.612+0000 7f3d6a5cb700 1 -- 172.21.15.203:0/2031524662 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d640b61e0 con 0x7f3d64067660 2021-01-06T22:21:25.619 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.612+0000 7f3d5bfff700 1 -- 172.21.15.203:0/2031524662 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+3568 (secure 0 0 0) 0x7f3d640b61e0 con 0x7f3d64067660 2021-01-06T22:21:25.620 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:25.621 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":87,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":83}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:19.698147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4724,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":204,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:25.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.616+0000 7f3d6a5cb700 1 -- 172.21.15.203:0/2031524662 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3d480563d0 msgr2=0x7f3d48058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:25.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.616+0000 7f3d6a5cb700 1 --2- 172.21.15.203:0/2031524662 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3d480563d0 0x7f3d48058890 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f3d54009fd0 tx=0x7f3d540091d0).stop 2021-01-06T22:21:25.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.616+0000 7f3d6a5cb700 1 -- 172.21.15.203:0/2031524662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d64067660 msgr2=0x7f3d640b6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:25.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.616+0000 7f3d6a5cb700 1 --2- 172.21.15.203:0/2031524662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d64067660 0x7f3d640b6f20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f3d640b8440 tx=0x7f3d4c016000).stop 2021-01-06T22:21:25.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.616+0000 7f3d6a5cb700 1 -- 172.21.15.203:0/2031524662 shutdown_connections 2021-01-06T22:21:25.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.616+0000 7f3d6a5cb700 1 --2- 172.21.15.203:0/2031524662 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3d480563d0 0x7f3d48058890 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:25.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.616+0000 7f3d6a5cb700 1 --2- 172.21.15.203:0/2031524662 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d64067a40 0x7f3d640b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:25.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.616+0000 7f3d6a5cb700 1 --2- 172.21.15.203:0/2031524662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d64061830 0x7f3d64062880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:25.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.616+0000 7f3d6a5cb700 1 --2- 172.21.15.203:0/2031524662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d64067660 0x7f3d640b6f20 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:25.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.616+0000 7f3d6a5cb700 1 -- 172.21.15.203:0/2031524662 >> 172.21.15.203:0/2031524662 conn(0x7f3d640c0bf0 msgr2=0x7f3d640b9840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:25.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.616+0000 7f3d6a5cb700 1 -- 172.21.15.203:0/2031524662 shutdown_connections 2021-01-06T22:21:25.625 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:25.616+0000 7f3d6a5cb700 1 -- 172.21.15.203:0/2031524662 wait complete. 2021-01-06T22:21:25.625 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 87 2021-01-06T22:21:25.640 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2021-01-06T22:21:27.641 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:27.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.768+0000 7fbf091af700 1 Processor -- start 2021-01-06T22:21:27.773 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.768+0000 7fbf091af700 1 -- start start 2021-01-06T22:21:27.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.768+0000 7fbf091af700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbf04147dc0 0x7fbf04148250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:27.776 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.768+0000 7fbf091af700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf040bbbb0 con 0x7fbf0414e9a0 2021-01-06T22:21:27.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.768+0000 7fbf091af700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf040bbe50 con 0x7fbf0414e5c0 2021-01-06T22:21:27.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.768+0000 7fbf091af700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf040bc0f0 con 0x7fbf04147dc0 2021-01-06T22:21:27.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.768+0000 7fbf0359e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbf04147dc0 0x7fbf04148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:27.777 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.768+0000 7fbf0359e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbf04147dc0 0x7fbf04148250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53412/0 (socket says 172.21.15.203:53412) 2021-01-06T22:21:27.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.768+0000 7fbf0359e700 1 -- 172.21.15.203:0/2846202600 learned_addr learned my addr 172.21.15.203:0/2846202600 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:27.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.768+0000 7fbf0359e700 1 -- 172.21.15.203:0/2846202600 >> v1:172.21.15.203:6789/0 conn(0x7fbf0414e5c0 legacy=0x7fbf040602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:27.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.768+0000 7fbf0359e700 1 -- 172.21.15.203:0/2846202600 >> v1:172.21.15.91:6789/0 conn(0x7fbf0414e9a0 legacy=0x7fbf04067f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:27.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.768+0000 7fbf0359e700 1 -- 172.21.15.203:0/2846202600 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf040bc390 con 0x7fbf04147dc0 2021-01-06T22:21:27.778 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.768+0000 7fbf0359e700 1 --2- 172.21.15.203:0/2846202600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf04147dc0 0x7fbf04148250 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7fbefc009d50 tx=0x7fbefc009290).ready entity=mon.1 client_cookie=b45763ca59563e36 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:27.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.768+0000 7fbf01d9b700 1 -- 172.21.15.203:0/2846202600 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc00bcd0 con 0x7fbf04147dc0 2021-01-06T22:21:27.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf01d9b700 1 -- 172.21.15.203:0/2846202600 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbefc00be30 con 0x7fbf04147dc0 2021-01-06T22:21:27.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf01d9b700 1 -- 172.21.15.203:0/2846202600 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc011610 con 0x7fbf04147dc0 2021-01-06T22:21:27.779 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf091af700 1 -- 172.21.15.203:0/2846202600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf04147dc0 msgr2=0x7fbf04148250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:27.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf091af700 1 --2- 172.21.15.203:0/2846202600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf04147dc0 0x7fbf04148250 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7fbefc009d50 tx=0x7fbefc009290).stop 2021-01-06T22:21:27.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf091af700 1 -- 172.21.15.203:0/2846202600 shutdown_connections 2021-01-06T22:21:27.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf091af700 1 --2- 172.21.15.203:0/2846202600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf04147dc0 0x7fbf04148250 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:27.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf091af700 1 -- 172.21.15.203:0/2846202600 >> 172.21.15.203:0/2846202600 conn(0x7fbf040631b0 msgr2=0x7fbf040c0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:27.780 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf091af700 1 -- 172.21.15.203:0/2846202600 shutdown_connections 2021-01-06T22:21:27.781 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf091af700 1 -- 172.21.15.203:0/2846202600 wait complete. 2021-01-06T22:21:27.783 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf091af700 1 Processor -- start 2021-01-06T22:21:27.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf091af700 1 -- start start 2021-01-06T22:21:27.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf091af700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf04147dc0 0x7fbf0414cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:27.784 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf091af700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbf0414e5c0 0x7fbf0415f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:27.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf091af700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbf0414e9a0 0x7fbf0415f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:27.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf091af700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0415eeb0 con 0x7fbf0414e5c0 2021-01-06T22:21:27.785 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf091af700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf040ba0c0 con 0x7fbf04147dc0 2021-01-06T22:21:27.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf091af700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0415fc80 con 0x7fbf0414e9a0 2021-01-06T22:21:27.786 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf02d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf04147dc0 0x7fbf0414cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:27.787 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf02d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf04147dc0 0x7fbf0414cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53418/0 (socket says 172.21.15.203:53418) 2021-01-06T22:21:27.787 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf02d9d700 1 -- 172.21.15.203:0/2253328949 learned_addr learned my addr 172.21.15.203:0/2253328949 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:27.787 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf0259c700 1 --2- 172.21.15.203:0/2253328949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbf0414e5c0 0x7fbf0415f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:27.787 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.772+0000 7fbf0359e700 1 --2- 172.21.15.203:0/2253328949 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbf0414e9a0 0x7fbf0415f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:27.788 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.776+0000 7fbf02d9d700 1 -- 172.21.15.203:0/2253328949 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbf0414e9a0 msgr2=0x7fbf0415f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:27.788 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.776+0000 7fbf02d9d700 1 --2- 172.21.15.203:0/2253328949 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbf0414e9a0 0x7fbf0415f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:27.788 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.776+0000 7fbf02d9d700 1 -- 172.21.15.203:0/2253328949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbf0414e5c0 msgr2=0x7fbf0415f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:27.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.776+0000 7fbf02d9d700 1 --2- 172.21.15.203:0/2253328949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbf0414e5c0 0x7fbf0415f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:27.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.776+0000 7fbf02d9d700 1 -- 172.21.15.203:0/2253328949 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf040a09f0 con 0x7fbf04147dc0 2021-01-06T22:21:27.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.776+0000 7fbf0259c700 1 --2- 172.21.15.203:0/2253328949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbf0414e5c0 0x7fbf0415f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:21:27.789 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.776+0000 7fbf0359e700 1 --2- 172.21.15.203:0/2253328949 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbf0414e9a0 0x7fbf0415f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:21:27.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.776+0000 7fbf02d9d700 1 --2- 172.21.15.203:0/2253328949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf04147dc0 0x7fbf0414cdc0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7fbef400c710 tx=0x7fbef4007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:27.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.776+0000 7fbeeaffd700 1 -- 172.21.15.203:0/2253328949 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4007020 con 0x7fbf04147dc0 2021-01-06T22:21:27.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.776+0000 7fbeeaffd700 1 -- 172.21.15.203:0/2253328949 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef4007180 con 0x7fbf04147dc0 2021-01-06T22:21:27.790 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.776+0000 7fbf091af700 1 -- 172.21.15.203:0/2253328949 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf040b9e20 con 0x7fbf04147dc0 2021-01-06T22:21:27.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.776+0000 7fbeeaffd700 1 -- 172.21.15.203:0/2253328949 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef400e560 con 0x7fbf04147dc0 2021-01-06T22:21:27.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.776+0000 7fbf091af700 1 -- 172.21.15.203:0/2253328949 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf040a0ef0 con 0x7fbf04147dc0 2021-01-06T22:21:27.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.776+0000 7fbf091af700 1 -- 172.21.15.203:0/2253328949 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0405cfe0 con 0x7fbf04147dc0 2021-01-06T22:21:27.791 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.776+0000 7fbeeaffd700 1 -- 172.21.15.203:0/2253328949 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fbef4018020 con 0x7fbf04147dc0 2021-01-06T22:21:27.792 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.780+0000 7fbeeaffd700 1 --2- 172.21.15.203:0/2253328949 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbee40563d0 0x7fbee4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:27.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.780+0000 7fbeeaffd700 1 -- 172.21.15.203:0/2253328949 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7fbef406f390 con 0x7fbf04147dc0 2021-01-06T22:21:27.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.780+0000 7fbf0259c700 1 --2- 172.21.15.203:0/2253328949 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbee40563d0 0x7fbee4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:27.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.780+0000 7fbf0259c700 1 --2- 172.21.15.203:0/2253328949 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbee40563d0 0x7fbee4058890 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fbf0414d6b0 tx=0x7fbef8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:27.793 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.784+0000 7fbeeaffd700 1 -- 172.21.15.203:0/2253328949 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fbef40447a0 con 0x7fbf04147dc0 2021-01-06T22:21:27.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.984+0000 7fbf091af700 1 -- 172.21.15.203:0/2253328949 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf04148aa0 con 0x7fbf04147dc0 2021-01-06T22:21:27.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.984+0000 7fbeeaffd700 1 -- 172.21.15.203:0/2253328949 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+3568 (secure 0 0 0) 0x7fbef4016030 con 0x7fbf04147dc0 2021-01-06T22:21:27.990 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:27.990 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":87,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":83}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:19.698147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4724,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":204,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:27.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.988+0000 7fbf091af700 1 -- 172.21.15.203:0/2253328949 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbee40563d0 msgr2=0x7fbee4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:27.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.988+0000 7fbf091af700 1 --2- 172.21.15.203:0/2253328949 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbee40563d0 0x7fbee4058890 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fbf0414d6b0 tx=0x7fbef8008040).stop 2021-01-06T22:21:27.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.988+0000 7fbf091af700 1 -- 172.21.15.203:0/2253328949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf04147dc0 msgr2=0x7fbf0414cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:27.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.988+0000 7fbf091af700 1 --2- 172.21.15.203:0/2253328949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf04147dc0 0x7fbf0414cdc0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7fbef400c710 tx=0x7fbef4007c40).stop 2021-01-06T22:21:27.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.988+0000 7fbf091af700 1 -- 172.21.15.203:0/2253328949 shutdown_connections 2021-01-06T22:21:27.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.988+0000 7fbf091af700 1 --2- 172.21.15.203:0/2253328949 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbee40563d0 0x7fbee4058890 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:27.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.988+0000 7fbf091af700 1 --2- 172.21.15.203:0/2253328949 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbf0414e9a0 0x7fbf0415f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:27.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.988+0000 7fbf091af700 1 --2- 172.21.15.203:0/2253328949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf04147dc0 0x7fbf0414cdc0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:27.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.988+0000 7fbf091af700 1 --2- 172.21.15.203:0/2253328949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbf0414e5c0 0x7fbf0415f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:27.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.988+0000 7fbf091af700 1 -- 172.21.15.203:0/2253328949 >> 172.21.15.203:0/2253328949 conn(0x7fbf040631b0 msgr2=0x7fbf041472c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:27.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.988+0000 7fbf091af700 1 -- 172.21.15.203:0/2253328949 shutdown_connections 2021-01-06T22:21:27.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:27.988+0000 7fbf091af700 1 -- 172.21.15.203:0/2253328949 wait complete. 2021-01-06T22:21:27.999 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 87 2021-01-06T22:21:28.004 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2021-01-06T22:21:30.005 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:30.142 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff822115700 1 Processor -- start 2021-01-06T22:21:30.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff822115700 1 -- start start 2021-01-06T22:21:30.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff822115700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff81c0c04c0 0x7ff81c0680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:30.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff822115700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff81c0c08a0 0x7ff81c063ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:30.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff822115700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c1531f0 con 0x7ff81c155b60 2021-01-06T22:21:30.143 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff822115700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c1522e0 con 0x7ff81c0c04c0 2021-01-06T22:21:30.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff822115700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c063a10 con 0x7ff81c0c08a0 2021-01-06T22:21:30.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff81b7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff81c0c04c0 0x7ff81c0680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:30.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff81b7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff81c0c04c0 0x7ff81c0680b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53424/0 (socket says 172.21.15.203:53424) 2021-01-06T22:21:30.144 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff81b7fe700 1 -- 172.21.15.203:0/1989302470 learned_addr learned my addr 172.21.15.203:0/1989302470 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:30.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff81b7fe700 1 -- 172.21.15.203:0/1989302470 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff81c0c08a0 msgr2=0x7ff81c063ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:30.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff81b7fe700 1 --2- 172.21.15.203:0/1989302470 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff81c0c08a0 0x7ff81c063ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:21:30.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff81b7fe700 1 -- 172.21.15.203:0/1989302470 >> v1:172.21.15.91:6789/0 conn(0x7ff81c155b60 legacy=0x7ff81c14f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:30.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff81b7fe700 1 -- 172.21.15.203:0/1989302470 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c1524d0 con 0x7ff81c0c04c0 2021-01-06T22:21:30.145 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff81b7fe700 1 --2- 172.21.15.203:0/1989302470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff81c0c04c0 0x7ff81c0680b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7ff80c00b9e0 tx=0x7ff80c006ea0).ready entity=mon.1 client_cookie=5f502ee5952ea8da server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:30.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff81a7fc700 1 -- 172.21.15.203:0/1989302470 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c009920 con 0x7ff81c0c04c0 2021-01-06T22:21:30.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff81a7fc700 1 -- 172.21.15.203:0/1989302470 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80c009a80 con 0x7ff81c0c04c0 2021-01-06T22:21:30.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff81a7fc700 1 -- 172.21.15.203:0/1989302470 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c009e00 con 0x7ff81c0c04c0 2021-01-06T22:21:30.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff822115700 1 -- 172.21.15.203:0/1989302470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff81c0c04c0 msgr2=0x7ff81c0680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:30.146 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff822115700 1 --2- 172.21.15.203:0/1989302470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff81c0c04c0 0x7ff81c0680b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7ff80c00b9e0 tx=0x7ff80c006ea0).stop 2021-01-06T22:21:30.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff822115700 1 -- 172.21.15.203:0/1989302470 shutdown_connections 2021-01-06T22:21:30.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff822115700 1 --2- 172.21.15.203:0/1989302470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff81c0c04c0 0x7ff81c0680b0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:30.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff822115700 1 --2- 172.21.15.203:0/1989302470 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff81c0c08a0 0x7ff81c063ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:21:30.147 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff822115700 1 -- 172.21.15.203:0/1989302470 >> 172.21.15.203:0/1989302470 conn(0x7ff81c056cd0 msgr2=0x7ff81c0626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:30.148 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff822115700 1 -- 172.21.15.203:0/1989302470 shutdown_connections 2021-01-06T22:21:30.148 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.136+0000 7ff822115700 1 -- 172.21.15.203:0/1989302470 wait complete. 2021-01-06T22:21:30.151 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.140+0000 7ff822115700 1 Processor -- start 2021-01-06T22:21:30.151 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.140+0000 7ff822115700 1 -- start start 2021-01-06T22:21:30.152 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.140+0000 7ff822115700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff81c0c04c0 0x7ff81c0bfa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:30.152 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.140+0000 7ff822115700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff81c0c08a0 0x7ff81c14be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:30.153 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.140+0000 7ff822115700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff81c155b60 0x7ff81c14c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:30.153 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.140+0000 7ff822115700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c15a120 con 0x7ff81c0c04c0 2021-01-06T22:21:30.153 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.140+0000 7ff822115700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c159e80 con 0x7ff81c155b60 2021-01-06T22:21:30.153 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.140+0000 7ff822115700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c159180 con 0x7ff81c0c08a0 2021-01-06T22:21:30.153 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.140+0000 7ff81bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff81c155b60 0x7ff81c14c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:30.154 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.140+0000 7ff81bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff81c155b60 0x7ff81c14c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53434/0 (socket says 172.21.15.203:53434) 2021-01-06T22:21:30.154 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.140+0000 7ff81bfff700 1 -- 172.21.15.203:0/703507233 learned_addr learned my addr 172.21.15.203:0/703507233 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:30.154 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.140+0000 7ff81b7fe700 1 --2- 172.21.15.203:0/703507233 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff81c0c04c0 0x7ff81c0bfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:30.154 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.140+0000 7ff81affd700 1 --2- 172.21.15.203:0/703507233 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff81c0c08a0 0x7ff81c14be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:30.155 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.144+0000 7ff81bfff700 1 -- 172.21.15.203:0/703507233 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff81c0c08a0 msgr2=0x7ff81c14be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:30.155 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.144+0000 7ff81bfff700 1 --2- 172.21.15.203:0/703507233 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff81c0c08a0 0x7ff81c14be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:30.155 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.144+0000 7ff81bfff700 1 -- 172.21.15.203:0/703507233 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff81c0c04c0 msgr2=0x7ff81c0bfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:30.155 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.144+0000 7ff81bfff700 1 --2- 172.21.15.203:0/703507233 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff81c0c04c0 0x7ff81c0bfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:30.155 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.144+0000 7ff81bfff700 1 -- 172.21.15.203:0/703507233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c159420 con 0x7ff81c155b60 2021-01-06T22:21:30.156 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.144+0000 7ff81bfff700 1 --2- 172.21.15.203:0/703507233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff81c155b60 0x7ff81c14c350 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7ff81400cb10 tx=0x7ff8140023b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:30.156 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.144+0000 7ff8037fe700 1 -- 172.21.15.203:0/703507233 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400a730 con 0x7ff81c155b60 2021-01-06T22:21:30.156 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.144+0000 7ff8037fe700 1 -- 172.21.15.203:0/703507233 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81400a8b0 con 0x7ff81c155b60 2021-01-06T22:21:30.156 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.144+0000 7ff822115700 1 -- 172.21.15.203:0/703507233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c1596c0 con 0x7ff81c155b60 2021-01-06T22:21:30.156 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.144+0000 7ff8037fe700 1 -- 172.21.15.203:0/703507233 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400ac70 con 0x7ff81c155b60 2021-01-06T22:21:30.157 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.144+0000 7ff822115700 1 -- 172.21.15.203:0/703507233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c0bfef0 con 0x7ff81c155b60 2021-01-06T22:21:30.157 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.144+0000 7ff822115700 1 -- 172.21.15.203:0/703507233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c14c840 con 0x7ff81c155b60 2021-01-06T22:21:30.157 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.144+0000 7ff8037fe700 1 -- 172.21.15.203:0/703507233 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff814019030 con 0x7ff81c155b60 2021-01-06T22:21:30.157 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.144+0000 7ff8037fe700 1 --2- 172.21.15.203:0/703507233 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff7fc0563d0 0x7ff7fc058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:30.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.148+0000 7ff8037fe700 1 -- 172.21.15.203:0/703507233 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6933+0+0 (secure 0 0 0) 0x7ff81406f070 con 0x7ff81c155b60 2021-01-06T22:21:30.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.148+0000 7ff81b7fe700 1 --2- 172.21.15.203:0/703507233 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff7fc0563d0 0x7ff7fc058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:30.158 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.148+0000 7ff81b7fe700 1 --2- 172.21.15.203:0/703507233 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff7fc0563d0 0x7ff7fc058890 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7ff80c007fd0 tx=0x7ff80c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:30.159 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.152+0000 7ff8037fe700 1 -- 172.21.15.203:0/703507233 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff814044640 con 0x7ff81c155b60 2021-01-06T22:21:30.357 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.352+0000 7ff822115700 1 -- 172.21.15.203:0/703507233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff81c064e60 con 0x7ff81c155b60 2021-01-06T22:21:30.358 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.352+0000 7ff8037fe700 1 -- 172.21.15.203:0/703507233 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+3568 (secure 0 0 0) 0x7ff814047c60 con 0x7ff81c155b60 2021-01-06T22:21:30.359 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:30.359 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":87,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":83}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:19.698147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4724,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4724":{"gid":4724,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":204,"addr":"172.21.15.91:6836/2428882934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2428882934},{"type":"v1","addr":"172.21.15.91:6836","nonce":2428882934}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:30.360 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.356+0000 7ff822115700 1 -- 172.21.15.203:0/703507233 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff7fc0563d0 msgr2=0x7ff7fc058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:30.361 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.356+0000 7ff822115700 1 --2- 172.21.15.203:0/703507233 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff7fc0563d0 0x7ff7fc058890 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7ff80c007fd0 tx=0x7ff80c016000).stop 2021-01-06T22:21:30.361 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.356+0000 7ff822115700 1 -- 172.21.15.203:0/703507233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff81c155b60 msgr2=0x7ff81c14c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:30.361 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.356+0000 7ff822115700 1 --2- 172.21.15.203:0/703507233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff81c155b60 0x7ff81c14c350 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7ff81400cb10 tx=0x7ff8140023b0).stop 2021-01-06T22:21:30.361 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.356+0000 7ff822115700 1 -- 172.21.15.203:0/703507233 shutdown_connections 2021-01-06T22:21:30.362 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.356+0000 7ff822115700 1 --2- 172.21.15.203:0/703507233 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff7fc0563d0 0x7ff7fc058890 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:30.362 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.356+0000 7ff822115700 1 --2- 172.21.15.203:0/703507233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff81c155b60 0x7ff81c14c350 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:30.362 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.356+0000 7ff822115700 1 --2- 172.21.15.203:0/703507233 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff81c0c04c0 0x7ff81c0bfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:30.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.356+0000 7ff822115700 1 --2- 172.21.15.203:0/703507233 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff81c0c08a0 0x7ff81c14be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:30.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.356+0000 7ff822115700 1 -- 172.21.15.203:0/703507233 >> 172.21.15.203:0/703507233 conn(0x7ff81c056cd0 msgr2=0x7ff81c063c70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:30.363 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.356+0000 7ff822115700 1 -- 172.21.15.203:0/703507233 shutdown_connections 2021-01-06T22:21:30.364 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:30.356+0000 7ff822115700 1 -- 172.21.15.203:0/703507233 wait complete. 2021-01-06T22:21:30.364 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 87 2021-01-06T22:21:30.373 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2021-01-06T22:21:32.374 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:32.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5e2f5700 1 Processor -- start 2021-01-06T22:21:32.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5e2f5700 1 -- start start 2021-01-06T22:21:32.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5e2f5700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7eff58067c10 0x7eff58067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:32.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5e2f5700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7eff58151c60 0x7eff58064550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:32.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5e2f5700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff5814f320 0x7eff5814f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:32.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5e2f5700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5814ddd0 con 0x7eff58067c10 2021-01-06T22:21:32.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5e2f5700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff58152110 con 0x7eff58151c60 2021-01-06T22:21:32.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5e2f5700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5814e000 con 0x7eff5814f320 2021-01-06T22:21:32.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5caf2700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7eff58151c60 0x7eff58064550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:32.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5caf2700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7eff58151c60 0x7eff58064550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53450/0 (socket says 172.21.15.203:53450) 2021-01-06T22:21:32.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5caf2700 1 -- 172.21.15.203:0/2265982046 learned_addr learned my addr 172.21.15.203:0/2265982046 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:32.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5daf4700 1 --2- 172.21.15.203:0/2265982046 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff5814f320 0x7eff5814f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:32.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5d2f3700 1 --2- 172.21.15.203:0/2265982046 >> v2:172.21.15.91:3300/0 conn(0x7eff58067c10 0x7eff58067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:32.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5caf2700 1 -- 172.21.15.203:0/2265982046 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff5814f320 msgr2=0x7eff5814f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:32.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5caf2700 1 --2- 172.21.15.203:0/2265982046 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff5814f320 0x7eff5814f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5caf2700 1 -- 172.21.15.203:0/2265982046 >> v2:172.21.15.91:3300/0 conn(0x7eff58067c10 msgr2=0x7eff58067ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:32.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5caf2700 1 --2- 172.21.15.203:0/2265982046 >> v2:172.21.15.91:3300/0 conn(0x7eff58067c10 0x7eff58067ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5caf2700 1 -- 172.21.15.203:0/2265982046 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5814e2a0 con 0x7eff58151c60 2021-01-06T22:21:32.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff5caf2700 1 --2- 172.21.15.203:0/2265982046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff58151c60 0x7eff58064550 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7eff58057d80 tx=0x7eff50006ea0).ready entity=mon.1 client_cookie=66fafc4833a776b0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:32.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff57fff700 1 -- 172.21.15.203:0/2265982046 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50009770 con 0x7eff58151c60 2021-01-06T22:21:32.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff57fff700 1 -- 172.21.15.203:0/2265982046 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff500098d0 con 0x7eff58151c60 2021-01-06T22:21:32.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.516+0000 7eff57fff700 1 -- 172.21.15.203:0/2265982046 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50009c50 con 0x7eff58151c60 2021-01-06T22:21:32.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.520+0000 7eff5e2f5700 1 -- 172.21.15.203:0/2265982046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff58151c60 msgr2=0x7eff58064550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:32.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.520+0000 7eff5e2f5700 1 --2- 172.21.15.203:0/2265982046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff58151c60 0x7eff58064550 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7eff58057d80 tx=0x7eff50006ea0).stop 2021-01-06T22:21:32.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.520+0000 7eff5e2f5700 1 -- 172.21.15.203:0/2265982046 shutdown_connections 2021-01-06T22:21:32.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.520+0000 7eff5e2f5700 1 --2- 172.21.15.203:0/2265982046 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff5814f320 0x7eff5814f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.520+0000 7eff5e2f5700 1 --2- 172.21.15.203:0/2265982046 >> v2:172.21.15.91:3300/0 conn(0x7eff58067c10 0x7eff58067ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.520+0000 7eff5e2f5700 1 --2- 172.21.15.203:0/2265982046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff58151c60 0x7eff58064550 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.520+0000 7eff5e2f5700 1 -- 172.21.15.203:0/2265982046 >> 172.21.15.203:0/2265982046 conn(0x7eff58061530 msgr2=0x7eff58063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:32.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.520+0000 7eff5e2f5700 1 -- 172.21.15.203:0/2265982046 shutdown_connections 2021-01-06T22:21:32.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.520+0000 7eff5e2f5700 1 -- 172.21.15.203:0/2265982046 wait complete. 2021-01-06T22:21:32.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.520+0000 7eff5e2f5700 1 Processor -- start 2021-01-06T22:21:32.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.520+0000 7eff5e2f5700 1 -- start start 2021-01-06T22:21:32.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5e2f5700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff58067c10 0x7eff5815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:32.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5e2f5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff5814f320 0x7eff58148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:32.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5e2f5700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff58151c60 0x7eff580bbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:32.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5e2f5700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff580b7460 con 0x7eff58151c60 2021-01-06T22:21:32.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5e2f5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff580b76c0 con 0x7eff5814f320 2021-01-06T22:21:32.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5e2f5700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff580b8a90 con 0x7eff58067c10 2021-01-06T22:21:32.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5caf2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff5814f320 0x7eff58148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:32.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5caf2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff5814f320 0x7eff58148560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53458/0 (socket says 172.21.15.203:53458) 2021-01-06T22:21:32.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5caf2700 1 -- 172.21.15.203:0/3054491987 learned_addr learned my addr 172.21.15.203:0/3054491987 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:32.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5daf4700 1 --2- 172.21.15.203:0/3054491987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff58151c60 0x7eff580bbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:32.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5d2f3700 1 --2- 172.21.15.203:0/3054491987 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff58067c10 0x7eff5815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:32.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5caf2700 1 -- 172.21.15.203:0/3054491987 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff58067c10 msgr2=0x7eff5815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:32.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5caf2700 1 --2- 172.21.15.203:0/3054491987 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff58067c10 0x7eff5815e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5caf2700 1 -- 172.21.15.203:0/3054491987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff58151c60 msgr2=0x7eff580bbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:32.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5caf2700 1 --2- 172.21.15.203:0/3054491987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff58151c60 0x7eff580bbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5caf2700 1 -- 172.21.15.203:0/3054491987 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff580b8d30 con 0x7eff5814f320 2021-01-06T22:21:32.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5daf4700 1 --2- 172.21.15.203:0/3054491987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff58151c60 0x7eff580bbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:21:32.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5caf2700 1 --2- 172.21.15.203:0/3054491987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff5814f320 0x7eff58148560 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7eff50006ee0 tx=0x7eff50016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:32.535 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff557fa700 1 -- 172.21.15.203:0/3054491987 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50011dd0 con 0x7eff5814f320 2021-01-06T22:21:32.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff557fa700 1 -- 172.21.15.203:0/3054491987 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5001c420 con 0x7eff5814f320 2021-01-06T22:21:32.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5e2f5700 1 -- 172.21.15.203:0/3054491987 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff580a5b10 con 0x7eff5814f320 2021-01-06T22:21:32.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff557fa700 1 -- 172.21.15.203:0/3054491987 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5001c7c0 con 0x7eff5814f320 2021-01-06T22:21:32.536 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5e2f5700 1 -- 172.21.15.203:0/3054491987 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff58152fa0 con 0x7eff5814f320 2021-01-06T22:21:32.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff5e2f5700 1 -- 172.21.15.203:0/3054491987 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff580b88e0 con 0x7eff5814f320 2021-01-06T22:21:32.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff557fa700 1 -- 172.21.15.203:0/3054491987 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7eff5000f070 con 0x7eff5814f320 2021-01-06T22:21:32.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.524+0000 7eff557fa700 1 --2- 172.21.15.203:0/3054491987 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7eff3c0563d0 0x7eff3c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:32.537 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.528+0000 7eff557fa700 1 -- 172.21.15.203:0/3054491987 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7019+0+0 (secure 0 0 0) 0x7eff50070f90 con 0x7eff5814f320 2021-01-06T22:21:32.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.528+0000 7eff5d2f3700 1 --2- 172.21.15.203:0/3054491987 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7eff3c0563d0 0x7eff3c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:32.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.528+0000 7eff5d2f3700 1 --2- 172.21.15.203:0/3054491987 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7eff3c0563d0 0x7eff3c058890 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7eff580682a0 tx=0x7eff40008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:32.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.532+0000 7eff557fa700 1 -- 172.21.15.203:0/3054491987 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7eff50046460 con 0x7eff5814f320 2021-01-06T22:21:32.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.732+0000 7eff5e2f5700 1 -- 172.21.15.203:0/3054491987 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff580b78c0 con 0x7eff5814f320 2021-01-06T22:21:32.743 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.736+0000 7eff557fa700 1 -- 172.21.15.203:0/3054491987 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+3215 (secure 0 0 0) 0x7eff50049a80 con 0x7eff5814f320 2021-01-06T22:21:32.745 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:32.745 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:32.723657+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:resolve","state_seq":31,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:32.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.736+0000 7eff5e2f5700 1 -- 172.21.15.203:0/3054491987 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7eff3c0563d0 msgr2=0x7eff3c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:32.752 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.736+0000 7eff5e2f5700 1 --2- 172.21.15.203:0/3054491987 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7eff3c0563d0 0x7eff3c058890 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7eff580682a0 tx=0x7eff40008040).stop 2021-01-06T22:21:32.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.736+0000 7eff5e2f5700 1 -- 172.21.15.203:0/3054491987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff5814f320 msgr2=0x7eff58148560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:32.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.736+0000 7eff5e2f5700 1 --2- 172.21.15.203:0/3054491987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff5814f320 0x7eff58148560 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7eff50006ee0 tx=0x7eff50016000).stop 2021-01-06T22:21:32.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.736+0000 7eff5e2f5700 1 -- 172.21.15.203:0/3054491987 shutdown_connections 2021-01-06T22:21:32.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.736+0000 7eff5e2f5700 1 --2- 172.21.15.203:0/3054491987 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7eff3c0563d0 0x7eff3c058890 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.753 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.736+0000 7eff5e2f5700 1 --2- 172.21.15.203:0/3054491987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff58151c60 0x7eff580bbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.736+0000 7eff5e2f5700 1 --2- 172.21.15.203:0/3054491987 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7eff58067c10 0x7eff5815e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.754 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.736+0000 7eff5e2f5700 1 --2- 172.21.15.203:0/3054491987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff5814f320 0x7eff58148560 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.736+0000 7eff5e2f5700 1 -- 172.21.15.203:0/3054491987 >> 172.21.15.203:0/3054491987 conn(0x7eff58061530 msgr2=0x7eff58152890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:32.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.736+0000 7eff5e2f5700 1 -- 172.21.15.203:0/3054491987 shutdown_connections 2021-01-06T22:21:32.755 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.736+0000 7eff5e2f5700 1 -- 172.21.15.203:0/3054491987 wait complete. 2021-01-06T22:21:32.755 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 89 2021-01-06T22:21:32.756 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2021-01-06T22:21:32.757 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2021-01-06T22:21:32.757 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:32.886 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.880+0000 7f2152f75700 1 Processor -- start 2021-01-06T22:21:32.886 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.880+0000 7f2152f75700 1 -- start start 2021-01-06T22:21:32.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.880+0000 7f2152f75700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f214c0bb3f0 0x7f214c0ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:32.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.880+0000 7f2152f75700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f214c0646b0 con 0x7f214c0bb7d0 2021-01-06T22:21:32.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.880+0000 7f2152f75700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f214c152cc0 con 0x7f214c0a5e50 2021-01-06T22:21:32.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.880+0000 7f2152f75700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f214c0670c0 con 0x7f214c0bb3f0 2021-01-06T22:21:32.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2151772700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f214c0a5e50 0x7f214c0669b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:52058/0 (socket says 172.21.15.203:52058) 2021-01-06T22:21:32.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2151772700 1 -- 172.21.15.203:0/2328715958 learned_addr learned my addr 172.21.15.203:0/2328715958 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:21:32.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2151f73700 1 --2- 172.21.15.203:0/2328715958 >> v2:172.21.15.91:3300/0 conn(0x7f214c0bb3f0 0x7f214c0ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:32.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 -- 172.21.15.203:0/2328715958 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3444255382 0 0) 0x7f214c152cc0 con 0x7f214c0a5e50 2021-01-06T22:21:32.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 -- 172.21.15.203:0/2328715958 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2138003230 con 0x7f214c0a5e50 2021-01-06T22:21:32.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 -- 172.21.15.203:0/2328715958 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3097098028 0 0) 0x7f2138003230 con 0x7f214c0a5e50 2021-01-06T22:21:32.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 -- 172.21.15.203:0/2328715958 >> v2:172.21.15.91:3300/0 conn(0x7f214c0bb3f0 msgr2=0x7f214c0ba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:32.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 --2- 172.21.15.203:0/2328715958 >> v2:172.21.15.91:3300/0 conn(0x7f214c0bb3f0 0x7f214c0ba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 -- 172.21.15.203:0/2328715958 >> v1:172.21.15.91:6789/0 conn(0x7f214c0bb7d0 legacy=0x7f214c14eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:32.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 -- 172.21.15.203:0/2328715958 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c064450 con 0x7f214c0a5e50 2021-01-06T22:21:32.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 -- 172.21.15.203:0/2328715958 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f2144001860 con 0x7f214c0a5e50 2021-01-06T22:21:32.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 -- 172.21.15.203:0/2328715958 >> v1:172.21.15.203:6789/0 conn(0x7f214c0a5e50 legacy=0x7f214c0669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:32.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 --2- 172.21.15.203:0/2328715958 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2138003720 0x7f2138003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:32.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 --2- 172.21.15.203:0/2328715958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21380044d0 0x7f2138004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:32.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 --2- 172.21.15.203:0/2328715958 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2138004e40 0x7f213800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:32.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 -- 172.21.15.203:0/2328715958 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c0670c0 con 0x7f2138004e40 2021-01-06T22:21:32.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 -- 172.21.15.203:0/2328715958 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2138004250 con 0x7f21380044d0 2021-01-06T22:21:32.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 -- 172.21.15.203:0/2328715958 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213800bc10 con 0x7f2138003720 2021-01-06T22:21:32.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2151772700 1 --2- 172.21.15.203:0/2328715958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21380044d0 0x7f2138004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:32.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2152774700 1 --2- 172.21.15.203:0/2328715958 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2138004e40 0x7f213800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:32.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2151f73700 1 --2- 172.21.15.203:0/2328715958 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2138003720 0x7f2138003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:32.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2151772700 1 -- 172.21.15.203:0/2328715958 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2138003720 msgr2=0x7f2138003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:32.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2151772700 1 --2- 172.21.15.203:0/2328715958 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2138003720 0x7f2138003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2151772700 1 -- 172.21.15.203:0/2328715958 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2138004e40 msgr2=0x7f213800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:32.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2151772700 1 --2- 172.21.15.203:0/2328715958 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2138004e40 0x7f213800b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2152774700 1 -- 172.21.15.203:0/2328715958 reap_dead start 2021-01-06T22:21:32.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2151772700 1 -- 172.21.15.203:0/2328715958 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2138003230 con 0x7f21380044d0 2021-01-06T22:21:32.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2151772700 1 --2- 172.21.15.203:0/2328715958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21380044d0 0x7f2138004950 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f214400c5d0 tx=0x7f214c1521a0).ready entity=mon.1 client_cookie=3d752555b5f88f33 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:32.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 -- 172.21.15.203:0/2328715958 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2144005b70 con 0x7f21380044d0 2021-01-06T22:21:32.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 -- 172.21.15.203:0/2328715958 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2144005cd0 con 0x7f21380044d0 2021-01-06T22:21:32.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2150f71700 1 -- 172.21.15.203:0/2328715958 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2144007300 con 0x7f21380044d0 2021-01-06T22:21:32.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2152f75700 1 -- 172.21.15.203:0/2328715958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21380044d0 msgr2=0x7f2138004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:32.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2152f75700 1 --2- 172.21.15.203:0/2328715958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21380044d0 0x7f2138004950 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f214400c5d0 tx=0x7f214c1521a0).stop 2021-01-06T22:21:32.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2152f75700 1 -- 172.21.15.203:0/2328715958 shutdown_connections 2021-01-06T22:21:32.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2152f75700 1 --2- 172.21.15.203:0/2328715958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21380044d0 0x7f2138004950 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2152f75700 1 -- 172.21.15.203:0/2328715958 >> 172.21.15.203:0/2328715958 conn(0x7f214c061530 msgr2=0x7f214c063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:32.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2152f75700 1 -- 172.21.15.203:0/2328715958 shutdown_connections 2021-01-06T22:21:32.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.884+0000 7f2152f75700 1 -- 172.21.15.203:0/2328715958 wait complete. 2021-01-06T22:21:32.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152f75700 1 Processor -- start 2021-01-06T22:21:32.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152f75700 1 -- start start 2021-01-06T22:21:32.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152f75700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21380044d0 0x7f214c0b97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:32.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152f75700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f214c150b80 0x7f214c14e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:32.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152f75700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c162720 0x7f214c0b8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:32.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152f75700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c150640 con 0x7f21380044d0 2021-01-06T22:21:32.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152f75700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c1508e0 con 0x7f214c162720 2021-01-06T22:21:32.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152f75700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214c0b8b40 con 0x7f214c150b80 2021-01-06T22:21:32.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152774700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c162720 0x7f214c0b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:32.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152774700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c162720 0x7f214c0b8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53476/0 (socket says 172.21.15.203:53476) 2021-01-06T22:21:32.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152774700 1 -- 172.21.15.203:0/2505750407 learned_addr learned my addr 172.21.15.203:0/2505750407 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:32.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2151772700 1 --2- 172.21.15.203:0/2505750407 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f214c150b80 0x7f214c14e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:32.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2151f73700 1 --2- 172.21.15.203:0/2505750407 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21380044d0 0x7f214c0b97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:32.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152774700 1 -- 172.21.15.203:0/2505750407 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f214c150b80 msgr2=0x7f214c14e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:32.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152774700 1 --2- 172.21.15.203:0/2505750407 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f214c150b80 0x7f214c14e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152774700 1 -- 172.21.15.203:0/2505750407 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21380044d0 msgr2=0x7f214c0b97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:32.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152774700 1 --2- 172.21.15.203:0/2505750407 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21380044d0 0x7f214c0b97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:32.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152774700 1 -- 172.21.15.203:0/2505750407 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c0b8de0 con 0x7f214c162720 2021-01-06T22:21:32.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152774700 1 --2- 172.21.15.203:0/2505750407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c162720 0x7f214c0b8040 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f214c0bbbb0 tx=0x7f213c007b80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:32.905 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2141ffb700 1 -- 172.21.15.203:0/2505750407 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c007240 con 0x7f214c162720 2021-01-06T22:21:32.905 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2141ffb700 1 -- 172.21.15.203:0/2505750407 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c0073a0 con 0x7f214c162720 2021-01-06T22:21:32.905 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152f75700 1 -- 172.21.15.203:0/2505750407 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214c0b9080 con 0x7f214c162720 2021-01-06T22:21:32.905 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2141ffb700 1 -- 172.21.15.203:0/2505750407 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c006730 con 0x7f214c162720 2021-01-06T22:21:32.905 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.888+0000 7f2152f75700 1 -- 172.21.15.203:0/2505750407 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f214c162b80 con 0x7f214c162720 2021-01-06T22:21:32.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.892+0000 7f2152f75700 1 -- 172.21.15.203:0/2505750407 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f214c0450a0 con 0x7f214c162720 2021-01-06T22:21:32.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.892+0000 7f2141ffb700 1 -- 172.21.15.203:0/2505750407 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f213c006ae0 con 0x7f214c162720 2021-01-06T22:21:32.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.892+0000 7f2141ffb700 1 --2- 172.21.15.203:0/2505750407 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f212c0563d0 0x7f212c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:32.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.892+0000 7f2151f73700 1 --2- 172.21.15.203:0/2505750407 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f212c0563d0 0x7f212c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:32.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.892+0000 7f2141ffb700 1 -- 172.21.15.203:0/2505750407 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7019+0+0 (secure 0 0 0) 0x7f213c06f930 con 0x7f214c162720 2021-01-06T22:21:32.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.892+0000 7f2151f73700 1 --2- 172.21.15.203:0/2505750407 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f212c0563d0 0x7f212c058890 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f214c14ff70 tx=0x7f2148008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:32.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:32.900+0000 7f2141ffb700 1 -- 172.21.15.203:0/2505750407 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f213c044e00 con 0x7f214c162720 2021-01-06T22:21:33.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:33.100+0000 7f2152f75700 1 -- 172.21.15.203:0/2505750407 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f214c0b8920 con 0x7f214c162720 2021-01-06T22:21:33.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:33.100+0000 7f2141ffb700 1 -- 172.21.15.203:0/2505750407 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+3215 (secure 0 0 0) 0x7f213c048420 con 0x7f214c162720 2021-01-06T22:21:33.106 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:33.107 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:32.723657+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:resolve","state_seq":31,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:33.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:33.104+0000 7f2152f75700 1 -- 172.21.15.203:0/2505750407 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f212c0563d0 msgr2=0x7f212c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:33.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:33.104+0000 7f2152f75700 1 --2- 172.21.15.203:0/2505750407 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f212c0563d0 0x7f212c058890 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f214c14ff70 tx=0x7f2148008040).stop 2021-01-06T22:21:33.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:33.104+0000 7f2152f75700 1 -- 172.21.15.203:0/2505750407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c162720 msgr2=0x7f214c0b8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:33.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:33.104+0000 7f2152f75700 1 --2- 172.21.15.203:0/2505750407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c162720 0x7f214c0b8040 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f214c0bbbb0 tx=0x7f213c007b80).stop 2021-01-06T22:21:33.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:33.104+0000 7f2152f75700 1 -- 172.21.15.203:0/2505750407 shutdown_connections 2021-01-06T22:21:33.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:33.104+0000 7f2152f75700 1 --2- 172.21.15.203:0/2505750407 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f212c0563d0 0x7f212c058890 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:33.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:33.104+0000 7f2152f75700 1 --2- 172.21.15.203:0/2505750407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c162720 0x7f214c0b8040 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:33.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:33.104+0000 7f2152f75700 1 --2- 172.21.15.203:0/2505750407 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21380044d0 0x7f214c0b97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:33.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:33.104+0000 7f2152f75700 1 --2- 172.21.15.203:0/2505750407 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f214c150b80 0x7f214c14e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:33.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:33.104+0000 7f2152f75700 1 -- 172.21.15.203:0/2505750407 >> 172.21.15.203:0/2505750407 conn(0x7f214c061530 msgr2=0x7f214c0ba6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:33.114 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:33.104+0000 7f2152f75700 1 -- 172.21.15.203:0/2505750407 shutdown_connections 2021-01-06T22:21:33.114 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:33.104+0000 7f2152f75700 1 -- 172.21.15.203:0/2505750407 wait complete. 2021-01-06T22:21:33.116 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 89 2021-01-06T22:21:33.128 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 89, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 89, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5632": { "addr": "172.21.15.203:6837/2424499644", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6836", "nonce": 2424499644, "type": "v2" }, { "addr": "172.21.15.203:6837", "nonce": 2424499644, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5632, "incarnation": 69, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 16 }, "gid_5836": { "addr": "172.21.15.203:6835/3960786803", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6834", "nonce": 3960786803, "type": "v2" }, { "addr": "172.21.15.203:6835", "nonce": 3960786803, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5836, "incarnation": 88, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:resolve", "state_seq": 31 } }, "last_failure": 0, "last_failure_osd_epoch": 34, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2021-01-06T22:21:32.723657+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 5836, "mds_1": 5632 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.91:6837/2972061575", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6835", "nonce": 2972061575, "type": "v2" }, { "addr": "172.21.15.91:6837", "nonce": 2972061575, "type": "v1" } ] }, "epoch": 87, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5384, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 2 } ] } 2021-01-06T22:21:35.129 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:35.263 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc05cb1700 1 Processor -- start 2021-01-06T22:21:35.264 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc05cb1700 1 -- start start 2021-01-06T22:21:35.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc05cb1700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbc000bb3f0 0x7fbc000ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:35.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc05cb1700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc000646b0 con 0x7fbc000a5e50 2021-01-06T22:21:35.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc05cb1700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc00152cc0 con 0x7fbc000bb7d0 2021-01-06T22:21:35.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc05cb1700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc000670c0 con 0x7fbc000bb3f0 2021-01-06T22:21:35.265 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc054b0700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fbc000bb7d0 0x7fbc0014eef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:52084/0 (socket says 172.21.15.203:52084) 2021-01-06T22:21:35.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc04caf700 1 --2- 172.21.15.203:0/3322010481 >> v2:172.21.15.91:3300/0 conn(0x7fbc000bb3f0 0x7fbc000ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:35.266 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc054b0700 1 -- 172.21.15.203:0/3322010481 learned_addr learned my addr 172.21.15.203:0/3322010481 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:21:35.268 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbbff7fe700 1 -- 172.21.15.203:0/3322010481 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684691963 0 0) 0x7fbc00152cc0 con 0x7fbc000bb7d0 2021-01-06T22:21:35.268 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbbff7fe700 1 -- 172.21.15.203:0/3322010481 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbe0003230 con 0x7fbc000bb7d0 2021-01-06T22:21:35.268 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc04caf700 1 -- 172.21.15.203:0/3322010481 >> v1:172.21.15.203:6789/0 conn(0x7fbc000bb7d0 legacy=0x7fbc0014eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:35.268 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc04caf700 1 -- 172.21.15.203:0/3322010481 >> v1:172.21.15.91:6789/0 conn(0x7fbc000a5e50 legacy=0x7fbc000669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:35.268 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc04caf700 1 -- 172.21.15.203:0/3322010481 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc00064450 con 0x7fbc000bb3f0 2021-01-06T22:21:35.269 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc04caf700 1 --2- 172.21.15.203:0/3322010481 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbc000bb3f0 0x7fbc000ba9d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fbbe800b980 tx=0x7fbbe8006de0).ready entity=mon.0 client_cookie=a332c889294ec731 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:35.269 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbbff7fe700 1 -- 172.21.15.203:0/3322010481 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80097b0 con 0x7fbc000bb3f0 2021-01-06T22:21:35.269 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbbff7fe700 1 -- 172.21.15.203:0/3322010481 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe8009910 con 0x7fbc000bb3f0 2021-01-06T22:21:35.269 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbbff7fe700 1 -- 172.21.15.203:0/3322010481 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8009c90 con 0x7fbc000bb3f0 2021-01-06T22:21:35.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc05cb1700 1 -- 172.21.15.203:0/3322010481 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbc000bb3f0 msgr2=0x7fbc000ba9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:35.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc05cb1700 1 --2- 172.21.15.203:0/3322010481 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbc000bb3f0 0x7fbc000ba9d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fbbe800b980 tx=0x7fbbe8006de0).stop 2021-01-06T22:21:35.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc05cb1700 1 -- 172.21.15.203:0/3322010481 shutdown_connections 2021-01-06T22:21:35.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc05cb1700 1 --2- 172.21.15.203:0/3322010481 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbc000bb3f0 0x7fbc000ba9d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:35.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc05cb1700 1 -- 172.21.15.203:0/3322010481 >> 172.21.15.203:0/3322010481 conn(0x7fbc00061530 msgr2=0x7fbc00063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:35.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc05cb1700 1 -- 172.21.15.203:0/3322010481 shutdown_connections 2021-01-06T22:21:35.271 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.260+0000 7fbc05cb1700 1 -- 172.21.15.203:0/3322010481 wait complete. 2021-01-06T22:21:35.272 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc05cb1700 1 Processor -- start 2021-01-06T22:21:35.272 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc05cb1700 1 -- start start 2021-01-06T22:21:35.274 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc05cb1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc000a5e50 0x7fbc001509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:35.274 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc05cb1700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbc000bb3f0 0x7fbc00162290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:35.275 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc05cb1700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbc000bb7d0 0x7fbc00151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:35.275 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc05cb1700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0009ef40 con 0x7fbc000bb7d0 2021-01-06T22:21:35.275 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc05cb1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc00162780 con 0x7fbc000a5e50 2021-01-06T22:21:35.277 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc05cb1700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc00150f50 con 0x7fbc000bb3f0 2021-01-06T22:21:35.277 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc04caf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc000a5e50 0x7fbc001509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:35.277 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc04caf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc000a5e50 0x7fbc001509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53492/0 (socket says 172.21.15.203:53492) 2021-01-06T22:21:35.277 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc04caf700 1 -- 172.21.15.203:0/902024834 learned_addr learned my addr 172.21.15.203:0/902024834 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:35.278 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbbfffff700 1 --2- 172.21.15.203:0/902024834 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbc000bb3f0 0x7fbc00162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:35.278 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc054b0700 1 --2- 172.21.15.203:0/902024834 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbc000bb7d0 0x7fbc00151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:35.278 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc04caf700 1 -- 172.21.15.203:0/902024834 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbc000bb3f0 msgr2=0x7fbc00162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:35.278 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc04caf700 1 --2- 172.21.15.203:0/902024834 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbc000bb3f0 0x7fbc00162290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:35.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc04caf700 1 -- 172.21.15.203:0/902024834 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbc000bb7d0 msgr2=0x7fbc00151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:35.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc04caf700 1 --2- 172.21.15.203:0/902024834 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbc000bb7d0 0x7fbc00151710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:35.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc04caf700 1 -- 172.21.15.203:0/902024834 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0009e760 con 0x7fbc000a5e50 2021-01-06T22:21:35.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc04caf700 1 --2- 172.21.15.203:0/902024834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc000a5e50 0x7fbc001509a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7fbbe800b130 tx=0x7fbbe8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:35.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbbfcff9700 1 -- 172.21.15.203:0/902024834 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8011cc0 con 0x7fbc000a5e50 2021-01-06T22:21:35.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbbfcff9700 1 -- 172.21.15.203:0/902024834 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe8011e20 con 0x7fbc000a5e50 2021-01-06T22:21:35.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc05cb1700 1 -- 172.21.15.203:0/902024834 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc0009ea00 con 0x7fbc000a5e50 2021-01-06T22:21:35.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbbfcff9700 1 -- 172.21.15.203:0/902024834 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe801d680 con 0x7fbc000a5e50 2021-01-06T22:21:35.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.264+0000 7fbc05cb1700 1 -- 172.21.15.203:0/902024834 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc000b8430 con 0x7fbc000a5e50 2021-01-06T22:21:35.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.268+0000 7fbc05cb1700 1 -- 172.21.15.203:0/902024834 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc000450a0 con 0x7fbc000a5e50 2021-01-06T22:21:35.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.268+0000 7fbbfcff9700 1 -- 172.21.15.203:0/902024834 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fbbe800f070 con 0x7fbc000a5e50 2021-01-06T22:21:35.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.268+0000 7fbbfcff9700 1 --2- 172.21.15.203:0/902024834 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbbe40563d0 0x7fbbe4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:35.282 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.268+0000 7fbbfcff9700 1 -- 172.21.15.203:0/902024834 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7019+0+0 (secure 0 0 0) 0x7fbbe8070e40 con 0x7fbc000a5e50 2021-01-06T22:21:35.282 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.268+0000 7fbbfffff700 1 --2- 172.21.15.203:0/902024834 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbbe40563d0 0x7fbbe4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:35.282 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.268+0000 7fbbfffff700 1 --2- 172.21.15.203:0/902024834 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbbe40563d0 0x7fbbe4058890 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fbc00150410 tx=0x7fbbf0007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:35.283 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.276+0000 7fbbfcff9700 1 -- 172.21.15.203:0/902024834 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fbbe8046390 con 0x7fbc000a5e50 2021-01-06T22:21:35.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.476+0000 7fbc05cb1700 1 -- 172.21.15.203:0/902024834 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc00057980 con 0x7fbc000a5e50 2021-01-06T22:21:35.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.480+0000 7fbbfcff9700 1 -- 172.21.15.203:0/902024834 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+3214 (secure 0 0 0) 0x7fbbe80499b0 con 0x7fbc000a5e50 2021-01-06T22:21:35.484 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:35.484 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:34.736804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:rejoin","state_seq":33,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:35.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.480+0000 7fbc05cb1700 1 -- 172.21.15.203:0/902024834 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbbe40563d0 msgr2=0x7fbbe4058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:35.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.480+0000 7fbc05cb1700 1 --2- 172.21.15.203:0/902024834 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbbe40563d0 0x7fbbe4058890 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fbc00150410 tx=0x7fbbf0007b80).stop 2021-01-06T22:21:35.486 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.480+0000 7fbc05cb1700 1 -- 172.21.15.203:0/902024834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc000a5e50 msgr2=0x7fbc001509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:35.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.480+0000 7fbc05cb1700 1 --2- 172.21.15.203:0/902024834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc000a5e50 0x7fbc001509a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7fbbe800b130 tx=0x7fbbe8016000).stop 2021-01-06T22:21:35.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.480+0000 7fbc05cb1700 1 -- 172.21.15.203:0/902024834 shutdown_connections 2021-01-06T22:21:35.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.480+0000 7fbc05cb1700 1 --2- 172.21.15.203:0/902024834 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fbbe40563d0 0x7fbbe4058890 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:35.487 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.480+0000 7fbc05cb1700 1 --2- 172.21.15.203:0/902024834 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbc000bb7d0 0x7fbc00151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:35.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.480+0000 7fbc05cb1700 1 --2- 172.21.15.203:0/902024834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc000a5e50 0x7fbc001509a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:35.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.480+0000 7fbc05cb1700 1 --2- 172.21.15.203:0/902024834 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbc000bb3f0 0x7fbc00162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:35.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.480+0000 7fbc05cb1700 1 -- 172.21.15.203:0/902024834 >> 172.21.15.203:0/902024834 conn(0x7fbc00061530 msgr2=0x7fbc001557f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:35.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.480+0000 7fbc05cb1700 1 -- 172.21.15.203:0/902024834 shutdown_connections 2021-01-06T22:21:35.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:35.480+0000 7fbc05cb1700 1 -- 172.21.15.203:0/902024834 wait complete. 2021-01-06T22:21:35.489 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 91 2021-01-06T22:21:35.499 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:21:36.468 INFO:tasks.ceph.mds.c.smithi203.stderr:2021-01-06T22:21:36.464+0000 7f5acf83b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2021-01-06T22:21:36.468 INFO:tasks.ceph.mds.c.smithi203.stderr:2021-01-06T22:21:36.464+0000 7f5acf83b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2021-01-06T22:21:37.500 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:37.630 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.624+0000 7fd6e67b7700 1 Processor -- start 2021-01-06T22:21:37.631 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.624+0000 7fd6e67b7700 1 -- start start 2021-01-06T22:21:37.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.624+0000 7fd6e67b7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd6e00bf420 0x7fd6e00bf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:37.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.624+0000 7fd6e67b7700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd6e0060d20 0x7fd6e014e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:37.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.624+0000 7fd6e67b7700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e0062ec0 con 0x7fd6e00606b0 2021-01-06T22:21:37.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.624+0000 7fd6e67b7700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e014ebb0 con 0x7fd6e0060d20 2021-01-06T22:21:37.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.624+0000 7fd6e67b7700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e00611f0 con 0x7fd6e00bf420 2021-01-06T22:21:37.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e57b5700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd6e00bf420 0x7fd6e00bf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:37.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e57b5700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd6e00bf420 0x7fd6e00bf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53506/0 (socket says 172.21.15.203:53506) 2021-01-06T22:21:37.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e57b5700 1 -- 172.21.15.203:0/3813454346 learned_addr learned my addr 172.21.15.203:0/3813454346 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:37.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e4fb4700 1 --2- 172.21.15.203:0/3813454346 >> v2:172.21.15.91:3300/0 conn(0x7fd6e0060d20 0x7fd6e014e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:37.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e57b5700 1 -- 172.21.15.203:0/3813454346 >> v2:172.21.15.91:3300/0 conn(0x7fd6e0060d20 msgr2=0x7fd6e014e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:37.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e57b5700 1 --2- 172.21.15.203:0/3813454346 >> v2:172.21.15.91:3300/0 conn(0x7fd6e0060d20 0x7fd6e014e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:37.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e57b5700 1 -- 172.21.15.203:0/3813454346 >> v1:172.21.15.91:6789/0 conn(0x7fd6e00606b0 legacy=0x7fd6e00c21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:37.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e57b5700 1 -- 172.21.15.203:0/3813454346 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e00626e0 con 0x7fd6e00bf420 2021-01-06T22:21:37.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e57b5700 1 --2- 172.21.15.203:0/3813454346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6e00bf420 0x7fd6e00bf800 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fd6e00bfd60 tx=0x7fd6dc006ea0).ready entity=mon.1 client_cookie=2cb8a3e869400be1 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:37.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6cffff700 1 -- 172.21.15.203:0/3813454346 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0098a0 con 0x7fd6e00bf420 2021-01-06T22:21:37.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6cffff700 1 -- 172.21.15.203:0/3813454346 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc009a00 con 0x7fd6e00bf420 2021-01-06T22:21:37.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6cffff700 1 -- 172.21.15.203:0/3813454346 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc009d80 con 0x7fd6e00bf420 2021-01-06T22:21:37.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e67b7700 1 -- 172.21.15.203:0/3813454346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6e00bf420 msgr2=0x7fd6e00bf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:37.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e67b7700 1 --2- 172.21.15.203:0/3813454346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6e00bf420 0x7fd6e00bf800 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fd6e00bfd60 tx=0x7fd6dc006ea0).stop 2021-01-06T22:21:37.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e67b7700 1 -- 172.21.15.203:0/3813454346 shutdown_connections 2021-01-06T22:21:37.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e67b7700 1 --2- 172.21.15.203:0/3813454346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6e00bf420 0x7fd6e00bf800 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:37.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e67b7700 1 --2- 172.21.15.203:0/3813454346 >> v2:172.21.15.91:3300/0 conn(0x7fd6e0060d20 0x7fd6e014e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:37.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e67b7700 1 -- 172.21.15.203:0/3813454346 >> 172.21.15.203:0/3813454346 conn(0x7fd6e00680f0 msgr2=0x7fd6e0056cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:37.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e67b7700 1 -- 172.21.15.203:0/3813454346 shutdown_connections 2021-01-06T22:21:37.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e67b7700 1 -- 172.21.15.203:0/3813454346 wait complete. 2021-01-06T22:21:37.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e67b7700 1 Processor -- start 2021-01-06T22:21:37.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.628+0000 7fd6e67b7700 1 -- start start 2021-01-06T22:21:37.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e67b7700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6e00606b0 0x7fd6e00a7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:37.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e67b7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6e0060d20 0x7fd6e014ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:37.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e67b7700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6e00bf420 0x7fd6e0154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:37.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e67b7700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e0154cd0 con 0x7fd6e00bf420 2021-01-06T22:21:37.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e67b7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e0161530 con 0x7fd6e0060d20 2021-01-06T22:21:37.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e67b7700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e01544f0 con 0x7fd6e00606b0 2021-01-06T22:21:37.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e4fb4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6e0060d20 0x7fd6e014ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:37.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e4fb4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6e0060d20 0x7fd6e014ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53514/0 (socket says 172.21.15.203:53514) 2021-01-06T22:21:37.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e4fb4700 1 -- 172.21.15.203:0/3261090831 learned_addr learned my addr 172.21.15.203:0/3261090831 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:37.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e57b5700 1 --2- 172.21.15.203:0/3261090831 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6e00606b0 0x7fd6e00a7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:37.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e5fb6700 1 --2- 172.21.15.203:0/3261090831 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6e00bf420 0x7fd6e0154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:37.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e4fb4700 1 -- 172.21.15.203:0/3261090831 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6e00606b0 msgr2=0x7fd6e00a7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:37.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e4fb4700 1 --2- 172.21.15.203:0/3261090831 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6e00606b0 0x7fd6e00a7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:37.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e4fb4700 1 -- 172.21.15.203:0/3261090831 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6e00bf420 msgr2=0x7fd6e0154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:37.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e4fb4700 1 --2- 172.21.15.203:0/3261090831 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6e00bf420 0x7fd6e0154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:37.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e4fb4700 1 -- 172.21.15.203:0/3261090831 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e0154790 con 0x7fd6e0060d20 2021-01-06T22:21:37.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e4fb4700 1 --2- 172.21.15.203:0/3261090831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6e0060d20 0x7fd6e014ff90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7fd6d000ebd0 tx=0x7fd6d0006de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:37.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6cd7fa700 1 -- 172.21.15.203:0/3261090831 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d000c950 con 0x7fd6e0060d20 2021-01-06T22:21:37.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6cd7fa700 1 -- 172.21.15.203:0/3261090831 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d000cab0 con 0x7fd6e0060d20 2021-01-06T22:21:37.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6cd7fa700 1 -- 172.21.15.203:0/3261090831 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d000ce30 con 0x7fd6e0060d20 2021-01-06T22:21:37.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e67b7700 1 -- 172.21.15.203:0/3261090831 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e0154a30 con 0x7fd6e0060d20 2021-01-06T22:21:37.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e67b7700 1 -- 172.21.15.203:0/3261090831 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e01554d0 con 0x7fd6e0060d20 2021-01-06T22:21:37.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6e67b7700 1 -- 172.21.15.203:0/3261090831 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e00a4a50 con 0x7fd6e0060d20 2021-01-06T22:21:37.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.632+0000 7fd6cd7fa700 1 -- 172.21.15.203:0/3261090831 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fd6d0012600 con 0x7fd6e0060d20 2021-01-06T22:21:37.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.636+0000 7fd6cd7fa700 1 --2- 172.21.15.203:0/3261090831 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd6c00563d0 0x7fd6c0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:37.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.636+0000 7fd6cd7fa700 1 -- 172.21.15.203:0/3261090831 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7019+0+0 (secure 0 0 0) 0x7fd6d006fd00 con 0x7fd6e0060d20 2021-01-06T22:21:37.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.636+0000 7fd6e57b5700 1 --2- 172.21.15.203:0/3261090831 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd6c00563d0 0x7fd6c0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:37.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.636+0000 7fd6e57b5700 1 --2- 172.21.15.203:0/3261090831 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd6c00563d0 0x7fd6c0058890 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fd6dc006ee0 tx=0x7fd6dc016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:37.649 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.640+0000 7fd6cd7fa700 1 -- 172.21.15.203:0/3261090831 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fd6d0073030 con 0x7fd6e0060d20 2021-01-06T22:21:37.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.896+0000 7fd6e67b7700 1 -- 172.21.15.203:0/3261090831 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6e00450a0 con 0x7fd6e0060d20 2021-01-06T22:21:37.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.896+0000 7fd6cd7fa700 1 -- 172.21.15.203:0/3261090831 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+3214 (secure 0 0 0) 0x7fd6d00451d0 con 0x7fd6e0060d20 2021-01-06T22:21:37.901 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:37.901 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":92,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:35.746075+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:37.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.896+0000 7fd6e67b7700 1 -- 172.21.15.203:0/3261090831 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd6c00563d0 msgr2=0x7fd6c0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:37.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.896+0000 7fd6e67b7700 1 --2- 172.21.15.203:0/3261090831 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd6c00563d0 0x7fd6c0058890 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fd6dc006ee0 tx=0x7fd6dc016000).stop 2021-01-06T22:21:37.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.896+0000 7fd6e67b7700 1 -- 172.21.15.203:0/3261090831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6e0060d20 msgr2=0x7fd6e014ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:37.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.896+0000 7fd6e67b7700 1 --2- 172.21.15.203:0/3261090831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6e0060d20 0x7fd6e014ff90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7fd6d000ebd0 tx=0x7fd6d0006de0).stop 2021-01-06T22:21:37.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.896+0000 7fd6e67b7700 1 -- 172.21.15.203:0/3261090831 shutdown_connections 2021-01-06T22:21:37.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.896+0000 7fd6e67b7700 1 --2- 172.21.15.203:0/3261090831 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd6c00563d0 0x7fd6c0058890 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:37.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.896+0000 7fd6e67b7700 1 --2- 172.21.15.203:0/3261090831 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6e00bf420 0x7fd6e0154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:37.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.896+0000 7fd6e67b7700 1 --2- 172.21.15.203:0/3261090831 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6e00606b0 0x7fd6e00a7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:37.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.896+0000 7fd6e67b7700 1 --2- 172.21.15.203:0/3261090831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6e0060d20 0x7fd6e014ff90 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:37.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.896+0000 7fd6e67b7700 1 -- 172.21.15.203:0/3261090831 >> 172.21.15.203:0/3261090831 conn(0x7fd6e00680f0 msgr2=0x7fd6e014f620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:37.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.896+0000 7fd6e67b7700 1 -- 172.21.15.203:0/3261090831 shutdown_connections 2021-01-06T22:21:37.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:37.896+0000 7fd6e67b7700 1 -- 172.21.15.203:0/3261090831 wait complete. 2021-01-06T22:21:37.907 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 92 2021-01-06T22:21:37.916 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=0, replacing gid=4724 2021-01-06T22:21:37.916 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 0 secs before reviving mds.b 2021-01-06T22:21:37.917 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2021-01-06T22:21:37.917 INFO:tasks.ceph.mds.b:Restarting daemon 2021-01-06T22:21:37.917 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-01-06T22:21:37.920 INFO:tasks.ceph.mds.b:Started 2021-01-06T22:21:37.921 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:38.000 INFO:tasks.ceph.mds.b.smithi091.stdout:starting mds.b at 2021-01-06T22:21:38.046 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.040+0000 7fd79d3ed700 1 Processor -- start 2021-01-06T22:21:38.047 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.040+0000 7fd79d3ed700 1 -- start start 2021-01-06T22:21:38.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd79d3ed700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd798148080 0x7fd798068110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:38.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd79d3ed700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7980bba10 con 0x7fd798067340 2021-01-06T22:21:38.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd79d3ed700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7980bb230 con 0x7fd798067720 2021-01-06T22:21:38.048 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd79d3ed700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7980bb4d0 con 0x7fd798148080 2021-01-06T22:21:38.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd7977fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fd798067720 0x7fd7981532f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:52116/0 (socket says 172.21.15.203:52116) 2021-01-06T22:21:38.049 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd7977fe700 1 -- 172.21.15.203:0/1642470172 learned_addr learned my addr 172.21.15.203:0/1642470172 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:21:38.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd79cbec700 1 --2- 172.21.15.203:0/1642470172 >> v2:172.21.15.91:3300/0 conn(0x7fd798148080 0x7fd798068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:38.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd796ffd700 1 -- 172.21.15.203:0/1642470172 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842748593 0 0) 0x7fd7980bb230 con 0x7fd798067720 2021-01-06T22:21:38.050 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd796ffd700 1 -- 172.21.15.203:0/1642470172 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd784003230 con 0x7fd798067720 2021-01-06T22:21:38.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd796ffd700 1 -- 172.21.15.203:0/1642470172 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1904623162 0 0) 0x7fd784003230 con 0x7fd798067720 2021-01-06T22:21:38.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd79cbec700 1 -- 172.21.15.203:0/1642470172 >> v1:172.21.15.203:6789/0 conn(0x7fd798067720 legacy=0x7fd7981532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:38.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd79cbec700 1 -- 172.21.15.203:0/1642470172 >> v1:172.21.15.91:6789/0 conn(0x7fd798067340 legacy=0x7fd798147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:38.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd79cbec700 1 -- 172.21.15.203:0/1642470172 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7980bb770 con 0x7fd798148080 2021-01-06T22:21:38.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd79cbec700 1 --2- 172.21.15.203:0/1642470172 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd798148080 0x7fd798068110 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fd790009c10 tx=0x7fd7900091d0).ready entity=mon.0 client_cookie=2dc719ba1db85005 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:38.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd796ffd700 1 -- 172.21.15.203:0/1642470172 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0014e0 con 0x7fd798148080 2021-01-06T22:21:38.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd796ffd700 1 -- 172.21.15.203:0/1642470172 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79000b710 con 0x7fd798148080 2021-01-06T22:21:38.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd796ffd700 1 -- 172.21.15.203:0/1642470172 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79000ba90 con 0x7fd798148080 2021-01-06T22:21:38.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd79d3ed700 1 -- 172.21.15.203:0/1642470172 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd798148080 msgr2=0x7fd798068110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:38.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd79d3ed700 1 --2- 172.21.15.203:0/1642470172 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd798148080 0x7fd798068110 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fd790009c10 tx=0x7fd7900091d0).stop 2021-01-06T22:21:38.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd79d3ed700 1 -- 172.21.15.203:0/1642470172 shutdown_connections 2021-01-06T22:21:38.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd79d3ed700 1 --2- 172.21.15.203:0/1642470172 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd798148080 0x7fd798068110 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:38.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd79d3ed700 1 -- 172.21.15.203:0/1642470172 >> 172.21.15.203:0/1642470172 conn(0x7fd7980667b0 msgr2=0x7fd7980c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:38.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd79d3ed700 1 -- 172.21.15.203:0/1642470172 shutdown_connections 2021-01-06T22:21:38.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.044+0000 7fd79d3ed700 1 -- 172.21.15.203:0/1642470172 wait complete. 2021-01-06T22:21:38.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79d3ed700 1 Processor -- start 2021-01-06T22:21:38.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79d3ed700 1 -- start start 2021-01-06T22:21:38.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79d3ed700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd798067340 0x7fd7981466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:38.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79d3ed700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd798067720 0x7fd79815dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:38.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79d3ed700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd798148080 0x7fd79815e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:38.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79d3ed700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7980b8870 con 0x7fd798067720 2021-01-06T22:21:38.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79d3ed700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79814e350 con 0x7fd798148080 2021-01-06T22:21:38.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79d3ed700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7980b9790 con 0x7fd798067340 2021-01-06T22:21:38.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79cbec700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd798148080 0x7fd79815e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:38.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79cbec700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd798148080 0x7fd79815e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53530/0 (socket says 172.21.15.203:53530) 2021-01-06T22:21:38.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79cbec700 1 -- 172.21.15.203:0/1619602794 learned_addr learned my addr 172.21.15.203:0/1619602794 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:38.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd797fff700 1 --2- 172.21.15.203:0/1619602794 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd798067340 0x7fd7981466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:38.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd7977fe700 1 --2- 172.21.15.203:0/1619602794 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd798067720 0x7fd79815dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:38.061 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79cbec700 1 -- 172.21.15.203:0/1619602794 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd798067340 msgr2=0x7fd7981466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:38.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79cbec700 1 --2- 172.21.15.203:0/1619602794 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd798067340 0x7fd7981466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:38.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79cbec700 1 -- 172.21.15.203:0/1619602794 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd798067720 msgr2=0x7fd79815dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:38.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79cbec700 1 --2- 172.21.15.203:0/1619602794 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd798067720 0x7fd79815dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:38.062 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79cbec700 1 -- 172.21.15.203:0/1619602794 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79814e580 con 0x7fd798148080 2021-01-06T22:21:38.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd7977fe700 1 --2- 172.21.15.203:0/1619602794 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd798067720 0x7fd79815dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:21:38.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79cbec700 1 --2- 172.21.15.203:0/1619602794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd798148080 0x7fd79815e260 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fd790012ba0 tx=0x7fd7980b8750).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:38.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd77ffff700 1 -- 172.21.15.203:0/1619602794 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd790012e40 con 0x7fd798148080 2021-01-06T22:21:38.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd77ffff700 1 -- 172.21.15.203:0/1619602794 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79001b590 con 0x7fd798148080 2021-01-06T22:21:38.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd77ffff700 1 -- 172.21.15.203:0/1619602794 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79001b930 con 0x7fd798148080 2021-01-06T22:21:38.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79d3ed700 1 -- 172.21.15.203:0/1619602794 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79814e820 con 0x7fd798148080 2021-01-06T22:21:38.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.048+0000 7fd79d3ed700 1 -- 172.21.15.203:0/1619602794 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79814eac0 con 0x7fd798148080 2021-01-06T22:21:38.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.052+0000 7fd79d3ed700 1 -- 172.21.15.203:0/1619602794 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7980450a0 con 0x7fd798148080 2021-01-06T22:21:38.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.052+0000 7fd77ffff700 1 -- 172.21.15.203:0/1619602794 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fd79000f070 con 0x7fd798148080 2021-01-06T22:21:38.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.052+0000 7fd77ffff700 1 --2- 172.21.15.203:0/1619602794 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd7780563d0 0x7fd778058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:38.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.052+0000 7fd797fff700 1 --2- 172.21.15.203:0/1619602794 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd7780563d0 0x7fd778058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:38.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.052+0000 7fd77ffff700 1 -- 172.21.15.203:0/1619602794 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7019+0+0 (secure 0 0 0) 0x7fd7900712f0 con 0x7fd798148080 2021-01-06T22:21:38.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.052+0000 7fd797fff700 1 --2- 172.21.15.203:0/1619602794 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd7780563d0 0x7fd778058890 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fd788003800 tx=0x7fd788007a80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:38.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.060+0000 7fd77ffff700 1 -- 172.21.15.203:0/1619602794 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fd7900467c0 con 0x7fd798148080 2021-01-06T22:21:38.276 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.272+0000 7fd79d3ed700 1 -- 172.21.15.203:0/1619602794 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd79805ca60 con 0x7fd798148080 2021-01-06T22:21:38.276 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.272+0000 7fd77ffff700 1 -- 172.21.15.203:0/1619602794 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+3214 (secure 0 0 0) 0x7fd790049de0 con 0x7fd798148080 2021-01-06T22:21:38.277 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:38.277 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":92,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:35.746075+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:38.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.276+0000 7fd79d3ed700 1 -- 172.21.15.203:0/1619602794 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd7780563d0 msgr2=0x7fd778058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:38.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.276+0000 7fd79d3ed700 1 --2- 172.21.15.203:0/1619602794 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd7780563d0 0x7fd778058890 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fd788003800 tx=0x7fd788007a80).stop 2021-01-06T22:21:38.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.276+0000 7fd79d3ed700 1 -- 172.21.15.203:0/1619602794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd798148080 msgr2=0x7fd79815e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:38.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.276+0000 7fd79d3ed700 1 --2- 172.21.15.203:0/1619602794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd798148080 0x7fd79815e260 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fd790012ba0 tx=0x7fd7980b8750).stop 2021-01-06T22:21:38.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.276+0000 7fd79d3ed700 1 -- 172.21.15.203:0/1619602794 shutdown_connections 2021-01-06T22:21:38.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.276+0000 7fd79d3ed700 1 --2- 172.21.15.203:0/1619602794 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd7780563d0 0x7fd778058890 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:38.282 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.276+0000 7fd79d3ed700 1 --2- 172.21.15.203:0/1619602794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd798148080 0x7fd79815e260 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:38.282 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.276+0000 7fd79d3ed700 1 --2- 172.21.15.203:0/1619602794 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd798067340 0x7fd7981466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:38.282 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.276+0000 7fd79d3ed700 1 --2- 172.21.15.203:0/1619602794 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd798067720 0x7fd79815dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:38.282 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.276+0000 7fd79d3ed700 1 -- 172.21.15.203:0/1619602794 >> 172.21.15.203:0/1619602794 conn(0x7fd7980667b0 msgr2=0x7fd798067e60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:38.282 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.276+0000 7fd79d3ed700 1 -- 172.21.15.203:0/1619602794 shutdown_connections 2021-01-06T22:21:38.283 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:38.276+0000 7fd79d3ed700 1 -- 172.21.15.203:0/1619602794 wait complete. 2021-01-06T22:21:38.283 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 92 2021-01-06T22:21:38.297 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is in active, standby or standby-replay 2021-01-06T22:21:40.298 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:21:40.432 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d30e27700 1 Processor -- start 2021-01-06T22:21:40.433 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d30e27700 1 -- start start 2021-01-06T22:21:40.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d30e27700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d2c064380 0x7f7d2c0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:40.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d30e27700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7d2c14e310 0x7f7d2c1471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:40.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d30e27700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2c0bb2d0 con 0x7f7d2c064760 2021-01-06T22:21:40.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d30e27700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2c14a560 con 0x7f7d2c14e310 2021-01-06T22:21:40.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d30e27700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c14a800 con 0x7f7d2c064380 2021-01-06T22:21:40.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d2bfff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f7d2c064760 0x7f7d2c0bb730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:52130/0 (socket says 172.21.15.203:52130) 2021-01-06T22:21:40.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d2bfff700 1 -- 172.21.15.203:0/878040579 learned_addr learned my addr 172.21.15.203:0/878040579 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:21:40.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d22ffd700 1 --2- 172.21.15.203:0/878040579 >> v2:172.21.15.203:3300/0 conn(0x7f7d2c14e310 0x7f7d2c1471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:40.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d2b7fe700 1 --2- 172.21.15.203:0/878040579 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d2c064380 0x7f7d2c0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:40.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d2affd700 1 -- 172.21.15.203:0/878040579 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443275875 0 0) 0x7f7d2c0bb2d0 con 0x7f7d2c064760 2021-01-06T22:21:40.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d2affd700 1 -- 172.21.15.203:0/878040579 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d0c003230 con 0x7f7d2c064760 2021-01-06T22:21:40.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d22ffd700 1 -- 172.21.15.203:0/878040579 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d2c064380 msgr2=0x7f7d2c0c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:21:40.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d22ffd700 1 --2- 172.21.15.203:0/878040579 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d2c064380 0x7f7d2c0c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:40.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d22ffd700 1 -- 172.21.15.203:0/878040579 >> v1:172.21.15.203:6789/0 conn(0x7f7d2c064760 legacy=0x7f7d2c0bb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:40.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d22ffd700 1 -- 172.21.15.203:0/878040579 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c14aaa0 con 0x7f7d2c14e310 2021-01-06T22:21:40.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.428+0000 7f7d22ffd700 1 --2- 172.21.15.203:0/878040579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d2c14e310 0x7f7d2c1471b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7f7d1400b980 tx=0x7f7d14006de0).ready entity=mon.1 client_cookie=c2c0977a59328f85 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:40.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d2affd700 1 -- 172.21.15.203:0/878040579 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d140096d0 con 0x7f7d2c14e310 2021-01-06T22:21:40.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d2affd700 1 -- 172.21.15.203:0/878040579 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d14009830 con 0x7f7d2c14e310 2021-01-06T22:21:40.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d2affd700 1 -- 172.21.15.203:0/878040579 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14009bb0 con 0x7f7d2c14e310 2021-01-06T22:21:40.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d30e27700 1 -- 172.21.15.203:0/878040579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d2c14e310 msgr2=0x7f7d2c1471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:40.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d30e27700 1 --2- 172.21.15.203:0/878040579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d2c14e310 0x7f7d2c1471b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7f7d1400b980 tx=0x7f7d14006de0).stop 2021-01-06T22:21:40.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d30e27700 1 -- 172.21.15.203:0/878040579 shutdown_connections 2021-01-06T22:21:40.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d30e27700 1 --2- 172.21.15.203:0/878040579 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d2c064380 0x7f7d2c0c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:40.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d30e27700 1 --2- 172.21.15.203:0/878040579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d2c14e310 0x7f7d2c1471b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:40.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d30e27700 1 -- 172.21.15.203:0/878040579 >> 172.21.15.203:0/878040579 conn(0x7f7d2c0667b0 msgr2=0x7f7d2c0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:40.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d30e27700 1 -- 172.21.15.203:0/878040579 shutdown_connections 2021-01-06T22:21:40.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d30e27700 1 -- 172.21.15.203:0/878040579 wait complete. 2021-01-06T22:21:40.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d30e27700 1 Processor -- start 2021-01-06T22:21:40.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d30e27700 1 -- start start 2021-01-06T22:21:40.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d30e27700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d2c064380 0x7f7d2c14c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:40.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d30e27700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d2c064760 0x7f7d2c160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:40.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d30e27700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d2c0b1820 0x7f7d2c1610e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:40.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d30e27700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c14bec0 con 0x7f7d2c064760 2021-01-06T22:21:40.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d30e27700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c161990 con 0x7f7d2c0b1820 2021-01-06T22:21:40.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.432+0000 7f7d30e27700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c14d990 con 0x7f7d2c064380 2021-01-06T22:21:40.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d2bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d2c0b1820 0x7f7d2c1610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:40.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d2bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d2c0b1820 0x7f7d2c1610e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53544/0 (socket says 172.21.15.203:53544) 2021-01-06T22:21:40.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d2bfff700 1 -- 172.21.15.203:0/2184278675 learned_addr learned my addr 172.21.15.203:0/2184278675 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:21:40.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d22ffd700 1 --2- 172.21.15.203:0/2184278675 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d2c064760 0x7f7d2c160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:40.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d2b7fe700 1 --2- 172.21.15.203:0/2184278675 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d2c064380 0x7f7d2c14c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:40.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d2bfff700 1 -- 172.21.15.203:0/2184278675 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d2c064380 msgr2=0x7f7d2c14c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:40.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d2bfff700 1 --2- 172.21.15.203:0/2184278675 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d2c064380 0x7f7d2c14c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:40.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d2bfff700 1 -- 172.21.15.203:0/2184278675 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d2c064760 msgr2=0x7f7d2c160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:40.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d2bfff700 1 --2- 172.21.15.203:0/2184278675 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d2c064760 0x7f7d2c160bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:40.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d2bfff700 1 -- 172.21.15.203:0/2184278675 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c14c0a0 con 0x7f7d2c0b1820 2021-01-06T22:21:40.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d2bfff700 1 --2- 172.21.15.203:0/2184278675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d2c0b1820 0x7f7d2c1610e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f7d2400ca90 tx=0x7f7d24007dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:40.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d23fff700 1 -- 172.21.15.203:0/2184278675 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24005b70 con 0x7f7d2c0b1820 2021-01-06T22:21:40.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d23fff700 1 -- 172.21.15.203:0/2184278675 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d24005cd0 con 0x7f7d2c0b1820 2021-01-06T22:21:40.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d30e27700 1 -- 172.21.15.203:0/2184278675 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2c14dbc0 con 0x7f7d2c0b1820 2021-01-06T22:21:40.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d23fff700 1 -- 172.21.15.203:0/2184278675 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24007320 con 0x7f7d2c0b1820 2021-01-06T22:21:40.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d30e27700 1 -- 172.21.15.203:0/2184278675 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2c14c300 con 0x7f7d2c0b1820 2021-01-06T22:21:40.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d30e27700 1 -- 172.21.15.203:0/2184278675 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2c0450a0 con 0x7f7d2c0b1820 2021-01-06T22:21:40.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d23fff700 1 -- 172.21.15.203:0/2184278675 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f7d24010030 con 0x7f7d2c0b1820 2021-01-06T22:21:40.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d23fff700 1 --2- 172.21.15.203:0/2184278675 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7d100563d0 0x7f7d10058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:21:40.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d23fff700 1 -- 172.21.15.203:0/2184278675 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7019+0+0 (secure 0 0 0) 0x7f7d2406f860 con 0x7f7d2c0b1820 2021-01-06T22:21:40.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d2b7fe700 1 --2- 172.21.15.203:0/2184278675 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7d100563d0 0x7f7d10058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:21:40.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.436+0000 7f7d2b7fe700 1 --2- 172.21.15.203:0/2184278675 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7d100563d0 0x7f7d10058890 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f7d1c00bb80 tx=0x7f7d1c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:21:40.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.444+0000 7f7d23fff700 1 -- 172.21.15.203:0/2184278675 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f7d24044d30 con 0x7f7d2c0b1820 2021-01-06T22:21:40.675 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.668+0000 7f7d30e27700 1 -- 172.21.15.203:0/2184278675 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d2c05ca60 con 0x7f7d2c0b1820 2021-01-06T22:21:40.676 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.672+0000 7f7d23fff700 1 -- 172.21.15.203:0/2184278675 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+3567 (secure 0 0 0) 0x7f7d24048350 con 0x7f7d2c0b1820 2021-01-06T22:21:40.677 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:21:40.677 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":5874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":93}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:35.746075+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:21:40.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.676+0000 7f7d30e27700 1 -- 172.21.15.203:0/2184278675 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7d100563d0 msgr2=0x7f7d10058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:40.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.676+0000 7f7d30e27700 1 --2- 172.21.15.203:0/2184278675 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7d100563d0 0x7f7d10058890 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f7d1c00bb80 tx=0x7f7d1c008040).stop 2021-01-06T22:21:40.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.676+0000 7f7d30e27700 1 -- 172.21.15.203:0/2184278675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d2c0b1820 msgr2=0x7f7d2c1610e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:21:40.680 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.676+0000 7f7d30e27700 1 --2- 172.21.15.203:0/2184278675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d2c0b1820 0x7f7d2c1610e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f7d2400ca90 tx=0x7f7d24007dd0).stop 2021-01-06T22:21:40.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.676+0000 7f7d30e27700 1 -- 172.21.15.203:0/2184278675 shutdown_connections 2021-01-06T22:21:40.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.676+0000 7f7d30e27700 1 --2- 172.21.15.203:0/2184278675 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f7d100563d0 0x7f7d10058890 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:40.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.676+0000 7f7d30e27700 1 --2- 172.21.15.203:0/2184278675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d2c0b1820 0x7f7d2c1610e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:40.681 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.676+0000 7f7d30e27700 1 --2- 172.21.15.203:0/2184278675 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d2c064380 0x7f7d2c14c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:40.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.676+0000 7f7d30e27700 1 --2- 172.21.15.203:0/2184278675 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d2c064760 0x7f7d2c160bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:21:40.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.676+0000 7f7d30e27700 1 -- 172.21.15.203:0/2184278675 >> 172.21.15.203:0/2184278675 conn(0x7f7d2c0667b0 msgr2=0x7f7d2c0bf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:21:40.682 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.676+0000 7f7d30e27700 1 -- 172.21.15.203:0/2184278675 shutdown_connections 2021-01-06T22:21:40.683 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:21:40.676+0000 7f7d30e27700 1 -- 172.21.15.203:0/2184278675 wait complete. 2021-01-06T22:21:40.683 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 93 2021-01-06T22:21:40.695 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2021-01-06T22:21:40.696 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 78 secs before thrashing 2021-01-06T22:21:51.919 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:21:51.921 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:22:21.955 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:22:21.962 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:22:25.425 INFO:tasks.workunit.client.0.smithi203.stdout:fs setup took 1012 secs 2021-01-06T22:22:27.989 INFO:tasks.workunit.client.0.smithi203.stdout:Syncing()...2 sec 2021-01-06T22:22:27.990 INFO:tasks.workunit.client.0.smithi203.stdout:Starting Actual Benchmark At: Wed Jan 6 22:22:27 2021 2021-01-06T22:22:27.990 INFO:tasks.workunit.client.0.smithi203.stdout: 2021-01-06T22:22:52.144 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:22:52.175 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:22:58.696 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:22:58.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c5dbc700 1 Processor -- start 2021-01-06T22:22:58.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c5dbc700 1 -- start start 2021-01-06T22:22:58.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c5dbc700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f33c00bcbf0 0x7f33c0152be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:22:58.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c5dbc700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33c00bcfd0 0x7f33c014ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:22:58.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c5dbc700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c00b6930 con 0x7f33c00b4fa0 2021-01-06T22:22:58.955 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c5dbc700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c01530d0 con 0x7f33c00bcbf0 2021-01-06T22:22:58.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c5dbc700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c00b6bd0 con 0x7f33c00bcfd0 2021-01-06T22:22:58.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c55bb700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33c00bcfd0 0x7f33c014ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:22:58.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c55bb700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33c00bcfd0 0x7f33c014ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:59624/0 (socket says 172.21.15.203:59624) 2021-01-06T22:22:58.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c55bb700 1 -- 172.21.15.203:0/1286611914 learned_addr learned my addr 172.21.15.203:0/1286611914 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:22:58.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c4dba700 1 --2- 172.21.15.203:0/1286611914 >> v2:172.21.15.203:3300/0 conn(0x7f33c00bcbf0 0x7f33c0152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:22:58.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c55bb700 1 -- 172.21.15.203:0/1286611914 >> v2:172.21.15.203:3300/0 conn(0x7f33c00bcbf0 msgr2=0x7f33c0152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:22:58.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c55bb700 1 --2- 172.21.15.203:0/1286611914 >> v2:172.21.15.203:3300/0 conn(0x7f33c00bcbf0 0x7f33c0152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:22:58.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c55bb700 1 -- 172.21.15.203:0/1286611914 >> v1:172.21.15.91:6789/0 conn(0x7f33c00b4fa0 legacy=0x7f33c0063b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:22:58.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c55bb700 1 -- 172.21.15.203:0/1286611914 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c00b6e70 con 0x7f33c00bcfd0 2021-01-06T22:22:58.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c4dba700 1 --2- 172.21.15.203:0/1286611914 >> v2:172.21.15.203:3300/0 conn(0x7f33c00bcbf0 0x7f33c0152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:22:58.958 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.950+0000 7f33c55bb700 1 --2- 172.21.15.203:0/1286611914 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33c00bcfd0 0x7f33c014ecf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f33b000b9e0 tx=0x7f33b0006ea0).ready entity=mon.2 client_cookie=1b7207c248ef2387 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:22:58.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.954+0000 7f33bf7fe700 1 -- 172.21.15.203:0/1286611914 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0009710 con 0x7f33c00bcfd0 2021-01-06T22:22:58.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.958+0000 7f33bf7fe700 1 -- 172.21.15.203:0/1286611914 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b0009870 con 0x7f33c00bcfd0 2021-01-06T22:22:58.960 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.958+0000 7f33bf7fe700 1 -- 172.21.15.203:0/1286611914 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0009bf0 con 0x7f33c00bcfd0 2021-01-06T22:22:58.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.962+0000 7f33c5dbc700 1 -- 172.21.15.203:0/1286611914 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33c00bcfd0 msgr2=0x7f33c014ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:22:58.965 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.962+0000 7f33c5dbc700 1 --2- 172.21.15.203:0/1286611914 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33c00bcfd0 0x7f33c014ecf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f33b000b9e0 tx=0x7f33b0006ea0).stop 2021-01-06T22:22:58.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.962+0000 7f33c5dbc700 1 -- 172.21.15.203:0/1286611914 shutdown_connections 2021-01-06T22:22:58.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.962+0000 7f33c5dbc700 1 --2- 172.21.15.203:0/1286611914 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33c00bcfd0 0x7f33c014ecf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:22:58.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.962+0000 7f33c5dbc700 1 --2- 172.21.15.203:0/1286611914 >> v2:172.21.15.203:3300/0 conn(0x7f33c00bcbf0 0x7f33c0152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:22:58.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.962+0000 7f33c5dbc700 1 -- 172.21.15.203:0/1286611914 >> 172.21.15.203:0/1286611914 conn(0x7f33c0056cd0 msgr2=0x7f33c00626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:22:58.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.970+0000 7f33c5dbc700 1 -- 172.21.15.203:0/1286611914 shutdown_connections 2021-01-06T22:22:58.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.970+0000 7f33c5dbc700 1 -- 172.21.15.203:0/1286611914 wait complete. 2021-01-06T22:22:58.975 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33c5dbc700 1 Processor -- start 2021-01-06T22:22:58.976 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33c5dbc700 1 -- start start 2021-01-06T22:22:58.977 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33c5dbc700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33c00bcbf0 0x7f33c00b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:22:58.977 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33c5dbc700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c00bcfd0 0x7f33c014f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:22:58.977 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33c5dbc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33c00b7370 0x7f33c0163080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:22:58.978 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33c4dba700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33c00bcbf0 0x7f33c00b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:22:58.978 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33bffff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c00bcfd0 0x7f33c014f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:22:58.978 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33c4dba700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33c00bcbf0 0x7f33c00b8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:59632/0 (socket says 172.21.15.203:59632) 2021-01-06T22:22:58.979 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33bffff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c00bcfd0 0x7f33c014f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:34168/0 (socket says 172.21.15.203:34168) 2021-01-06T22:22:58.979 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33c4dba700 1 -- 172.21.15.203:0/2709877747 learned_addr learned my addr 172.21.15.203:0/2709877747 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:22:58.979 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33c55bb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33c00b7370 0x7f33c0163080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:22:58.979 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33c5dbc700 1 -- 172.21.15.203:0/2709877747 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c014d8f0 con 0x7f33c00bcfd0 2021-01-06T22:22:58.980 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33c5dbc700 1 -- 172.21.15.203:0/2709877747 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c00b9380 con 0x7f33c00b7370 2021-01-06T22:22:58.980 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33c5dbc700 1 -- 172.21.15.203:0/2709877747 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c014c9d0 con 0x7f33c00bcbf0 2021-01-06T22:22:58.981 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33bffff700 1 -- 172.21.15.203:0/2709877747 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33c00bcbf0 msgr2=0x7f33c00b8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:22:58.981 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33bffff700 1 --2- 172.21.15.203:0/2709877747 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33c00bcbf0 0x7f33c00b8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:22:58.981 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33bffff700 1 -- 172.21.15.203:0/2709877747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33c00b7370 msgr2=0x7f33c0163080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:22:58.981 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.974+0000 7f33bffff700 1 --2- 172.21.15.203:0/2709877747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33c00b7370 0x7f33c0163080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:22:58.982 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.978+0000 7f33bffff700 1 -- 172.21.15.203:0/2709877747 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c014cc70 con 0x7f33c00bcfd0 2021-01-06T22:22:58.982 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.978+0000 7f33c55bb700 1 --2- 172.21.15.203:0/2709877747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33c00b7370 0x7f33c0163080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:22:58.982 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.978+0000 7f33bffff700 1 --2- 172.21.15.203:0/2709877747 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c00bcfd0 0x7f33c014f0e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f33b400c7b0 tx=0x7f33b4007b60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:22:58.982 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.978+0000 7f33bcff9700 1 -- 172.21.15.203:0/2709877747 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b400e560 con 0x7f33c00bcfd0 2021-01-06T22:22:58.983 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.978+0000 7f33bcff9700 1 -- 172.21.15.203:0/2709877747 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b400e6c0 con 0x7f33c00bcfd0 2021-01-06T22:22:58.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.978+0000 7f33c5dbc700 1 -- 172.21.15.203:0/2709877747 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c014cf10 con 0x7f33c00bcfd0 2021-01-06T22:22:58.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.978+0000 7f33c5dbc700 1 -- 172.21.15.203:0/2709877747 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c014d450 con 0x7f33c00bcfd0 2021-01-06T22:22:58.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.982+0000 7f33bcff9700 1 -- 172.21.15.203:0/2709877747 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b400ea40 con 0x7f33c00bcfd0 2021-01-06T22:22:58.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.982+0000 7f33c5dbc700 1 -- 172.21.15.203:0/2709877747 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c014d610 con 0x7f33c00bcfd0 2021-01-06T22:22:58.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.986+0000 7f33bcff9700 1 -- 172.21.15.203:0/2709877747 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f33b4019030 con 0x7f33c00bcfd0 2021-01-06T22:22:58.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.986+0000 7f33bcff9700 1 --2- 172.21.15.203:0/2709877747 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f33a0056420 0x7f33a00588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:22:58.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.986+0000 7f33bcff9700 1 -- 172.21.15.203:0/2709877747 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7019+0+0 (secure 0 0 0) 0x7f33b406ec10 con 0x7f33c00bcfd0 2021-01-06T22:22:58.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.986+0000 7f33c4dba700 1 --2- 172.21.15.203:0/2709877747 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f33a0056420 0x7f33a00588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:22:58.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.986+0000 7f33c4dba700 1 --2- 172.21.15.203:0/2709877747 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f33a0056420 0x7f33a00588e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f33b80113e0 tx=0x7f33b80091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:22:58.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:58.990+0000 7f33bcff9700 1 -- 172.21.15.203:0/2709877747 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f33b40441b0 con 0x7f33c00bcfd0 2021-01-06T22:22:59.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.530+0000 7f33c5dbc700 1 -- 172.21.15.203:0/2709877747 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33c00450a0 con 0x7f33c00bcfd0 2021-01-06T22:22:59.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.530+0000 7f33bcff9700 1 -- 172.21.15.203:0/2709877747 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+3568 (secure 0 0 0) 0x7f33b4001e60 con 0x7f33c00bcfd0 2021-01-06T22:22:59.537 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:22:59.537 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":5874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":93}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:57.700077+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:22:59.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.538+0000 7f33a67fc700 1 -- 172.21.15.203:0/2709877747 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f33a0056420 msgr2=0x7f33a00588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:22:59.541 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.538+0000 7f33a67fc700 1 --2- 172.21.15.203:0/2709877747 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f33a0056420 0x7f33a00588e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f33b80113e0 tx=0x7f33b80091d0).stop 2021-01-06T22:22:59.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.538+0000 7f33a67fc700 1 -- 172.21.15.203:0/2709877747 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c00bcfd0 msgr2=0x7f33c014f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:22:59.542 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.538+0000 7f33a67fc700 1 --2- 172.21.15.203:0/2709877747 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c00bcfd0 0x7f33c014f0e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f33b400c7b0 tx=0x7f33b4007b60).stop 2021-01-06T22:22:59.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.538+0000 7f33a67fc700 1 -- 172.21.15.203:0/2709877747 shutdown_connections 2021-01-06T22:22:59.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.538+0000 7f33a67fc700 1 --2- 172.21.15.203:0/2709877747 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f33a0056420 0x7f33a00588e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:22:59.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.538+0000 7f33a67fc700 1 --2- 172.21.15.203:0/2709877747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33c00b7370 0x7f33c0163080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:22:59.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.538+0000 7f33a67fc700 1 --2- 172.21.15.203:0/2709877747 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33c00bcbf0 0x7f33c00b8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:22:59.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.538+0000 7f33a67fc700 1 --2- 172.21.15.203:0/2709877747 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c00bcfd0 0x7f33c014f0e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:22:59.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.538+0000 7f33a67fc700 1 -- 172.21.15.203:0/2709877747 >> 172.21.15.203:0/2709877747 conn(0x7f33c0056cd0 msgr2=0x7f33c00be2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:22:59.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.538+0000 7f33a67fc700 1 -- 172.21.15.203:0/2709877747 shutdown_connections 2021-01-06T22:22:59.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.538+0000 7f33a67fc700 1 -- 172.21.15.203:0/2709877747 wait complete. 2021-01-06T22:22:59.545 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 94 2021-01-06T22:22:59.565 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=1) 2021-01-06T22:22:59.566 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-01-06T22:22:59.566 INFO:teuthology.orchestra.run:waiting for 300 2021-01-06T22:22:59.581 INFO:tasks.ceph.mds.a:Stopped 2021-01-06T22:22:59.581 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:22:59.832 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.830+0000 7fb6f622d700 1 Processor -- start 2021-01-06T22:22:59.834 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.830+0000 7fb6f622d700 1 -- start start 2021-01-06T22:22:59.834 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.830+0000 7fb6f622d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb6e804f8d0 0x7fb6e80dd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:22:59.834 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.830+0000 7fb6f622d700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb6e80333c0 0x7fb6e8033830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:22:59.835 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.830+0000 7fb6f622d700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e80da5c0 con 0x7fb6e804fcb0 2021-01-06T22:22:59.835 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.830+0000 7fb6f622d700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e80dda50 con 0x7fb6e80333c0 2021-01-06T22:22:59.835 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.830+0000 7fb6f622d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e80d9de0 con 0x7fb6e804f8d0 2021-01-06T22:22:59.837 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.830+0000 7fb6ef7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb6e804f8d0 0x7fb6e80dd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:22:59.837 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.830+0000 7fb6ef7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb6e804f8d0 0x7fb6e80dd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:54608/0 (socket says 172.21.15.203:54608) 2021-01-06T22:22:59.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.830+0000 7fb6ef7fe700 1 -- 172.21.15.203:0/2139471030 learned_addr learned my addr 172.21.15.203:0/2139471030 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:22:59.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.830+0000 7fb6effff700 1 --2- 172.21.15.203:0/2139471030 >> v2:172.21.15.91:3300/0 conn(0x7fb6e80333c0 0x7fb6e8033830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:22:59.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.830+0000 7fb6ef7fe700 1 -- 172.21.15.203:0/2139471030 >> v2:172.21.15.91:3300/0 conn(0x7fb6e80333c0 msgr2=0x7fb6e8033830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:22:59.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.830+0000 7fb6ef7fe700 1 --2- 172.21.15.203:0/2139471030 >> v2:172.21.15.91:3300/0 conn(0x7fb6e80333c0 0x7fb6e8033830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:22:59.838 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.830+0000 7fb6ef7fe700 1 -- 172.21.15.203:0/2139471030 >> v1:172.21.15.91:6789/0 conn(0x7fb6e804fcb0 legacy=0x7fb6e80d9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:22:59.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.830+0000 7fb6ef7fe700 1 -- 172.21.15.203:0/2139471030 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e80da080 con 0x7fb6e804f8d0 2021-01-06T22:22:59.839 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.830+0000 7fb6ef7fe700 1 --2- 172.21.15.203:0/2139471030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e804f8d0 0x7fb6e80dd560 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7fb6e800d4e0 tx=0x7fb6dc006f30).ready entity=mon.1 client_cookie=18362b0237ab10b6 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:22:59.852 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.850+0000 7fb6ee7fc700 1 -- 172.21.15.203:0/2139471030 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc011290 con 0x7fb6e804f8d0 2021-01-06T22:22:59.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.850+0000 7fb6ee7fc700 1 -- 172.21.15.203:0/2139471030 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc009980 con 0x7fb6e804f8d0 2021-01-06T22:22:59.853 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.850+0000 7fb6ee7fc700 1 -- 172.21.15.203:0/2139471030 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc009d00 con 0x7fb6e804f8d0 2021-01-06T22:22:59.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.850+0000 7fb6f622d700 1 -- 172.21.15.203:0/2139471030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e804f8d0 msgr2=0x7fb6e80dd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:22:59.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.850+0000 7fb6f622d700 1 --2- 172.21.15.203:0/2139471030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e804f8d0 0x7fb6e80dd560 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7fb6e800d4e0 tx=0x7fb6dc006f30).stop 2021-01-06T22:22:59.854 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.850+0000 7fb6f622d700 1 -- 172.21.15.203:0/2139471030 shutdown_connections 2021-01-06T22:22:59.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.850+0000 7fb6f622d700 1 --2- 172.21.15.203:0/2139471030 >> v2:172.21.15.91:3300/0 conn(0x7fb6e80333c0 0x7fb6e8033830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:22:59.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.850+0000 7fb6f622d700 1 --2- 172.21.15.203:0/2139471030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e804f8d0 0x7fb6e80dd560 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:22:59.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.850+0000 7fb6f622d700 1 -- 172.21.15.203:0/2139471030 >> 172.21.15.203:0/2139471030 conn(0x7fb6e8006e30 msgr2=0x7fb6e8007210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:22:59.855 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.850+0000 7fb6f622d700 1 -- 172.21.15.203:0/2139471030 shutdown_connections 2021-01-06T22:22:59.856 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.850+0000 7fb6f622d700 1 -- 172.21.15.203:0/2139471030 wait complete. 2021-01-06T22:22:59.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6f622d700 1 Processor -- start 2021-01-06T22:22:59.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6f622d700 1 -- start start 2021-01-06T22:22:59.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6f622d700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6e80333c0 0x7fb6e8047ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:22:59.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6f622d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb6e804fcb0 0x7fb6e80e9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:22:59.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6f622d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e8045f30 0x7fb6e80e9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:22:59.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6f622d700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80dd070 con 0x7fb6e80333c0 2021-01-06T22:22:59.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6f622d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e8046400 con 0x7fb6e8045f30 2021-01-06T22:22:59.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6f622d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80dcb30 con 0x7fb6e804fcb0 2021-01-06T22:22:59.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6eeffd700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb6e804fcb0 0x7fb6e80e9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:22:59.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6effff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e8045f30 0x7fb6e80e9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:22:59.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6eeffd700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb6e804fcb0 0x7fb6e80e9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:59700/0 (socket says 172.21.15.203:59700) 2021-01-06T22:22:59.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6effff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e8045f30 0x7fb6e80e9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:54620/0 (socket says 172.21.15.203:54620) 2021-01-06T22:22:59.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6eeffd700 1 -- 172.21.15.203:0/2945520976 learned_addr learned my addr 172.21.15.203:0/2945520976 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:22:59.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6ef7fe700 1 --2- 172.21.15.203:0/2945520976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6e80333c0 0x7fb6e8047ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:22:59.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6effff700 1 -- 172.21.15.203:0/2945520976 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb6e804fcb0 msgr2=0x7fb6e80e9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:22:59.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6effff700 1 --2- 172.21.15.203:0/2945520976 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb6e804fcb0 0x7fb6e80e9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:22:59.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6effff700 1 -- 172.21.15.203:0/2945520976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6e80333c0 msgr2=0x7fb6e8047ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:22:59.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6effff700 1 --2- 172.21.15.203:0/2945520976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6e80333c0 0x7fb6e8047ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:22:59.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6effff700 1 -- 172.21.15.203:0/2945520976 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e80dcdd0 con 0x7fb6e8045f30 2021-01-06T22:22:59.865 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6effff700 1 --2- 172.21.15.203:0/2945520976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e8045f30 0x7fb6e80e9710 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7fb6f00455c0 tx=0x7fb6f0064040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:22:59.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6db7fe700 1 -- 172.21.15.203:0/2945520976 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0062e20 con 0x7fb6e8045f30 2021-01-06T22:22:59.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6f622d700 1 -- 172.21.15.203:0/2945520976 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e80e8d80 con 0x7fb6e8045f30 2021-01-06T22:22:59.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.854+0000 7fb6f622d700 1 -- 172.21.15.203:0/2945520976 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e8048390 con 0x7fb6e8045f30 2021-01-06T22:22:59.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.858+0000 7fb6db7fe700 1 -- 172.21.15.203:0/2945520976 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f0068030 con 0x7fb6e8045f30 2021-01-06T22:22:59.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.858+0000 7fb6db7fe700 1 -- 172.21.15.203:0/2945520976 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00685e0 con 0x7fb6e8045f30 2021-01-06T22:22:59.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.858+0000 7fb6db7fe700 1 -- 172.21.15.203:0/2945520976 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fb6f00923f0 con 0x7fb6e8045f30 2021-01-06T22:22:59.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.858+0000 7fb6db7fe700 1 --2- 172.21.15.203:0/2945520976 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb6d40565b0 0x7fb6d4058a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:22:59.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.858+0000 7fb6db7fe700 1 -- 172.21.15.203:0/2945520976 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7019+0+0 (secure 0 0 0) 0x7fb6f00dfeb0 con 0x7fb6e8045f30 2021-01-06T22:22:59.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.858+0000 7fb6ef7fe700 1 --2- 172.21.15.203:0/2945520976 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb6d40565b0 0x7fb6d4058a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:22:59.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.858+0000 7fb6ef7fe700 1 --2- 172.21.15.203:0/2945520976 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb6d40565b0 0x7fb6d4058a70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fb6e800b4a0 tx=0x7fb6dc01c000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:22:59.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.862+0000 7fb6f622d700 1 -- 172.21.15.203:0/2945520976 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d0004f60 con 0x7fb6e8045f30 2021-01-06T22:22:59.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:22:59.870+0000 7fb6db7fe700 1 -- 172.21.15.203:0/2945520976 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fb6f00df070 con 0x7fb6e8045f30 2021-01-06T22:23:00.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:00.266+0000 7fb6f622d700 1 -- 172.21.15.203:0/2945520976 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6d0005790 con 0x7fb6e8045f30 2021-01-06T22:23:00.271 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:23:00.271 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":5874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":93}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:57.700077+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:23:00.272 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:00.266+0000 7fb6db7fe700 1 -- 172.21.15.203:0/2945520976 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+3568 (secure 0 0 0) 0x7fb6f00b5450 con 0x7fb6e8045f30 2021-01-06T22:23:00.273 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:00.270+0000 7fb6d97fa700 1 -- 172.21.15.203:0/2945520976 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb6d40565b0 msgr2=0x7fb6d4058a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:00.274 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:00.270+0000 7fb6d97fa700 1 --2- 172.21.15.203:0/2945520976 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb6d40565b0 0x7fb6d4058a70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fb6e800b4a0 tx=0x7fb6dc01c000).stop 2021-01-06T22:23:00.274 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:00.270+0000 7fb6d97fa700 1 -- 172.21.15.203:0/2945520976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e8045f30 msgr2=0x7fb6e80e9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:00.274 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:00.270+0000 7fb6d97fa700 1 --2- 172.21.15.203:0/2945520976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e8045f30 0x7fb6e80e9710 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7fb6f00455c0 tx=0x7fb6f0064040).stop 2021-01-06T22:23:00.274 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:00.270+0000 7fb6d97fa700 1 -- 172.21.15.203:0/2945520976 shutdown_connections 2021-01-06T22:23:00.275 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:00.270+0000 7fb6d97fa700 1 --2- 172.21.15.203:0/2945520976 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fb6d40565b0 0x7fb6d4058a70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:00.275 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:00.270+0000 7fb6d97fa700 1 --2- 172.21.15.203:0/2945520976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e8045f30 0x7fb6e80e9710 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:00.275 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:00.270+0000 7fb6d97fa700 1 --2- 172.21.15.203:0/2945520976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6e80333c0 0x7fb6e8047ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:00.275 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:00.270+0000 7fb6d97fa700 1 --2- 172.21.15.203:0/2945520976 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb6e804fcb0 0x7fb6e80e9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:00.275 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:00.270+0000 7fb6d97fa700 1 -- 172.21.15.203:0/2945520976 >> 172.21.15.203:0/2945520976 conn(0x7fb6e8006e30 msgr2=0x7fb6e804ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:00.276 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:00.270+0000 7fb6d97fa700 1 -- 172.21.15.203:0/2945520976 shutdown_connections 2021-01-06T22:23:00.276 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:00.270+0000 7fb6d97fa700 1 -- 172.21.15.203:0/2945520976 wait complete. 2021-01-06T22:23:00.276 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 94 2021-01-06T22:23:00.292 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:23:02.293 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:23:02.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.506+0000 7f7941297700 1 Processor -- start 2021-01-06T22:23:02.510 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.506+0000 7f7941297700 1 -- start start 2021-01-06T22:23:02.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.510+0000 7f7941297700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f793c0c0520 0x7f793c14dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:02.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.510+0000 7f7941297700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f793c149f10 0x7f793c0badb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:02.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.510+0000 7f7941297700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793c0bbbf0 con 0x7f793c0c0900 2021-01-06T22:23:02.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.510+0000 7f7941297700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793c14a360 con 0x7f793c0c0520 2021-01-06T22:23:02.513 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.510+0000 7f7941297700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0bb6b0 con 0x7f793c149f10 2021-01-06T22:23:02.513 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.510+0000 7f793affd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f793c0c0520 0x7f793c14dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:02.514 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.510+0000 7f793b7fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f793c149f10 0x7f793c0badb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:02.514 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.510+0000 7f793affd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f793c0c0520 0x7f793c14dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:54728/0 (socket says 172.21.15.203:54728) 2021-01-06T22:23:02.514 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.510+0000 7f793affd700 1 -- 172.21.15.203:0/3225106643 learned_addr learned my addr 172.21.15.203:0/3225106643 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:23:02.514 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.510+0000 7f793affd700 1 -- 172.21.15.203:0/3225106643 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f793c149f10 msgr2=0x7f793c0badb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:23:02.515 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.510+0000 7f793affd700 1 --2- 172.21.15.203:0/3225106643 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f793c149f10 0x7f793c0badb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:02.515 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.510+0000 7f793affd700 1 -- 172.21.15.203:0/3225106643 >> v1:172.21.15.203:6789/0 conn(0x7f793c0c0900 legacy=0x7f793c0b7c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-01-06T22:23:02.515 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.510+0000 7f793affd700 1 -- 172.21.15.203:0/3225106643 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c0bb950 con 0x7f793c0c0520 2021-01-06T22:23:02.516 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.514+0000 7f793affd700 1 --2- 172.21.15.203:0/3225106643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f793c0c0520 0x7f793c14dfd0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f792c0078a0 tx=0x7f792c00acc0).ready entity=mon.1 client_cookie=7d1d8a69485fc794 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:02.516 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.514+0000 7f793a7fc700 1 -- 172.21.15.203:0/3225106643 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c00f070 con 0x7f793c0c0520 2021-01-06T22:23:02.517 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.514+0000 7f793a7fc700 1 -- 172.21.15.203:0/3225106643 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792c00cbd0 con 0x7f793c0c0520 2021-01-06T22:23:02.517 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.514+0000 7f793a7fc700 1 -- 172.21.15.203:0/3225106643 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c0089e0 con 0x7f793c0c0520 2021-01-06T22:23:02.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.514+0000 7f7941297700 1 -- 172.21.15.203:0/3225106643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f793c0c0520 msgr2=0x7f793c14dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:02.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.514+0000 7f7941297700 1 --2- 172.21.15.203:0/3225106643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f793c0c0520 0x7f793c14dfd0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f792c0078a0 tx=0x7f792c00acc0).stop 2021-01-06T22:23:02.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.514+0000 7f7941297700 1 -- 172.21.15.203:0/3225106643 shutdown_connections 2021-01-06T22:23:02.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.514+0000 7f7941297700 1 --2- 172.21.15.203:0/3225106643 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f793c149f10 0x7f793c0badb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:02.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.514+0000 7f7941297700 1 --2- 172.21.15.203:0/3225106643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f793c0c0520 0x7f793c14dfd0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:02.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.514+0000 7f7941297700 1 -- 172.21.15.203:0/3225106643 >> 172.21.15.203:0/3225106643 conn(0x7f793c056cd0 msgr2=0x7f793c0626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:02.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.514+0000 7f7941297700 1 -- 172.21.15.203:0/3225106643 shutdown_connections 2021-01-06T22:23:02.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.514+0000 7f7941297700 1 -- 172.21.15.203:0/3225106643 wait complete. 2021-01-06T22:23:02.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.518+0000 7f7941297700 1 Processor -- start 2021-01-06T22:23:02.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f7941297700 1 -- start start 2021-01-06T22:23:02.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f7941297700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f793c0c0520 0x7f793c1628e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:02.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f7941297700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f793c0c0900 0x7f793c0672b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:02.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f7941297700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f793c149f10 0x7f793c145890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:02.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f7941297700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0afa30 con 0x7f793c0c0900 2021-01-06T22:23:02.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f7941297700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0afcd0 con 0x7f793c149f10 2021-01-06T22:23:02.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f7941297700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0aff70 con 0x7f793c0c0520 2021-01-06T22:23:02.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f79327fc700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f793c0c0900 0x7f793c0672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:02.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f79327fc700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f793c0c0900 0x7f793c0672b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:34350/0 (socket says 172.21.15.203:34350) 2021-01-06T22:23:02.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f79327fc700 1 -- 172.21.15.203:0/3931056948 learned_addr learned my addr 172.21.15.203:0/3931056948 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:23:02.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f793affd700 1 --2- 172.21.15.203:0/3931056948 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f793c0c0520 0x7f793c1628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:02.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f793b7fe700 1 --2- 172.21.15.203:0/3931056948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f793c149f10 0x7f793c145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:02.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f793affd700 1 -- 172.21.15.203:0/3931056948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f793c149f10 msgr2=0x7f793c145890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:02.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f793affd700 1 --2- 172.21.15.203:0/3931056948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f793c149f10 0x7f793c145890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:02.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f793affd700 1 -- 172.21.15.203:0/3931056948 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f793c0c0900 msgr2=0x7f793c0672b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:02.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f793affd700 1 --2- 172.21.15.203:0/3931056948 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f793c0c0900 0x7f793c0672b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:02.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f793affd700 1 -- 172.21.15.203:0/3931056948 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c1621a0 con 0x7f793c0c0520 2021-01-06T22:23:02.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f793affd700 1 --2- 172.21.15.203:0/3931056948 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f793c0c0520 0x7f793c1628e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f792c014000 tx=0x7f793c063d60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:02.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f793b7fe700 1 --2- 172.21.15.203:0/3931056948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f793c149f10 0x7f793c145890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:23:02.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.522+0000 7f79337fe700 1 -- 172.21.15.203:0/3931056948 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c00a530 con 0x7f793c0c0520 2021-01-06T22:23:02.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.526+0000 7f7941297700 1 -- 172.21.15.203:0/3931056948 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793c162440 con 0x7f793c0c0520 2021-01-06T22:23:02.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.526+0000 7f7941297700 1 -- 172.21.15.203:0/3931056948 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793c162dd0 con 0x7f793c0c0520 2021-01-06T22:23:02.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.526+0000 7f79337fe700 1 -- 172.21.15.203:0/3931056948 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792c00a690 con 0x7f793c0c0520 2021-01-06T22:23:02.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.530+0000 7f7930ff9700 1 -- 172.21.15.203:0/3931056948 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793c0b0130 con 0x7f793c0c0520 2021-01-06T22:23:02.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.530+0000 7f79337fe700 1 -- 172.21.15.203:0/3931056948 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c014420 con 0x7f793c0c0520 2021-01-06T22:23:02.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.530+0000 7f79337fe700 1 -- 172.21.15.203:0/3931056948 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f792c014790 con 0x7f793c0c0520 2021-01-06T22:23:02.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.530+0000 7f79337fe700 1 --2- 172.21.15.203:0/3931056948 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f79200564a0 0x7f7920058960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:02.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.530+0000 7f79337fe700 1 -- 172.21.15.203:0/3931056948 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7019+0+0 (secure 0 0 0) 0x7f792c0714a0 con 0x7f793c0c0520 2021-01-06T22:23:02.533 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.530+0000 7f79327fc700 1 --2- 172.21.15.203:0/3931056948 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f79200564a0 0x7f7920058960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:02.534 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.530+0000 7f79327fc700 1 --2- 172.21.15.203:0/3931056948 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f79200564a0 0x7f7920058960 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f793c064d00 tx=0x7f7924008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:02.538 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.534+0000 7f79337fe700 1 -- 172.21.15.203:0/3931056948 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f792c046930 con 0x7f793c0c0520 2021-01-06T22:23:02.928 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.926+0000 7f7930ff9700 1 -- 172.21.15.203:0/3931056948 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f793c05cfe0 con 0x7f793c0c0520 2021-01-06T22:23:02.929 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.926+0000 7f79337fe700 1 -- 172.21.15.203:0/3931056948 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+3568 (secure 0 0 0) 0x7f792c049f50 con 0x7f793c0c0520 2021-01-06T22:23:02.932 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:23:02.932 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":5874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":93}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:57.700077+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:23:02.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.930+0000 7f7941297700 1 -- 172.21.15.203:0/3931056948 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f79200564a0 msgr2=0x7f7920058960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:02.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.930+0000 7f7941297700 1 --2- 172.21.15.203:0/3931056948 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f79200564a0 0x7f7920058960 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f793c064d00 tx=0x7f7924008040).stop 2021-01-06T22:23:02.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.930+0000 7f7941297700 1 -- 172.21.15.203:0/3931056948 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f793c0c0520 msgr2=0x7f793c1628e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:02.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.930+0000 7f7941297700 1 --2- 172.21.15.203:0/3931056948 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f793c0c0520 0x7f793c1628e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f792c014000 tx=0x7f793c063d60).stop 2021-01-06T22:23:02.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.930+0000 7f7941297700 1 -- 172.21.15.203:0/3931056948 shutdown_connections 2021-01-06T22:23:02.935 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.930+0000 7f7941297700 1 --2- 172.21.15.203:0/3931056948 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f79200564a0 0x7f7920058960 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:02.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.930+0000 7f7941297700 1 --2- 172.21.15.203:0/3931056948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f793c149f10 0x7f793c145890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:02.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.930+0000 7f7941297700 1 --2- 172.21.15.203:0/3931056948 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f793c0c0520 0x7f793c1628e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:02.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.930+0000 7f7941297700 1 --2- 172.21.15.203:0/3931056948 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f793c0c0900 0x7f793c0672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:02.936 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.930+0000 7f7941297700 1 -- 172.21.15.203:0/3931056948 >> 172.21.15.203:0/3931056948 conn(0x7f793c056cd0 msgr2=0x7f793c0626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:02.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.930+0000 7f7941297700 1 -- 172.21.15.203:0/3931056948 shutdown_connections 2021-01-06T22:23:02.937 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:02.930+0000 7f7941297700 1 -- 172.21.15.203:0/3931056948 wait complete. 2021-01-06T22:23:02.943 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 94 2021-01-06T22:23:02.954 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:23:04.956 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:23:05.196 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.190+0000 7fef6ead0700 1 Processor -- start 2021-01-06T22:23:05.199 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.190+0000 7fef6ead0700 1 -- start start 2021-01-06T22:23:05.199 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.194+0000 7fef6ead0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef680c0540 0x7fef68132c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:05.200 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.194+0000 7fef6ead0700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fef680bd890 0x7fef68133140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:05.200 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.194+0000 7fef6ead0700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef68149e90 con 0x7fef680c0920 2021-01-06T22:23:05.200 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.194+0000 7fef6ead0700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef68133630 con 0x7fef680bd890 2021-01-06T22:23:05.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.194+0000 7fef6ead0700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6814a130 con 0x7fef680c0540 2021-01-06T22:23:05.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.194+0000 7fef67fff700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fef680c0920 0x7fef680bd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:48072/0 (socket says 172.21.15.203:48072) 2021-01-06T22:23:05.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.194+0000 7fef67fff700 1 -- 172.21.15.203:0/2287887659 learned_addr learned my addr 172.21.15.203:0/2287887659 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:23:05.204 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.194+0000 7fef6c86c700 1 --2- 172.21.15.203:0/2287887659 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef680c0540 0x7fef68132c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:05.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.194+0000 7fef6d06d700 1 --2- 172.21.15.203:0/2287887659 >> v2:172.21.15.91:3300/0 conn(0x7fef680bd890 0x7fef68133140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:05.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.194+0000 7fef6c86c700 1 -- 172.21.15.203:0/2287887659 >> v2:172.21.15.91:3300/0 conn(0x7fef680bd890 msgr2=0x7fef68133140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:23:05.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.194+0000 7fef6c86c700 1 --2- 172.21.15.203:0/2287887659 >> v2:172.21.15.91:3300/0 conn(0x7fef680bd890 0x7fef68133140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:05.205 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.194+0000 7fef6c86c700 1 -- 172.21.15.203:0/2287887659 >> v1:172.21.15.91:6789/0 conn(0x7fef680c0920 legacy=0x7fef680bd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:05.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.194+0000 7fef6c86c700 1 -- 172.21.15.203:0/2287887659 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6814a3d0 con 0x7fef680c0540 2021-01-06T22:23:05.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.194+0000 7fef6c86c700 1 --2- 172.21.15.203:0/2287887659 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef680c0540 0x7fef68132c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fef5c00ca90 tx=0x7fef5c006de0).ready entity=mon.2 client_cookie=19e92155c9730117 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:05.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.198+0000 7fef677fe700 1 -- 172.21.15.203:0/2287887659 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c009770 con 0x7fef680c0540 2021-01-06T22:23:05.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.198+0000 7fef677fe700 1 -- 172.21.15.203:0/2287887659 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c0098d0 con 0x7fef680c0540 2021-01-06T22:23:05.206 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.198+0000 7fef677fe700 1 -- 172.21.15.203:0/2287887659 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c009c50 con 0x7fef680c0540 2021-01-06T22:23:05.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.198+0000 7fef6ead0700 1 -- 172.21.15.203:0/2287887659 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef680c0540 msgr2=0x7fef68132c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:05.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.198+0000 7fef6ead0700 1 --2- 172.21.15.203:0/2287887659 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef680c0540 0x7fef68132c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fef5c00ca90 tx=0x7fef5c006de0).stop 2021-01-06T22:23:05.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.198+0000 7fef6ead0700 1 -- 172.21.15.203:0/2287887659 shutdown_connections 2021-01-06T22:23:05.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.198+0000 7fef6ead0700 1 --2- 172.21.15.203:0/2287887659 >> v2:172.21.15.91:3300/0 conn(0x7fef680bd890 0x7fef68133140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:05.207 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.198+0000 7fef6ead0700 1 --2- 172.21.15.203:0/2287887659 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef680c0540 0x7fef68132c50 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0x7fef5c00ca90 tx=0x7fef5c006de0).stop 2021-01-06T22:23:05.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.198+0000 7fef6ead0700 1 -- 172.21.15.203:0/2287887659 >> 172.21.15.203:0/2287887659 conn(0x7fef68056cd0 msgr2=0x7fef680626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:05.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.198+0000 7fef6ead0700 1 -- 172.21.15.203:0/2287887659 shutdown_connections 2021-01-06T22:23:05.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.198+0000 7fef6ead0700 1 -- 172.21.15.203:0/2287887659 wait complete. 2021-01-06T22:23:05.208 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.198+0000 7fef6ead0700 1 Processor -- start 2021-01-06T22:23:05.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.198+0000 7fef6ead0700 1 -- start start 2021-01-06T22:23:05.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef6ead0700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef680bd890 0x7fef68146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:05.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef6ead0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef680c0920 0x7fef681280c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:05.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef6ead0700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef68147870 0x7fef68154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:05.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef6ead0700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef681461e0 con 0x7fef68147870 2021-01-06T22:23:05.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef6ead0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef681465c0 con 0x7fef680c0920 2021-01-06T22:23:05.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef6ead0700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef68146860 con 0x7fef680bd890 2021-01-06T22:23:05.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef67fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef680c0920 0x7fef681280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:05.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef67fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef680c0920 0x7fef681280c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:54826/0 (socket says 172.21.15.203:54826) 2021-01-06T22:23:05.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef67fff700 1 -- 172.21.15.203:0/3034559257 learned_addr learned my addr 172.21.15.203:0/3034559257 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:23:05.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef6d06d700 1 --2- 172.21.15.203:0/3034559257 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef68147870 0x7fef68154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:05.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef67fff700 1 -- 172.21.15.203:0/3034559257 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef680bd890 msgr2=0x7fef68146ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-01-06T22:23:05.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef67fff700 1 --2- 172.21.15.203:0/3034559257 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef680bd890 0x7fef68146ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-01-06T22:23:05.212 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef67fff700 1 -- 172.21.15.203:0/3034559257 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef68147870 msgr2=0x7fef68154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:05.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef67fff700 1 --2- 172.21.15.203:0/3034559257 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef68147870 0x7fef68154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:05.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef67fff700 1 -- 172.21.15.203:0/3034559257 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef68146b00 con 0x7fef680c0920 2021-01-06T22:23:05.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef67fff700 1 --2- 172.21.15.203:0/3034559257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef680c0920 0x7fef681280c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7fef6000cae0 tx=0x7fef6000cea0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:05.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef64ff9700 1 -- 172.21.15.203:0/3034559257 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60005b70 con 0x7fef680c0920 2021-01-06T22:23:05.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef6ead0700 1 -- 172.21.15.203:0/3034559257 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef68127980 con 0x7fef680c0920 2021-01-06T22:23:05.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.202+0000 7fef6ead0700 1 -- 172.21.15.203:0/3034559257 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef68144fe0 con 0x7fef680c0920 2021-01-06T22:23:05.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.206+0000 7fef64ff9700 1 -- 172.21.15.203:0/3034559257 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef60005cd0 con 0x7fef680c0920 2021-01-06T22:23:05.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.206+0000 7fef64ff9700 1 -- 172.21.15.203:0/3034559257 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60007340 con 0x7fef680c0920 2021-01-06T22:23:05.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.206+0000 7fef64ff9700 1 -- 172.21.15.203:0/3034559257 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fef60010030 con 0x7fef680c0920 2021-01-06T22:23:05.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.206+0000 7fef6ead0700 1 -- 172.21.15.203:0/3034559257 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef40004f60 con 0x7fef680c0920 2021-01-06T22:23:05.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.214+0000 7fef64ff9700 1 --2- 172.21.15.203:0/3034559257 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fef480563d0 0x7fef48058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:05.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.214+0000 7fef6c86c700 1 --2- 172.21.15.203:0/3034559257 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fef480563d0 0x7fef48058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:05.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.214+0000 7fef6c86c700 1 --2- 172.21.15.203:0/3034559257 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fef480563d0 0x7fef48058890 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fef68065720 tx=0x7fef5c018000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:05.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.218+0000 7fef64ff9700 1 -- 172.21.15.203:0/3034559257 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7019+0+0 (secure 0 0 0) 0x7fef6006fb20 con 0x7fef680c0920 2021-01-06T22:23:05.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.218+0000 7fef64ff9700 1 -- 172.21.15.203:0/3034559257 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fef60071e60 con 0x7fef680c0920 2021-01-06T22:23:05.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.642+0000 7fef6ead0700 1 -- 172.21.15.203:0/3034559257 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef40005790 con 0x7fef680c0920 2021-01-06T22:23:05.651 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.642+0000 7fef64ff9700 1 -- 172.21.15.203:0/3034559257 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+3568 (secure 0 0 0) 0x7fef60044fb0 con 0x7fef680c0920 2021-01-06T22:23:05.652 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:23:05.652 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":5874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":93}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:57.700077+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:23:05.665 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.662+0000 7fef4e7fc700 1 -- 172.21.15.203:0/3034559257 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fef480563d0 msgr2=0x7fef48058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:05.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.662+0000 7fef4e7fc700 1 --2- 172.21.15.203:0/3034559257 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fef480563d0 0x7fef48058890 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fef68065720 tx=0x7fef5c018000).stop 2021-01-06T22:23:05.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.662+0000 7fef4e7fc700 1 -- 172.21.15.203:0/3034559257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef680c0920 msgr2=0x7fef681280c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:05.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.662+0000 7fef4e7fc700 1 --2- 172.21.15.203:0/3034559257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef680c0920 0x7fef681280c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7fef6000cae0 tx=0x7fef6000cea0).stop 2021-01-06T22:23:05.666 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.662+0000 7fef4e7fc700 1 -- 172.21.15.203:0/3034559257 shutdown_connections 2021-01-06T22:23:05.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.662+0000 7fef4e7fc700 1 --2- 172.21.15.203:0/3034559257 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fef480563d0 0x7fef48058890 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:05.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.662+0000 7fef4e7fc700 1 --2- 172.21.15.203:0/3034559257 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef68147870 0x7fef68154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:05.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.662+0000 7fef4e7fc700 1 --2- 172.21.15.203:0/3034559257 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef680bd890 0x7fef68146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-01-06T22:23:05.667 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.662+0000 7fef4e7fc700 1 --2- 172.21.15.203:0/3034559257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef680c0920 0x7fef681280c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:05.668 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.662+0000 7fef4e7fc700 1 -- 172.21.15.203:0/3034559257 >> 172.21.15.203:0/3034559257 conn(0x7fef68056cd0 msgr2=0x7fef68157470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:05.668 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.662+0000 7fef4e7fc700 1 -- 172.21.15.203:0/3034559257 shutdown_connections 2021-01-06T22:23:05.668 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:05.662+0000 7fef4e7fc700 1 -- 172.21.15.203:0/3034559257 wait complete. 2021-01-06T22:23:05.674 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 94 2021-01-06T22:23:05.698 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:23:07.700 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:23:07.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.977+0000 7f27fe440700 1 Processor -- start 2021-01-06T22:23:07.984 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.977+0000 7f27fe440700 1 -- start start 2021-01-06T22:23:07.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.977+0000 7f27fe440700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f27f8155fb0 0x7f27f80668d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:07.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.977+0000 7f27fe440700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27f80a5f90 0x7f27f80ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:07.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.977+0000 7f27fe440700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f80606b0 con 0x7f27f80c0160 2021-01-06T22:23:07.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.977+0000 7f27fe440700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f80baec0 con 0x7f27f8155fb0 2021-01-06T22:23:07.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.977+0000 7f27fe440700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f8066ef0 con 0x7f27f80a5f90 2021-01-06T22:23:07.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27fd43e700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f27f80c0160 0x7f27f81558a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:53528/0 (socket says 172.21.15.203:53528) 2021-01-06T22:23:07.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27fd43e700 1 -- 172.21.15.203:0/362413446 learned_addr learned my addr 172.21.15.203:0/362413446 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:23:07.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27fcc3d700 1 --2- 172.21.15.203:0/362413446 >> v2:172.21.15.91:3300/0 conn(0x7f27f8155fb0 0x7f27f80668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:07.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27fdc3f700 1 --2- 172.21.15.203:0/362413446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27f80a5f90 0x7f27f80ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:07.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 -- 172.21.15.203:0/362413446 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 859730129 0 0) 0x7f27f80606b0 con 0x7f27f80c0160 2021-01-06T22:23:07.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 -- 172.21.15.203:0/362413446 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27dc003230 con 0x7f27f80c0160 2021-01-06T22:23:07.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 -- 172.21.15.203:0/362413446 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3033220973 0 0) 0x7f27dc003230 con 0x7f27f80c0160 2021-01-06T22:23:07.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 -- 172.21.15.203:0/362413446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27f80a5f90 msgr2=0x7f27f80ba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:23:07.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 --2- 172.21.15.203:0/362413446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27f80a5f90 0x7f27f80ba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:07.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 -- 172.21.15.203:0/362413446 >> v2:172.21.15.91:3300/0 conn(0x7f27f8155fb0 msgr2=0x7f27f80668d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:23:07.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 --2- 172.21.15.203:0/362413446 >> v2:172.21.15.91:3300/0 conn(0x7f27f8155fb0 0x7f27f80668d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:07.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 -- 172.21.15.203:0/362413446 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f80608e0 con 0x7f27f80c0160 2021-01-06T22:23:07.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 -- 172.21.15.203:0/362413446 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1627105206 0 0) 0x7f27f40014e0 con 0x7f27f80c0160 2021-01-06T22:23:07.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 -- 172.21.15.203:0/362413446 >> v1:172.21.15.203:6789/0 conn(0x7f27f80c0160 legacy=0x7f27f81558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:07.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 --2- 172.21.15.203:0/362413446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27dc003720 0x7f27dc003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:07.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 --2- 172.21.15.203:0/362413446 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27dc0040a0 0x7f27dc004570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:07.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 --2- 172.21.15.203:0/362413446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27f80a5f90 0x7f27dc004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:07.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 -- 172.21.15.203:0/362413446 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f80baec0 con 0x7f27dc0040a0 2021-01-06T22:23:07.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 -- 172.21.15.203:0/362413446 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f8066ef0 con 0x7f27dc003720 2021-01-06T22:23:07.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 -- 172.21.15.203:0/362413446 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc005190 con 0x7f27f80a5f90 2021-01-06T22:23:07.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27fcc3d700 1 --2- 172.21.15.203:0/362413446 >> v2:172.21.15.91:3300/0 conn(0x7f27f8155fb0 0x7f27f80668d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:23:07.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27fdc3f700 1 --2- 172.21.15.203:0/362413446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27f80a5f90 0x7f27dc004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:07.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27fcc3d700 1 --2- 172.21.15.203:0/362413446 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27dc0040a0 0x7f27dc004570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:07.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27fdc3f700 1 -- 172.21.15.203:0/362413446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27dc003720 msgr2=0x7f27dc003b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-01-06T22:23:07.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27fdc3f700 1 --2- 172.21.15.203:0/362413446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27dc003720 0x7f27dc003b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:23:07.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27fdc3f700 1 -- 172.21.15.203:0/362413446 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27dc0040a0 msgr2=0x7f27dc004570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:23:07.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27fdc3f700 1 --2- 172.21.15.203:0/362413446 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27dc0040a0 0x7f27dc004570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:07.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27fdc3f700 1 -- 172.21.15.203:0/362413446 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f80608e0 con 0x7f27f80a5f90 2021-01-06T22:23:07.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27fdc3f700 1 --2- 172.21.15.203:0/362413446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27f80a5f90 0x7f27dc004bd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f27e800bb40 tx=0x7f27e8008040).ready entity=mon.2 client_cookie=c88158452592311a server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:07.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 -- 172.21.15.203:0/362413446 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e800b2b0 con 0x7f27f80a5f90 2021-01-06T22:23:07.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 -- 172.21.15.203:0/362413446 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e800ea90 con 0x7f27f80a5f90 2021-01-06T22:23:07.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27effff700 1 -- 172.21.15.203:0/362413446 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e800ee10 con 0x7f27f80a5f90 2021-01-06T22:23:07.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27fe440700 1 -- 172.21.15.203:0/362413446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27f80a5f90 msgr2=0x7f27dc004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:07.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.981+0000 7f27fe440700 1 --2- 172.21.15.203:0/362413446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27f80a5f90 0x7f27dc004bd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f27e800bb40 tx=0x7f27e8008040).stop 2021-01-06T22:23:07.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.985+0000 7f27fdc3f700 1 -- 172.21.15.203:0/362413446 reap_dead start 2021-01-06T22:23:07.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.985+0000 7f27fe440700 1 -- 172.21.15.203:0/362413446 shutdown_connections 2021-01-06T22:23:07.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.985+0000 7f27fe440700 1 -- 172.21.15.203:0/362413446 >> 172.21.15.203:0/362413446 conn(0x7f27f8056cd0 msgr2=0x7f27f8061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:07.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.985+0000 7f27fe440700 1 -- 172.21.15.203:0/362413446 shutdown_connections 2021-01-06T22:23:07.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.985+0000 7f27fe440700 1 -- 172.21.15.203:0/362413446 wait complete. 2021-01-06T22:23:08.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.985+0000 7f27fe440700 1 Processor -- start 2021-01-06T22:23:08.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.989+0000 7f27fe440700 1 -- start start 2021-01-06T22:23:08.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.989+0000 7f27fe440700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27f80a5f90 0x7f27f8132bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:08.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.989+0000 7f27fe440700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27f80a5700 0x7f27f80b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:08.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.989+0000 7f27fe440700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27f80b3530 0x7f27f81522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:08.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.989+0000 7f27fe440700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f8060520 con 0x7f27f80a5700 2021-01-06T22:23:08.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.989+0000 7f27fe440700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f80a35c0 con 0x7f27f80b3530 2021-01-06T22:23:08.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.989+0000 7f27fe440700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f80b6a10 con 0x7f27f80a5f90 2021-01-06T22:23:08.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.989+0000 7f27fdc3f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27f80b3530 0x7f27f81522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:08.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.989+0000 7f27fcc3d700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27f80a5700 0x7f27f80b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:08.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.989+0000 7f27fdc3f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27f80b3530 0x7f27f81522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:54942/0 (socket says 172.21.15.203:54942) 2021-01-06T22:23:08.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.989+0000 7f27fdc3f700 1 -- 172.21.15.203:0/2884940323 learned_addr learned my addr 172.21.15.203:0/2884940323 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:23:08.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.989+0000 7f27fd43e700 1 --2- 172.21.15.203:0/2884940323 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27f80a5f90 0x7f27f8132bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:08.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.993+0000 7f27fdc3f700 1 -- 172.21.15.203:0/2884940323 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27f80a5f90 msgr2=0x7f27f8132bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:08.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.993+0000 7f27fdc3f700 1 --2- 172.21.15.203:0/2884940323 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27f80a5f90 0x7f27f8132bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:08.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.993+0000 7f27fdc3f700 1 -- 172.21.15.203:0/2884940323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27f80a5700 msgr2=0x7f27f80b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:08.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.993+0000 7f27fdc3f700 1 --2- 172.21.15.203:0/2884940323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27f80a5700 0x7f27f80b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:08.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.993+0000 7f27fdc3f700 1 -- 172.21.15.203:0/2884940323 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f80b6cb0 con 0x7f27f80b3530 2021-01-06T22:23:08.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.993+0000 7f27fd43e700 1 --2- 172.21.15.203:0/2884940323 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27f80a5f90 0x7f27f8132bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:23:08.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.993+0000 7f27fdc3f700 1 --2- 172.21.15.203:0/2884940323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27f80b3530 0x7f27f81522c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f27f805c990 tx=0x7f27e8019000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:08.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:07.993+0000 7f27ed7fa700 1 -- 172.21.15.203:0/2884940323 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e8020030 con 0x7f27f80b3530 2021-01-06T22:23:08.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.001+0000 7f27ed7fa700 1 -- 172.21.15.203:0/2884940323 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e8012dc0 con 0x7f27f80b3530 2021-01-06T22:23:08.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.001+0000 7f27fe440700 1 -- 172.21.15.203:0/2884940323 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f80a37f0 con 0x7f27f80b3530 2021-01-06T22:23:08.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.001+0000 7f27fe440700 1 -- 172.21.15.203:0/2884940323 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f80a3d30 con 0x7f27f80b3530 2021-01-06T22:23:08.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.001+0000 7f27ed7fa700 1 -- 172.21.15.203:0/2884940323 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e801c6a0 con 0x7f27f80b3530 2021-01-06T22:23:08.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.005+0000 7f27ed7fa700 1 -- 172.21.15.203:0/2884940323 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f27e801ca10 con 0x7f27f80b3530 2021-01-06T22:23:08.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.005+0000 7f27ed7fa700 1 --2- 172.21.15.203:0/2884940323 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f27d8056420 0x7f27d80588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:08.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.005+0000 7f27ed7fa700 1 -- 172.21.15.203:0/2884940323 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7019+0+0 (secure 0 0 0) 0x7f27e8021070 con 0x7f27f80b3530 2021-01-06T22:23:08.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.005+0000 7f27fd43e700 1 --2- 172.21.15.203:0/2884940323 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f27d8056420 0x7f27d80588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:08.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.005+0000 7f27fd43e700 1 --2- 172.21.15.203:0/2884940323 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f27d8056420 0x7f27d80588e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f27f4000f30 tx=0x7f27f4000b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:08.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.009+0000 7f27fe440700 1 -- 172.21.15.203:0/2884940323 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d0004f60 con 0x7f27f80b3530 2021-01-06T22:23:08.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.021+0000 7f27ed7fa700 1 -- 172.21.15.203:0/2884940323 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f27e8046640 con 0x7f27f80b3530 2021-01-06T22:23:08.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.341+0000 7f27fe440700 1 -- 172.21.15.203:0/2884940323 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27d0005790 con 0x7f27f80b3530 2021-01-06T22:23:08.347 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.341+0000 7f27ed7fa700 1 -- 172.21.15.203:0/2884940323 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+3568 (secure 0 0 0) 0x7f27e8049c60 con 0x7f27f80b3530 2021-01-06T22:23:08.347 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:23:08.348 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":5874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":93}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:57.700077+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:23:08.352 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.349+0000 7f27fe440700 1 -- 172.21.15.203:0/2884940323 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f27d8056420 msgr2=0x7f27d80588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:08.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.349+0000 7f27fe440700 1 --2- 172.21.15.203:0/2884940323 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f27d8056420 0x7f27d80588e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f27f4000f30 tx=0x7f27f4000b80).stop 2021-01-06T22:23:08.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.349+0000 7f27fe440700 1 -- 172.21.15.203:0/2884940323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27f80b3530 msgr2=0x7f27f81522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:08.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.349+0000 7f27fe440700 1 --2- 172.21.15.203:0/2884940323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27f80b3530 0x7f27f81522c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f27f805c990 tx=0x7f27e8019000).stop 2021-01-06T22:23:08.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.349+0000 7f27fe440700 1 -- 172.21.15.203:0/2884940323 shutdown_connections 2021-01-06T22:23:08.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.349+0000 7f27fe440700 1 --2- 172.21.15.203:0/2884940323 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f27d8056420 0x7f27d80588e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:08.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.349+0000 7f27fe440700 1 --2- 172.21.15.203:0/2884940323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27f80b3530 0x7f27f81522c0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:08.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.349+0000 7f27fe440700 1 --2- 172.21.15.203:0/2884940323 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27f80a5f90 0x7f27f8132bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:08.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.349+0000 7f27fe440700 1 --2- 172.21.15.203:0/2884940323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27f80a5700 0x7f27f80b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:08.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.349+0000 7f27fe440700 1 -- 172.21.15.203:0/2884940323 >> 172.21.15.203:0/2884940323 conn(0x7f27f8056cd0 msgr2=0x7f27f8061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:08.359 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.357+0000 7f27fe440700 1 -- 172.21.15.203:0/2884940323 shutdown_connections 2021-01-06T22:23:08.359 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:08.357+0000 7f27fe440700 1 -- 172.21.15.203:0/2884940323 wait complete. 2021-01-06T22:23:08.360 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 94 2021-01-06T22:23:08.375 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:23:10.376 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:23:10.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.501+0000 7f111bfff700 1 Processor -- start 2021-01-06T22:23:10.506 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.501+0000 7f111bfff700 1 -- start start 2021-01-06T22:23:10.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111bfff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f111c14df70 0x7f111c14e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:10.507 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111bfff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f111c0ba9e0 0x7f111c14ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:10.508 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111bfff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111c0bccb0 con 0x7f111c0c0fa0 2021-01-06T22:23:10.508 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111bfff700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111c067ac0 con 0x7f111c0ba9e0 2021-01-06T22:23:10.508 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111bfff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0bcee0 con 0x7f111c14df70 2021-01-06T22:23:10.508 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111affd700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f111c14df70 0x7f111c14e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:10.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111affd700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f111c14df70 0x7f111c14e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:60108/0 (socket says 172.21.15.203:60108) 2021-01-06T22:23:10.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111affd700 1 -- 172.21.15.203:0/2402871402 learned_addr learned my addr 172.21.15.203:0/2402871402 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:23:10.509 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111b7fe700 1 --2- 172.21.15.203:0/2402871402 >> v2:172.21.15.91:3300/0 conn(0x7f111c0ba9e0 0x7f111c14ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:10.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111b7fe700 1 -- 172.21.15.203:0/2402871402 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f111c14df70 msgr2=0x7f111c14e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:23:10.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111b7fe700 1 --2- 172.21.15.203:0/2402871402 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f111c14df70 0x7f111c14e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:10.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111b7fe700 1 -- 172.21.15.203:0/2402871402 >> v1:172.21.15.91:6789/0 conn(0x7f111c0c0fa0 legacy=0x7f111c0641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:10.511 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111b7fe700 1 -- 172.21.15.203:0/2402871402 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c0bd180 con 0x7f111c0ba9e0 2021-01-06T22:23:10.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111a7fc700 1 -- 172.21.15.203:0/2402871402 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3340270945 0 0) 0x7f111c0bccb0 con 0x7f111c0c0fa0 2021-01-06T22:23:10.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111affd700 1 --2- 172.21.15.203:0/2402871402 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f111c14df70 0x7f111c14e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:23:10.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111b7fe700 1 --2- 172.21.15.203:0/2402871402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f111c0ba9e0 0x7f111c14ad90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f1114009c90 tx=0x7f11140091d0).ready entity=mon.0 client_cookie=f30465127f017a42 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:10.512 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111a7fc700 1 -- 172.21.15.203:0/2402871402 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111400bb00 con 0x7f111c0ba9e0 2021-01-06T22:23:10.513 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111a7fc700 1 -- 172.21.15.203:0/2402871402 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111400bc60 con 0x7f111c0ba9e0 2021-01-06T22:23:10.513 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111a7fc700 1 -- 172.21.15.203:0/2402871402 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114011400 con 0x7f111c0ba9e0 2021-01-06T22:23:10.513 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111bfff700 1 -- 172.21.15.203:0/2402871402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f111c0ba9e0 msgr2=0x7f111c14ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:10.513 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111bfff700 1 --2- 172.21.15.203:0/2402871402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f111c0ba9e0 0x7f111c14ad90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f1114009c90 tx=0x7f11140091d0).stop 2021-01-06T22:23:10.513 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111bfff700 1 -- 172.21.15.203:0/2402871402 shutdown_connections 2021-01-06T22:23:10.514 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111bfff700 1 --2- 172.21.15.203:0/2402871402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f111c0ba9e0 0x7f111c14ad90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:10.514 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111bfff700 1 --2- 172.21.15.203:0/2402871402 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f111c14df70 0x7f111c14e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:10.514 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111bfff700 1 -- 172.21.15.203:0/2402871402 >> 172.21.15.203:0/2402871402 conn(0x7f111c0667b0 msgr2=0x7f111c0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:10.514 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111bfff700 1 -- 172.21.15.203:0/2402871402 shutdown_connections 2021-01-06T22:23:10.514 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111bfff700 1 -- 172.21.15.203:0/2402871402 wait complete. 2021-01-06T22:23:10.515 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.505+0000 7f111bfff700 1 Processor -- start 2021-01-06T22:23:10.515 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111bfff700 1 -- start start 2021-01-06T22:23:10.517 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111bfff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f111c0ba9e0 0x7f111c15dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:10.517 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f111c0c0fa0 0x7f111c0b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:10.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111bfff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f111c14df70 0x7f111c0b9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:10.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111bfff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c151f40 con 0x7f111c14df70 2021-01-06T22:23:10.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111bfff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c15e1c0 con 0x7f111c0c0fa0 2021-01-06T22:23:10.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111bfff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c150df0 con 0x7f111c0ba9e0 2021-01-06T22:23:10.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f1113fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f111c0c0fa0 0x7f111c0b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:10.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111b7fe700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f111c14df70 0x7f111c0b9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:10.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111b7fe700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f111c14df70 0x7f111c0b9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:34652/0 (socket says 172.21.15.203:34652) 2021-01-06T22:23:10.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f1113fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f111c0c0fa0 0x7f111c0b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:55034/0 (socket says 172.21.15.203:55034) 2021-01-06T22:23:10.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111b7fe700 1 -- 172.21.15.203:0/693565714 learned_addr learned my addr 172.21.15.203:0/693565714 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:23:10.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111affd700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f111c0ba9e0 0x7f111c15dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:10.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111b7fe700 1 -- 172.21.15.203:0/693565714 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f111c0ba9e0 msgr2=0x7f111c15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:10.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111b7fe700 1 --2- 172.21.15.203:0/693565714 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f111c0ba9e0 0x7f111c15dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:10.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111b7fe700 1 -- 172.21.15.203:0/693565714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f111c0c0fa0 msgr2=0x7f111c0b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:10.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111b7fe700 1 --2- 172.21.15.203:0/693565714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f111c0c0fa0 0x7f111c0b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:10.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111b7fe700 1 -- 172.21.15.203:0/693565714 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c151020 con 0x7f111c14df70 2021-01-06T22:23:10.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111b7fe700 1 --2- 172.21.15.203:0/693565714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f111c14df70 0x7f111c0b9ba0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f1114006ef0 tx=0x7f1114017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:10.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f1112ffd700 1 -- 172.21.15.203:0/693565714 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111401c700 con 0x7f111c14df70 2021-01-06T22:23:10.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f1112ffd700 1 -- 172.21.15.203:0/693565714 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111401c860 con 0x7f111c14df70 2021-01-06T22:23:10.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f1112ffd700 1 -- 172.21.15.203:0/693565714 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111401cc00 con 0x7f111c14df70 2021-01-06T22:23:10.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111bfff700 1 -- 172.21.15.203:0/693565714 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111c1512c0 con 0x7f111c14df70 2021-01-06T22:23:10.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111bfff700 1 -- 172.21.15.203:0/693565714 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c151800 con 0x7f111c14df70 2021-01-06T22:23:10.522 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.509+0000 7f111bfff700 1 -- 172.21.15.203:0/693565714 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111c14eb30 con 0x7f111c14df70 2021-01-06T22:23:10.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.513+0000 7f1112ffd700 1 -- 172.21.15.203:0/693565714 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f111400f070 con 0x7f111c14df70 2021-01-06T22:23:10.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.513+0000 7f1112ffd700 1 --2- 172.21.15.203:0/693565714 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f11000563d0 0x7f1100058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:10.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.513+0000 7f1112ffd700 1 -- 172.21.15.203:0/693565714 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7019+0+0 (secure 0 0 0) 0x7f1114071a90 con 0x7f111c14df70 2021-01-06T22:23:10.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.513+0000 7f111affd700 1 --2- 172.21.15.203:0/693565714 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f11000563d0 0x7f1100058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:10.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.513+0000 7f111affd700 1 --2- 172.21.15.203:0/693565714 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f11000563d0 0x7f1100058890 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f110c0069e0 tx=0x7f110c006ea0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:10.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.521+0000 7f1112ffd700 1 -- 172.21.15.203:0/693565714 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f111404aa10 con 0x7f111c14df70 2021-01-06T22:23:10.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.713+0000 7f111bfff700 1 -- 172.21.15.203:0/693565714 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f111c14eb30 con 0x7f111c14df70 2021-01-06T22:23:10.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.717+0000 7f1112ffd700 1 -- 172.21.15.203:0/693565714 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+3568 (secure 0 0 0) 0x7f111c14eb30 con 0x7f111c14df70 2021-01-06T22:23:10.720 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:23:10.720 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":5874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":93}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:57.700077+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:23:10.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.717+0000 7f111bfff700 1 -- 172.21.15.203:0/693565714 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f11000563d0 msgr2=0x7f1100058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:10.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.717+0000 7f111bfff700 1 --2- 172.21.15.203:0/693565714 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f11000563d0 0x7f1100058890 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f110c0069e0 tx=0x7f110c006ea0).stop 2021-01-06T22:23:10.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.717+0000 7f111bfff700 1 -- 172.21.15.203:0/693565714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f111c14df70 msgr2=0x7f111c0b9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:10.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.717+0000 7f111bfff700 1 --2- 172.21.15.203:0/693565714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f111c14df70 0x7f111c0b9ba0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f1114006ef0 tx=0x7f1114017000).stop 2021-01-06T22:23:10.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.717+0000 7f111bfff700 1 -- 172.21.15.203:0/693565714 shutdown_connections 2021-01-06T22:23:10.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.717+0000 7f111bfff700 1 --2- 172.21.15.203:0/693565714 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f11000563d0 0x7f1100058890 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:10.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.717+0000 7f111bfff700 1 --2- 172.21.15.203:0/693565714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f111c14df70 0x7f111c0b9ba0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:10.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.717+0000 7f111bfff700 1 --2- 172.21.15.203:0/693565714 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f111c0ba9e0 0x7f111c15dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:10.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.717+0000 7f111bfff700 1 --2- 172.21.15.203:0/693565714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f111c0c0fa0 0x7f111c0b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:10.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.717+0000 7f111bfff700 1 -- 172.21.15.203:0/693565714 >> 172.21.15.203:0/693565714 conn(0x7f111c0667b0 msgr2=0x7f111c14a470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:10.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.717+0000 7f111bfff700 1 -- 172.21.15.203:0/693565714 shutdown_connections 2021-01-06T22:23:10.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:10.717+0000 7f111bfff700 1 -- 172.21.15.203:0/693565714 wait complete. 2021-01-06T22:23:10.725 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 94 2021-01-06T22:23:10.734 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:23:12.735 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:23:12.857 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.853+0000 7ff50e37d700 1 Processor -- start 2021-01-06T22:23:12.858 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.853+0000 7ff50e37d700 1 -- start start 2021-01-06T22:23:12.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.853+0000 7ff50e37d700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff5080636c0 0x7ff50814ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:12.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff50e37d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff508063aa0 0x7ff508147f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:12.860 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff50e37d700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff508060780 con 0x7ff5080602b0 2021-01-06T22:23:12.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff50e37d700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50814f340 con 0x7ff5080636c0 2021-01-06T22:23:12.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff50e37d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff508060a20 con 0x7ff508063aa0 2021-01-06T22:23:12.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff507fff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff5080636c0 0x7ff50814ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:12.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff5077fe700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff5080602b0 0x7ff5080bb650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:48376/0 (socket says 172.21.15.203:48376) 2021-01-06T22:23:12.861 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff5077fe700 1 -- 172.21.15.203:0/875345436 learned_addr learned my addr 172.21.15.203:0/875345436 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:23:12.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff50c91a700 1 --2- 172.21.15.203:0/875345436 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff508063aa0 0x7ff508147f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:12.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff507fff700 1 -- 172.21.15.203:0/875345436 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff508063aa0 msgr2=0x7ff508147f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:23:12.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff507fff700 1 --2- 172.21.15.203:0/875345436 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff508063aa0 0x7ff508147f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:12.862 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff507fff700 1 -- 172.21.15.203:0/875345436 >> v1:172.21.15.91:6789/0 conn(0x7ff5080602b0 legacy=0x7ff5080bb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:12.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff507fff700 1 -- 172.21.15.203:0/875345436 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff508060cc0 con 0x7ff5080636c0 2021-01-06T22:23:12.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff507fff700 1 --2- 172.21.15.203:0/875345436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5080636c0 0x7ff50814ee50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7ff4f800b980 tx=0x7ff4f8006de0).ready entity=mon.0 client_cookie=917d9cb2558103e1 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:12.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff506ffd700 1 -- 172.21.15.203:0/875345436 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8009840 con 0x7ff5080636c0 2021-01-06T22:23:12.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff506ffd700 1 -- 172.21.15.203:0/875345436 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f80099a0 con 0x7ff5080636c0 2021-01-06T22:23:12.864 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff506ffd700 1 -- 172.21.15.203:0/875345436 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8009d20 con 0x7ff5080636c0 2021-01-06T22:23:12.866 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff50e37d700 1 -- 172.21.15.203:0/875345436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5080636c0 msgr2=0x7ff50814ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:12.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff50e37d700 1 --2- 172.21.15.203:0/875345436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5080636c0 0x7ff50814ee50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7ff4f800b980 tx=0x7ff4f8006de0).stop 2021-01-06T22:23:12.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff50e37d700 1 -- 172.21.15.203:0/875345436 shutdown_connections 2021-01-06T22:23:12.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff50e37d700 1 --2- 172.21.15.203:0/875345436 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff508063aa0 0x7ff508147f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:12.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff50e37d700 1 --2- 172.21.15.203:0/875345436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5080636c0 0x7ff50814ee50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:12.867 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff50e37d700 1 -- 172.21.15.203:0/875345436 >> 172.21.15.203:0/875345436 conn(0x7ff5080c0cb0 msgr2=0x7ff5080c1090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:12.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff50e37d700 1 -- 172.21.15.203:0/875345436 shutdown_connections 2021-01-06T22:23:12.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.857+0000 7ff50e37d700 1 -- 172.21.15.203:0/875345436 wait complete. 2021-01-06T22:23:12.868 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50e37d700 1 Processor -- start 2021-01-06T22:23:12.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50e37d700 1 -- start start 2021-01-06T22:23:12.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50e37d700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5080602b0 0x7ff50814d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:12.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50e37d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff5080636c0 0x7ff50813bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:12.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50e37d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff508063aa0 0x7ff50813b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:12.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50e37d700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080bffd0 con 0x7ff5080602b0 2021-01-06T22:23:12.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50e37d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff508145400 con 0x7ff508063aa0 2021-01-06T22:23:12.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50e37d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50813bd20 con 0x7ff5080636c0 2021-01-06T22:23:12.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50c91a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff508063aa0 0x7ff50813b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:12.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50c91a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff508063aa0 0x7ff50813b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:55138/0 (socket says 172.21.15.203:55138) 2021-01-06T22:23:12.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50c91a700 1 -- 172.21.15.203:0/1384436798 learned_addr learned my addr 172.21.15.203:0/1384436798 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:23:12.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff507fff700 1 --2- 172.21.15.203:0/1384436798 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5080602b0 0x7ff50814d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:12.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff5077fe700 1 --2- 172.21.15.203:0/1384436798 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff5080636c0 0x7ff50813bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:12.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50c91a700 1 -- 172.21.15.203:0/1384436798 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff5080636c0 msgr2=0x7ff50813bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:12.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50c91a700 1 --2- 172.21.15.203:0/1384436798 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff5080636c0 0x7ff50813bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:12.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50c91a700 1 -- 172.21.15.203:0/1384436798 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5080602b0 msgr2=0x7ff50814d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:12.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50c91a700 1 --2- 172.21.15.203:0/1384436798 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5080602b0 0x7ff50814d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:12.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50c91a700 1 -- 172.21.15.203:0/1384436798 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5080bf7f0 con 0x7ff508063aa0 2021-01-06T22:23:12.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50c91a700 1 --2- 172.21.15.203:0/1384436798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff508063aa0 0x7ff50813b570 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7ff50000ed20 tx=0x7ff5000091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:12.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff4effff700 1 -- 172.21.15.203:0/1384436798 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500009580 con 0x7ff508063aa0 2021-01-06T22:23:12.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50e37d700 1 -- 172.21.15.203:0/1384436798 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5080bfa90 con 0x7ff508063aa0 2021-01-06T22:23:12.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff4effff700 1 -- 172.21.15.203:0/1384436798 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50000cb00 con 0x7ff508063aa0 2021-01-06T22:23:12.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff4effff700 1 -- 172.21.15.203:0/1384436798 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50000eee0 con 0x7ff508063aa0 2021-01-06T22:23:12.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.861+0000 7ff50e37d700 1 -- 172.21.15.203:0/1384436798 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50814cdc0 con 0x7ff508063aa0 2021-01-06T22:23:12.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.865+0000 7ff50e37d700 1 -- 172.21.15.203:0/1384436798 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50805d600 con 0x7ff508063aa0 2021-01-06T22:23:12.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.865+0000 7ff4effff700 1 -- 172.21.15.203:0/1384436798 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff500022070 con 0x7ff508063aa0 2021-01-06T22:23:12.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.865+0000 7ff4effff700 1 --2- 172.21.15.203:0/1384436798 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff4e80563d0 0x7ff4e8058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:12.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.865+0000 7ff4effff700 1 -- 172.21.15.203:0/1384436798 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7019+0+0 (secure 0 0 0) 0x7ff50006ff50 con 0x7ff508063aa0 2021-01-06T22:23:12.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.865+0000 7ff507fff700 1 --2- 172.21.15.203:0/1384436798 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff4e80563d0 0x7ff4e8058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:12.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.865+0000 7ff507fff700 1 --2- 172.21.15.203:0/1384436798 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff4e80563d0 0x7ff4e8058890 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7ff4f800be40 tx=0x7ff4f8016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:12.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:12.869+0000 7ff4effff700 1 -- 172.21.15.203:0/1384436798 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff500045420 con 0x7ff508063aa0 2021-01-06T22:23:13.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:13.065+0000 7ff50e37d700 1 -- 172.21.15.203:0/1384436798 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff50813ba60 con 0x7ff508063aa0 2021-01-06T22:23:13.071 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:13.069+0000 7ff4effff700 1 -- 172.21.15.203:0/1384436798 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+3568 (secure 0 0 0) 0x7ff500016030 con 0x7ff508063aa0 2021-01-06T22:23:13.072 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:23:13.072 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":5874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":93}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:57.700077+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:23:13.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:13.069+0000 7ff50e37d700 1 -- 172.21.15.203:0/1384436798 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff4e80563d0 msgr2=0x7ff4e8058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:13.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:13.069+0000 7ff50e37d700 1 --2- 172.21.15.203:0/1384436798 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff4e80563d0 0x7ff4e8058890 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7ff4f800be40 tx=0x7ff4f8016000).stop 2021-01-06T22:23:13.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:13.069+0000 7ff50e37d700 1 -- 172.21.15.203:0/1384436798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff508063aa0 msgr2=0x7ff50813b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:13.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:13.069+0000 7ff50e37d700 1 --2- 172.21.15.203:0/1384436798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff508063aa0 0x7ff50813b570 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7ff50000ed20 tx=0x7ff5000091d0).stop 2021-01-06T22:23:13.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:13.069+0000 7ff50e37d700 1 -- 172.21.15.203:0/1384436798 shutdown_connections 2021-01-06T22:23:13.075 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:13.069+0000 7ff50e37d700 1 --2- 172.21.15.203:0/1384436798 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff4e80563d0 0x7ff4e8058890 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:13.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:13.069+0000 7ff50e37d700 1 --2- 172.21.15.203:0/1384436798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff508063aa0 0x7ff50813b570 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:13.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:13.069+0000 7ff50e37d700 1 --2- 172.21.15.203:0/1384436798 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5080602b0 0x7ff50814d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:13.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:13.069+0000 7ff50e37d700 1 --2- 172.21.15.203:0/1384436798 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff5080636c0 0x7ff50813bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:13.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:13.069+0000 7ff50e37d700 1 -- 172.21.15.203:0/1384436798 >> 172.21.15.203:0/1384436798 conn(0x7ff5080c0cb0 msgr2=0x7ff50814e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:13.076 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:13.069+0000 7ff50e37d700 1 -- 172.21.15.203:0/1384436798 shutdown_connections 2021-01-06T22:23:13.077 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:13.069+0000 7ff50e37d700 1 -- 172.21.15.203:0/1384436798 wait complete. 2021-01-06T22:23:13.077 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 94 2021-01-06T22:23:13.087 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:23:15.089 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:23:15.209 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.205+0000 7f8f7a276700 1 Processor -- start 2021-01-06T22:23:15.210 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.205+0000 7f8f7a276700 1 -- start start 2021-01-06T22:23:15.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.205+0000 7f8f7a276700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8f740676d0 0x7f8f740682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:15.213 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.205+0000 7f8f7a276700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8f74067ab0 0x7f8f7414e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:15.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.205+0000 7f8f7a276700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7414e7d0 0x7f8f740ba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:15.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.205+0000 7f8f7a276700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7414dc10 con 0x7f8f74067ab0 2021-01-06T22:23:15.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.205+0000 7f8f7a276700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f740b8a50 con 0x7f8f740676d0 2021-01-06T22:23:15.214 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.205+0000 7f8f7a276700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f740b8cf0 con 0x7f8f7414e7d0 2021-01-06T22:23:15.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.205+0000 7f8f73fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8f740676d0 0x7f8f740682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:15.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.205+0000 7f8f73fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8f740676d0 0x7f8f740682c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:55212/0 (socket says 172.21.15.203:55212) 2021-01-06T22:23:15.215 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.205+0000 7f8f73fff700 1 -- 172.21.15.203:0/365254459 learned_addr learned my addr 172.21.15.203:0/365254459 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:23:15.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f737fe700 1 --2- 172.21.15.203:0/365254459 >> v2:172.21.15.91:3300/0 conn(0x7f8f74067ab0 0x7f8f7414e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:15.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f78813700 1 --2- 172.21.15.203:0/365254459 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7414e7d0 0x7f8f740ba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:15.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f737fe700 1 -- 172.21.15.203:0/365254459 >> v2:172.21.15.91:3300/0 conn(0x7f8f74067ab0 msgr2=0x7f8f7414e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2021-01-06T22:23:15.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f737fe700 1 -- 172.21.15.203:0/365254459 >> v2:172.21.15.91:3300/0 conn(0x7f8f74067ab0 msgr2=0x7f8f7414e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-01-06T22:23:15.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f737fe700 1 --2- 172.21.15.203:0/365254459 >> v2:172.21.15.91:3300/0 conn(0x7f8f74067ab0 0x7f8f7414e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-01-06T22:23:15.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f737fe700 1 --2- 172.21.15.203:0/365254459 >> v2:172.21.15.91:3300/0 conn(0x7f8f74067ab0 0x7f8f7414e2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-01-06T22:23:15.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f73fff700 1 -- 172.21.15.203:0/365254459 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7414e7d0 msgr2=0x7f8f740ba6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:23:15.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f73fff700 1 --2- 172.21.15.203:0/365254459 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7414e7d0 0x7f8f740ba6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:15.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f73fff700 1 -- 172.21.15.203:0/365254459 >> v2:172.21.15.91:3300/0 conn(0x7f8f74067ab0 msgr2=0x7f8f7414e2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-01-06T22:23:15.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f73fff700 1 --2- 172.21.15.203:0/365254459 >> v2:172.21.15.91:3300/0 conn(0x7f8f74067ab0 0x7f8f7414e2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:15.218 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f73fff700 1 -- 172.21.15.203:0/365254459 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f740b8f90 con 0x7f8f740676d0 2021-01-06T22:23:15.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f73fff700 1 --2- 172.21.15.203:0/365254459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f740676d0 0x7f8f740682c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f8f740be760 tx=0x7f8f64006ea0).ready entity=mon.1 client_cookie=d1824cd9bd012c69 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:15.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f72ffd700 1 -- 172.21.15.203:0/365254459 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64009860 con 0x7f8f740676d0 2021-01-06T22:23:15.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f72ffd700 1 -- 172.21.15.203:0/365254459 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f640099c0 con 0x7f8f740676d0 2021-01-06T22:23:15.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f72ffd700 1 -- 172.21.15.203:0/365254459 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64009d40 con 0x7f8f740676d0 2021-01-06T22:23:15.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f7a276700 1 -- 172.21.15.203:0/365254459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f740676d0 msgr2=0x7f8f740682c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:15.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f7a276700 1 --2- 172.21.15.203:0/365254459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f740676d0 0x7f8f740682c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f8f740be760 tx=0x7f8f64006ea0).stop 2021-01-06T22:23:15.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f7a276700 1 -- 172.21.15.203:0/365254459 shutdown_connections 2021-01-06T22:23:15.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f7a276700 1 --2- 172.21.15.203:0/365254459 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7414e7d0 0x7f8f740ba6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:15.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f7a276700 1 --2- 172.21.15.203:0/365254459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f740676d0 0x7f8f740682c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:15.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f7a276700 1 --2- 172.21.15.203:0/365254459 >> v2:172.21.15.91:3300/0 conn(0x7f8f74067ab0 0x7f8f7414e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:15.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f7a276700 1 -- 172.21.15.203:0/365254459 >> 172.21.15.203:0/365254459 conn(0x7f8f740c0cb0 msgr2=0x7f8f740c1090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:15.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f7a276700 1 -- 172.21.15.203:0/365254459 shutdown_connections 2021-01-06T22:23:15.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f7a276700 1 -- 172.21.15.203:0/365254459 wait complete. 2021-01-06T22:23:15.224 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f7a276700 1 Processor -- start 2021-01-06T22:23:15.224 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.209+0000 7f8f7a276700 1 -- start start 2021-01-06T22:23:15.225 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f7a276700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f740676d0 0x7f8f740b7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:15.225 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f7a276700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f74067ab0 0x7f8f740bbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:15.225 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f7a276700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f7414e7d0 0x7f8f740bc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:15.226 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f7a276700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f741463c0 con 0x7f8f74067ab0 2021-01-06T22:23:15.226 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f7a276700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f74145e80 con 0x7f8f7414e7d0 2021-01-06T22:23:15.226 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f7a276700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f74146120 con 0x7f8f740676d0 2021-01-06T22:23:15.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f78813700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f7414e7d0 0x7f8f740bc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:15.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f73fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f740676d0 0x7f8f740b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:15.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f78813700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f7414e7d0 0x7f8f740bc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:55222/0 (socket says 172.21.15.203:55222) 2021-01-06T22:23:15.227 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f78813700 1 -- 172.21.15.203:0/1427339846 learned_addr learned my addr 172.21.15.203:0/1427339846 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:23:15.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f78813700 1 -- 172.21.15.203:0/1427339846 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f740676d0 msgr2=0x7f8f740b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:15.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f78813700 1 --2- 172.21.15.203:0/1427339846 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f740676d0 0x7f8f740b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:15.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f78813700 1 -- 172.21.15.203:0/1427339846 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f74067ab0 msgr2=0x7f8f740bbe50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2021-01-06T22:23:15.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f78813700 1 --2- 172.21.15.203:0/1427339846 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f74067ab0 0x7f8f740bbe50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-01-06T22:23:15.228 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f78813700 1 -- 172.21.15.203:0/1427339846 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f74146a40 con 0x7f8f7414e7d0 2021-01-06T22:23:15.229 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f78813700 1 --2- 172.21.15.203:0/1427339846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f7414e7d0 0x7f8f740bc340 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f8f6c00ed20 tx=0x7f8f6c0091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:15.229 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f6bfff700 1 -- 172.21.15.203:0/1427339846 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c009580 con 0x7f8f7414e7d0 2021-01-06T22:23:15.229 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f6bfff700 1 -- 172.21.15.203:0/1427339846 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c00cbb0 con 0x7f8f7414e7d0 2021-01-06T22:23:15.229 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f7a276700 1 -- 172.21.15.203:0/1427339846 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f74146ce0 con 0x7f8f7414e7d0 2021-01-06T22:23:15.230 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f7a276700 1 -- 172.21.15.203:0/1427339846 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7414a8a0 con 0x7f8f7414e7d0 2021-01-06T22:23:15.230 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.213+0000 7f8f6bfff700 1 -- 172.21.15.203:0/1427339846 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c00eee0 con 0x7f8f7414e7d0 2021-01-06T22:23:15.230 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.217+0000 7f8f7a276700 1 -- 172.21.15.203:0/1427339846 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4c004f60 con 0x7f8f7414e7d0 2021-01-06T22:23:15.230 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.217+0000 7f8f6bfff700 1 -- 172.21.15.203:0/1427339846 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f8f6c022070 con 0x7f8f7414e7d0 2021-01-06T22:23:15.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.217+0000 7f8f6bfff700 1 --2- 172.21.15.203:0/1427339846 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8f58056400 0x7f8f580588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:15.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.217+0000 7f8f6bfff700 1 -- 172.21.15.203:0/1427339846 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7019+0+0 (secure 0 0 0) 0x7f8f6c06ff40 con 0x7f8f7414e7d0 2021-01-06T22:23:15.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.217+0000 7f8f73fff700 1 --2- 172.21.15.203:0/1427339846 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8f58056400 0x7f8f580588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:15.231 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.217+0000 7f8f73fff700 1 --2- 172.21.15.203:0/1427339846 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8f58056400 0x7f8f580588c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f8f6400c010 tx=0x7f8f64016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:15.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.225+0000 7f8f6bfff700 1 -- 172.21.15.203:0/1427339846 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f8f6c045410 con 0x7f8f7414e7d0 2021-01-06T22:23:15.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.561+0000 7f8f7a276700 1 -- 172.21.15.203:0/1427339846 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f4c005790 con 0x7f8f7414e7d0 2021-01-06T22:23:15.565 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.561+0000 7f8f6bfff700 1 -- 172.21.15.203:0/1427339846 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+3568 (secure 0 0 0) 0x7f8f6c016070 con 0x7f8f7414e7d0 2021-01-06T22:23:15.566 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:23:15.566 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":5874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":93}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:21:57.700077+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5632},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5632":{"gid":5632,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":16,"addr":"172.21.15.203:6837/2424499644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2424499644},{"type":"v1","addr":"172.21.15.203:6837","nonce":2424499644}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:23:15.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.565+0000 7f8f69ffb700 1 -- 172.21.15.203:0/1427339846 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8f58056400 msgr2=0x7f8f580588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:15.570 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.565+0000 7f8f69ffb700 1 --2- 172.21.15.203:0/1427339846 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8f58056400 0x7f8f580588c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f8f6400c010 tx=0x7f8f64016000).stop 2021-01-06T22:23:15.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.565+0000 7f8f69ffb700 1 -- 172.21.15.203:0/1427339846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f7414e7d0 msgr2=0x7f8f740bc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:15.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.565+0000 7f8f69ffb700 1 --2- 172.21.15.203:0/1427339846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f7414e7d0 0x7f8f740bc340 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f8f6c00ed20 tx=0x7f8f6c0091d0).stop 2021-01-06T22:23:15.571 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.565+0000 7f8f69ffb700 1 -- 172.21.15.203:0/1427339846 shutdown_connections 2021-01-06T22:23:15.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.565+0000 7f8f69ffb700 1 --2- 172.21.15.203:0/1427339846 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8f58056400 0x7f8f580588c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:15.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.565+0000 7f8f69ffb700 1 --2- 172.21.15.203:0/1427339846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f7414e7d0 0x7f8f740bc340 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:15.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.565+0000 7f8f69ffb700 1 --2- 172.21.15.203:0/1427339846 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f740676d0 0x7f8f740b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:15.572 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.565+0000 7f8f69ffb700 1 --2- 172.21.15.203:0/1427339846 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f74067ab0 0x7f8f740bbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-01-06T22:23:15.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.565+0000 7f8f69ffb700 1 -- 172.21.15.203:0/1427339846 >> 172.21.15.203:0/1427339846 conn(0x7f8f740c0cb0 msgr2=0x7f8f74062d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:15.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.565+0000 7f8f69ffb700 1 -- 172.21.15.203:0/1427339846 shutdown_connections 2021-01-06T22:23:15.573 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:15.569+0000 7f8f69ffb700 1 -- 172.21.15.203:0/1427339846 wait complete. 2021-01-06T22:23:15.574 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 94 2021-01-06T22:23:15.592 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2021-01-06T22:23:17.593 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:23:17.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.705+0000 7fc965d80700 1 Processor -- start 2021-01-06T22:23:17.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.705+0000 7fc965d80700 1 -- start start 2021-01-06T22:23:17.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.705+0000 7fc965d80700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc960064380 0x7fc9600c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:17.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.705+0000 7fc965d80700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9600bb2d0 con 0x7fc96014e310 2021-01-06T22:23:17.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.705+0000 7fc965d80700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96014a560 con 0x7fc9601478c0 2021-01-06T22:23:17.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.705+0000 7fc965d80700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96014a800 con 0x7fc960064380 2021-01-06T22:23:17.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.705+0000 7fc96557f700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fc9601478c0 0x7fc9600bb730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:53880/0 (socket says 172.21.15.203:53880) 2021-01-06T22:23:17.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.705+0000 7fc96557f700 1 -- 172.21.15.203:0/3835726399 learned_addr learned my addr 172.21.15.203:0/3835726399 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:23:17.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc964d7e700 1 --2- 172.21.15.203:0/3835726399 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc960064380 0x7fc9600c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:17.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc9577fe700 1 -- 172.21.15.203:0/3835726399 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2349920138 0 0) 0x7fc96014a560 con 0x7fc9601478c0 2021-01-06T22:23:17.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc9577fe700 1 -- 172.21.15.203:0/3835726399 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc94c003230 con 0x7fc9601478c0 2021-01-06T22:23:17.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc964d7e700 1 -- 172.21.15.203:0/3835726399 >> v1:172.21.15.203:6789/0 conn(0x7fc9601478c0 legacy=0x7fc9600bb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:17.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc964d7e700 1 -- 172.21.15.203:0/3835726399 >> v1:172.21.15.91:6789/0 conn(0x7fc96014e310 legacy=0x7fc9601471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:17.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc964d7e700 1 -- 172.21.15.203:0/3835726399 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96014aaa0 con 0x7fc960064380 2021-01-06T22:23:17.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc964d7e700 1 --2- 172.21.15.203:0/3835726399 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc960064380 0x7fc9600c0fa0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fc95c00cb60 tx=0x7fc95c006e70).ready entity=mon.2 client_cookie=89997e1251e3ffe3 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:17.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc9577fe700 1 -- 172.21.15.203:0/3835726399 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c00a6d0 con 0x7fc960064380 2021-01-06T22:23:17.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc9577fe700 1 -- 172.21.15.203:0/3835726399 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95c00a830 con 0x7fc960064380 2021-01-06T22:23:17.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc9577fe700 1 -- 172.21.15.203:0/3835726399 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c00abb0 con 0x7fc960064380 2021-01-06T22:23:17.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc965d80700 1 -- 172.21.15.203:0/3835726399 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc960064380 msgr2=0x7fc9600c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:17.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc965d80700 1 --2- 172.21.15.203:0/3835726399 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc960064380 0x7fc9600c0fa0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fc95c00cb60 tx=0x7fc95c006e70).stop 2021-01-06T22:23:17.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc965d80700 1 -- 172.21.15.203:0/3835726399 shutdown_connections 2021-01-06T22:23:17.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc965d80700 1 --2- 172.21.15.203:0/3835726399 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc960064380 0x7fc9600c0fa0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:17.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc965d80700 1 -- 172.21.15.203:0/3835726399 >> 172.21.15.203:0/3835726399 conn(0x7fc9600667b0 msgr2=0x7fc9600c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:17.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc965d80700 1 -- 172.21.15.203:0/3835726399 shutdown_connections 2021-01-06T22:23:17.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.709+0000 7fc965d80700 1 -- 172.21.15.203:0/3835726399 wait complete. 2021-01-06T22:23:17.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc965d80700 1 Processor -- start 2021-01-06T22:23:17.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc965d80700 1 -- start start 2021-01-06T22:23:17.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc965d80700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc960064380 0x7fc960148b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:17.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc965d80700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9601478c0 0x7fc9600b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:17.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc965d80700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc96014e310 0x7fc960159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:17.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc965d80700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc960147fb0 con 0x7fc9601478c0 2021-01-06T22:23:17.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc965d80700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96015a2b0 con 0x7fc96014e310 2021-01-06T22:23:17.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc965d80700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600b4710 con 0x7fc960064380 2021-01-06T22:23:17.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc96557f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc96014e310 0x7fc960159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:17.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc96557f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc96014e310 0x7fc960159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:55292/0 (socket says 172.21.15.203:55292) 2021-01-06T22:23:17.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc964d7e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc960064380 0x7fc960148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:17.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc96557f700 1 -- 172.21.15.203:0/2795108044 learned_addr learned my addr 172.21.15.203:0/2795108044 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:23:17.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc957fff700 1 --2- 172.21.15.203:0/2795108044 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9601478c0 0x7fc9600b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:17.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc96557f700 1 -- 172.21.15.203:0/2795108044 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc960064380 msgr2=0x7fc960148b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:17.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc96557f700 1 --2- 172.21.15.203:0/2795108044 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc960064380 0x7fc960148b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:17.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc96557f700 1 -- 172.21.15.203:0/2795108044 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9601478c0 msgr2=0x7fc9600b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:17.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc96557f700 1 --2- 172.21.15.203:0/2795108044 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9601478c0 0x7fc9600b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:17.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc96557f700 1 -- 172.21.15.203:0/2795108044 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc960149ac0 con 0x7fc96014e310 2021-01-06T22:23:17.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.713+0000 7fc964d7e700 1 --2- 172.21.15.203:0/2795108044 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc960064380 0x7fc960148b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:23:17.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.717+0000 7fc96557f700 1 --2- 172.21.15.203:0/2795108044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc96014e310 0x7fc960159dc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7fc958005910 tx=0x7fc958007e10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:17.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.717+0000 7fc954ff9700 1 -- 172.21.15.203:0/2795108044 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958005b70 con 0x7fc96014e310 2021-01-06T22:23:17.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.717+0000 7fc965d80700 1 -- 172.21.15.203:0/2795108044 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc960149d60 con 0x7fc96014e310 2021-01-06T22:23:17.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.717+0000 7fc954ff9700 1 -- 172.21.15.203:0/2795108044 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc958005cd0 con 0x7fc96014e310 2021-01-06T22:23:17.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.717+0000 7fc954ff9700 1 -- 172.21.15.203:0/2795108044 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958007320 con 0x7fc96014e310 2021-01-06T22:23:17.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.717+0000 7fc965d80700 1 -- 172.21.15.203:0/2795108044 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc960148440 con 0x7fc96014e310 2021-01-06T22:23:17.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.717+0000 7fc954ff9700 1 -- 172.21.15.203:0/2795108044 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fc958010030 con 0x7fc96014e310 2021-01-06T22:23:17.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.717+0000 7fc965d80700 1 -- 172.21.15.203:0/2795108044 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9600450a0 con 0x7fc96014e310 2021-01-06T22:23:17.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.717+0000 7fc954ff9700 1 --2- 172.21.15.203:0/2795108044 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc9400563d0 0x7fc940058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:17.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.717+0000 7fc954ff9700 1 -- 172.21.15.203:0/2795108044 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7105+0+0 (secure 0 0 0) 0x7fc95806f7f0 con 0x7fc96014e310 2021-01-06T22:23:17.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.717+0000 7fc964d7e700 1 --2- 172.21.15.203:0/2795108044 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc9400563d0 0x7fc940058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:17.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.717+0000 7fc964d7e700 1 --2- 172.21.15.203:0/2795108044 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc9400563d0 0x7fc940058890 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fc95c006eb0 tx=0x7fc95c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:17.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.721+0000 7fc954ff9700 1 -- 172.21.15.203:0/2795108044 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fc958044c60 con 0x7fc96014e310 2021-01-06T22:23:17.928 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.925+0000 7fc965d80700 1 -- 172.21.15.203:0/2795108044 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9600c0050 con 0x7fc96014e310 2021-01-06T22:23:17.928 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.925+0000 7fc954ff9700 1 -- 172.21.15.203:0/2795108044 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+3213 (secure 0 0 0) 0x7fc958048280 con 0x7fc96014e310 2021-01-06T22:23:17.929 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:23:17.929 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:23:17.734384+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:resolve","state_seq":26,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:23:17.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.925+0000 7fc965d80700 1 -- 172.21.15.203:0/2795108044 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc9400563d0 msgr2=0x7fc940058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:17.931 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.925+0000 7fc965d80700 1 --2- 172.21.15.203:0/2795108044 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc9400563d0 0x7fc940058890 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fc95c006eb0 tx=0x7fc95c016000).stop 2021-01-06T22:23:17.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.925+0000 7fc965d80700 1 -- 172.21.15.203:0/2795108044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc96014e310 msgr2=0x7fc960159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:17.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.925+0000 7fc965d80700 1 --2- 172.21.15.203:0/2795108044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc96014e310 0x7fc960159dc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7fc958005910 tx=0x7fc958007e10).stop 2021-01-06T22:23:17.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.925+0000 7fc965d80700 1 -- 172.21.15.203:0/2795108044 shutdown_connections 2021-01-06T22:23:17.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.925+0000 7fc965d80700 1 --2- 172.21.15.203:0/2795108044 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fc9400563d0 0x7fc940058890 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:17.932 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.925+0000 7fc965d80700 1 --2- 172.21.15.203:0/2795108044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc96014e310 0x7fc960159dc0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:17.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.925+0000 7fc965d80700 1 --2- 172.21.15.203:0/2795108044 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc960064380 0x7fc960148b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:17.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.925+0000 7fc965d80700 1 --2- 172.21.15.203:0/2795108044 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9601478c0 0x7fc9600b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:17.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.925+0000 7fc965d80700 1 -- 172.21.15.203:0/2795108044 >> 172.21.15.203:0/2795108044 conn(0x7fc9600667b0 msgr2=0x7fc960068120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:17.933 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.925+0000 7fc965d80700 1 -- 172.21.15.203:0/2795108044 shutdown_connections 2021-01-06T22:23:17.934 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:17.925+0000 7fc965d80700 1 -- 172.21.15.203:0/2795108044 wait complete. 2021-01-06T22:23:17.934 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 96 2021-01-06T22:23:17.944 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2021-01-06T22:23:17.944 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2021-01-06T22:23:17.945 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:23:18.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9abb59e700 1 Processor -- start 2021-01-06T22:23:18.052 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9abb59e700 1 -- start start 2021-01-06T22:23:18.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9abb59e700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9abc067d50 0x7f9abc064620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:18.054 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9abb59e700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9abc0bd220 con 0x7f9abc064240 2021-01-06T22:23:18.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9abb59e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9abc14e020 con 0x7f9abc064e50 2021-01-06T22:23:18.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9abb59e700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9abc14e2c0 con 0x7f9abc067d50 2021-01-06T22:23:18.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9ab3fff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f9abc064e50 0x7f9abc067640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:53904/0 (socket says 172.21.15.203:53904) 2021-01-06T22:23:18.055 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9ab3fff700 1 -- 172.21.15.203:0/1460325137 learned_addr learned my addr 172.21.15.203:0/1460325137 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:23:18.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9abad9d700 1 --2- 172.21.15.203:0/1460325137 >> v2:172.21.15.91:3300/0 conn(0x7f9abc067d50 0x7f9abc064620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:18.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9ab9d9b700 1 -- 172.21.15.203:0/1460325137 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899201934 0 0) 0x7f9abc14e020 con 0x7f9abc064e50 2021-01-06T22:23:18.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9ab9d9b700 1 -- 172.21.15.203:0/1460325137 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a9c003230 con 0x7f9abc064e50 2021-01-06T22:23:18.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9ab9d9b700 1 -- 172.21.15.203:0/1460325137 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1847422369 0 0) 0x7f9a9c003230 con 0x7f9abc064e50 2021-01-06T22:23:18.056 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9abad9d700 1 -- 172.21.15.203:0/1460325137 >> v1:172.21.15.203:6789/0 conn(0x7f9abc064e50 legacy=0x7f9abc067640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:18.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9abad9d700 1 -- 172.21.15.203:0/1460325137 >> v1:172.21.15.91:6789/0 conn(0x7f9abc064240 legacy=0x7f9abc14ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:18.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9abad9d700 1 -- 172.21.15.203:0/1460325137 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9abc14e560 con 0x7f9abc067d50 2021-01-06T22:23:18.057 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9abad9d700 1 --2- 172.21.15.203:0/1460325137 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9abc067d50 0x7f9abc064620 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f9ab4009c40 tx=0x7f9ab40091d0).ready entity=mon.0 client_cookie=5b523555772456 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:18.058 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9ab9d9b700 1 -- 172.21.15.203:0/1460325137 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa40014e0 con 0x7f9abc067d50 2021-01-06T22:23:18.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9ab9d9b700 1 -- 172.21.15.203:0/1460325137 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab400b710 con 0x7f9abc067d50 2021-01-06T22:23:18.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.049+0000 7f9ab9d9b700 1 -- 172.21.15.203:0/1460325137 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab400ba90 con 0x7f9abc067d50 2021-01-06T22:23:18.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.053+0000 7f9abb59e700 1 -- 172.21.15.203:0/1460325137 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9abc067d50 msgr2=0x7f9abc064620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:18.059 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.053+0000 7f9abb59e700 1 --2- 172.21.15.203:0/1460325137 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9abc067d50 0x7f9abc064620 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f9ab4009c40 tx=0x7f9ab40091d0).stop 2021-01-06T22:23:18.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.053+0000 7f9abb59e700 1 -- 172.21.15.203:0/1460325137 shutdown_connections 2021-01-06T22:23:18.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.053+0000 7f9abb59e700 1 --2- 172.21.15.203:0/1460325137 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9abc067d50 0x7f9abc064620 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:18.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.053+0000 7f9abb59e700 1 -- 172.21.15.203:0/1460325137 >> 172.21.15.203:0/1460325137 conn(0x7f9abc0bd3d0 msgr2=0x7f9abc0bd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:18.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.053+0000 7f9abb59e700 1 -- 172.21.15.203:0/1460325137 shutdown_connections 2021-01-06T22:23:18.060 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.053+0000 7f9abb59e700 1 -- 172.21.15.203:0/1460325137 wait complete. 2021-01-06T22:23:18.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abb59e700 1 Processor -- start 2021-01-06T22:23:18.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abb59e700 1 -- start start 2021-01-06T22:23:18.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abb59e700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9abc064240 0x7f9abc0b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:18.063 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abb59e700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9abc064e50 0x7f9abc146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:18.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abb59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9abc067d50 0x7f9abc146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:18.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abb59e700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc160be0 con 0x7f9abc064e50 2021-01-06T22:23:18.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abb59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc147e10 con 0x7f9abc067d50 2021-01-06T22:23:18.064 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abb59e700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc1517f0 con 0x7f9abc064240 2021-01-06T22:23:18.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9abc067d50 0x7f9abc146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:18.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9abc067d50 0x7f9abc146520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:55318/0 (socket says 172.21.15.203:55318) 2021-01-06T22:23:18.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abad9d700 1 -- 172.21.15.203:0/158734548 learned_addr learned my addr 172.21.15.203:0/158734548 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:23:18.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9ab3fff700 1 --2- 172.21.15.203:0/158734548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9abc064e50 0x7f9abc146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:18.065 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9aba59c700 1 --2- 172.21.15.203:0/158734548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9abc064240 0x7f9abc0b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:18.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abad9d700 1 -- 172.21.15.203:0/158734548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9abc064240 msgr2=0x7f9abc0b8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:18.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abad9d700 1 --2- 172.21.15.203:0/158734548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9abc064240 0x7f9abc0b8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:18.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abad9d700 1 -- 172.21.15.203:0/158734548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9abc064e50 msgr2=0x7f9abc146030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:18.066 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abad9d700 1 --2- 172.21.15.203:0/158734548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9abc064e50 0x7f9abc146030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:18.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abad9d700 1 -- 172.21.15.203:0/158734548 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9abc151a90 con 0x7f9abc067d50 2021-01-06T22:23:18.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abad9d700 1 --2- 172.21.15.203:0/158734548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9abc067d50 0x7f9abc146520 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f9ab400bb90 tx=0x7f9abc0bcf80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:18.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9ab27fc700 1 -- 172.21.15.203:0/158734548 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4012cd0 con 0x7f9abc067d50 2021-01-06T22:23:18.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9ab27fc700 1 -- 172.21.15.203:0/158734548 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab4012e30 con 0x7f9abc067d50 2021-01-06T22:23:18.067 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9ab27fc700 1 -- 172.21.15.203:0/158734548 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab401c630 con 0x7f9abc067d50 2021-01-06T22:23:18.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abb59e700 1 -- 172.21.15.203:0/158734548 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9abc151d30 con 0x7f9abc067d50 2021-01-06T22:23:18.068 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.057+0000 7f9abb59e700 1 -- 172.21.15.203:0/158734548 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9abc0b9110 con 0x7f9abc067d50 2021-01-06T22:23:18.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.061+0000 7f9ab27fc700 1 -- 172.21.15.203:0/158734548 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f9ab400f070 con 0x7f9abc067d50 2021-01-06T22:23:18.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.061+0000 7f9ab27fc700 1 --2- 172.21.15.203:0/158734548 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9aa00563d0 0x7f9aa0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:18.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.061+0000 7f9ab27fc700 1 -- 172.21.15.203:0/158734548 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7105+0+0 (secure 0 0 0) 0x7f9ab4071140 con 0x7f9abc067d50 2021-01-06T22:23:18.069 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.061+0000 7f9aba59c700 1 --2- 172.21.15.203:0/158734548 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9aa00563d0 0x7f9aa0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:18.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.061+0000 7f9aba59c700 1 --2- 172.21.15.203:0/158734548 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9aa00563d0 0x7f9aa0058890 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f9aac003840 tx=0x7f9aac007ac0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:18.070 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.065+0000 7f9abb59e700 1 -- 172.21.15.203:0/158734548 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9abc160d20 con 0x7f9abc067d50 2021-01-06T22:23:18.074 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.069+0000 7f9ab27fc700 1 -- 172.21.15.203:0/158734548 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f9ab40465b0 con 0x7f9abc067d50 2021-01-06T22:23:18.276 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.273+0000 7f9abb59e700 1 -- 172.21.15.203:0/158734548 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9abc0450a0 con 0x7f9abc067d50 2021-01-06T22:23:18.277 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.273+0000 7f9ab27fc700 1 -- 172.21.15.203:0/158734548 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+3213 (secure 0 0 0) 0x7f9ab4049bd0 con 0x7f9abc067d50 2021-01-06T22:23:18.277 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:23:18.277 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:23:17.734384+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:resolve","state_seq":26,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:23:18.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.273+0000 7f9abb59e700 1 -- 172.21.15.203:0/158734548 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9aa00563d0 msgr2=0x7f9aa0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:18.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.273+0000 7f9abb59e700 1 --2- 172.21.15.203:0/158734548 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9aa00563d0 0x7f9aa0058890 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f9aac003840 tx=0x7f9aac007ac0).stop 2021-01-06T22:23:18.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.273+0000 7f9abb59e700 1 -- 172.21.15.203:0/158734548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9abc067d50 msgr2=0x7f9abc146520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:18.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.273+0000 7f9abb59e700 1 --2- 172.21.15.203:0/158734548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9abc067d50 0x7f9abc146520 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f9ab400bb90 tx=0x7f9abc0bcf80).stop 2021-01-06T22:23:18.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.273+0000 7f9abb59e700 1 -- 172.21.15.203:0/158734548 shutdown_connections 2021-01-06T22:23:18.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.273+0000 7f9abb59e700 1 --2- 172.21.15.203:0/158734548 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9aa00563d0 0x7f9aa0058890 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:18.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.273+0000 7f9abb59e700 1 --2- 172.21.15.203:0/158734548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9abc067d50 0x7f9abc146520 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:18.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.273+0000 7f9abb59e700 1 --2- 172.21.15.203:0/158734548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9abc064240 0x7f9abc0b8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:18.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.273+0000 7f9abb59e700 1 --2- 172.21.15.203:0/158734548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9abc064e50 0x7f9abc146030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:18.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.273+0000 7f9abb59e700 1 -- 172.21.15.203:0/158734548 >> 172.21.15.203:0/158734548 conn(0x7f9abc0bd3d0 msgr2=0x7f9abc061690 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:18.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.273+0000 7f9abb59e700 1 -- 172.21.15.203:0/158734548 shutdown_connections 2021-01-06T22:23:18.281 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:18.273+0000 7f9abb59e700 1 -- 172.21.15.203:0/158734548 wait complete. 2021-01-06T22:23:18.282 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 96 2021-01-06T22:23:18.295 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 96, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 96, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5836": { "addr": "172.21.15.203:6835/3960786803", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6834", "nonce": 3960786803, "type": "v2" }, { "addr": "172.21.15.203:6835", "nonce": 3960786803, "type": "v1" } ] }, "export_targets": [ 1 ], "features": 4540138297136906239, "flags": 0, "gid": 5836, "incarnation": 88, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 34 }, "gid_5874": { "addr": "172.21.15.91:6836/1921314962", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6834", "nonce": 1921314962, "type": "v2" }, { "addr": "172.21.15.91:6836", "nonce": 1921314962, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5874, "incarnation": 95, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:resolve", "state_seq": 26 } }, "last_failure": 0, "last_failure_osd_epoch": 35, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2021-01-06T22:23:17.734384+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 5836, "mds_1": 5874 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.91:6837/2972061575", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6835", "nonce": 2972061575, "type": "v2" }, { "addr": "172.21.15.91:6837", "nonce": 2972061575, "type": "v1" } ] }, "epoch": 87, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5384, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 2 } ] } 2021-01-06T22:23:20.296 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:23:20.425 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.421+0000 7f49b7bb6700 1 Processor -- start 2021-01-06T22:23:20.426 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.421+0000 7f49b7bb6700 1 -- start start 2021-01-06T22:23:20.428 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.421+0000 7f49b7bb6700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49b01517a0 0x7f49b0066a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:20.428 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.421+0000 7f49b7bb6700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f49b01505a0 0x7f49b0150a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:20.428 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.421+0000 7f49b7bb6700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b0150f40 con 0x7f49b0151b80 2021-01-06T22:23:20.428 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.421+0000 7f49b7bb6700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b014bc00 con 0x7f49b01505a0 2021-01-06T22:23:20.429 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.421+0000 7f49b7bb6700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b0066f30 con 0x7f49b01517a0 2021-01-06T22:23:20.429 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.421+0000 7f49b5151700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f49b0151b80 0x7f49b00b8a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:53988/0 (socket says 172.21.15.203:53988) 2021-01-06T22:23:20.429 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.421+0000 7f49b5151700 1 -- 172.21.15.203:0/3770024329 learned_addr learned my addr 172.21.15.203:0/3770024329 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:23:20.429 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.421+0000 7f49b5952700 1 --2- 172.21.15.203:0/3770024329 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49b01517a0 0x7f49b0066a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:20.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.421+0000 7f49b6153700 1 --2- 172.21.15.203:0/3770024329 >> v2:172.21.15.91:3300/0 conn(0x7f49b01505a0 0x7f49b0150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:20.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b4950700 1 -- 172.21.15.203:0/3770024329 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261371316 0 0) 0x7f49b0150f40 con 0x7f49b0151b80 2021-01-06T22:23:20.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b4950700 1 -- 172.21.15.203:0/3770024329 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4998003230 con 0x7f49b0151b80 2021-01-06T22:23:20.430 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b5952700 1 -- 172.21.15.203:0/3770024329 >> v2:172.21.15.91:3300/0 conn(0x7f49b01505a0 msgr2=0x7f49b0150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:23:20.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b5952700 1 --2- 172.21.15.203:0/3770024329 >> v2:172.21.15.91:3300/0 conn(0x7f49b01505a0 0x7f49b0150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:20.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b5952700 1 -- 172.21.15.203:0/3770024329 >> v1:172.21.15.203:6789/0 conn(0x7f49b0151b80 legacy=0x7f49b00b8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:20.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b5952700 1 -- 172.21.15.203:0/3770024329 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b014f680 con 0x7f49b01517a0 2021-01-06T22:23:20.431 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b5952700 1 --2- 172.21.15.203:0/3770024329 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49b01517a0 0x7f49b0066a40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f49ac00b980 tx=0x7f49ac006de0).ready entity=mon.2 client_cookie=934eb7755ca48b1c server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:20.432 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b4950700 1 -- 172.21.15.203:0/3770024329 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac0096d0 con 0x7f49b01517a0 2021-01-06T22:23:20.432 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b4950700 1 -- 172.21.15.203:0/3770024329 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac009830 con 0x7f49b01517a0 2021-01-06T22:23:20.432 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b4950700 1 -- 172.21.15.203:0/3770024329 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac009bb0 con 0x7f49b01517a0 2021-01-06T22:23:20.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b7bb6700 1 -- 172.21.15.203:0/3770024329 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49b01517a0 msgr2=0x7f49b0066a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:20.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b7bb6700 1 --2- 172.21.15.203:0/3770024329 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49b01517a0 0x7f49b0066a40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f49ac00b980 tx=0x7f49ac006de0).stop 2021-01-06T22:23:20.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b7bb6700 1 -- 172.21.15.203:0/3770024329 shutdown_connections 2021-01-06T22:23:20.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b7bb6700 1 --2- 172.21.15.203:0/3770024329 >> v2:172.21.15.91:3300/0 conn(0x7f49b01505a0 0x7f49b0150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:20.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b7bb6700 1 --2- 172.21.15.203:0/3770024329 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49b01517a0 0x7f49b0066a40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:20.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b7bb6700 1 -- 172.21.15.203:0/3770024329 >> 172.21.15.203:0/3770024329 conn(0x7f49b0060d80 msgr2=0x7f49b00661a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:20.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b7bb6700 1 -- 172.21.15.203:0/3770024329 shutdown_connections 2021-01-06T22:23:20.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b7bb6700 1 -- 172.21.15.203:0/3770024329 wait complete. 2021-01-06T22:23:20.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b7bb6700 1 Processor -- start 2021-01-06T22:23:20.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b7bb6700 1 -- start start 2021-01-06T22:23:20.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b7bb6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49b01505a0 0x7f49b015e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:20.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.425+0000 7f49b7bb6700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49b01517a0 0x7f49b00bb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:20.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b7bb6700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49b0151b80 0x7f49b0063840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:20.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b7bb6700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b0148600 con 0x7f49b0151b80 2021-01-06T22:23:20.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b7bb6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b0064ef0 con 0x7f49b01505a0 2021-01-06T22:23:20.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b7bb6700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b0065190 con 0x7f49b01517a0 2021-01-06T22:23:20.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b5952700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49b01505a0 0x7f49b015e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:20.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b5952700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49b01505a0 0x7f49b015e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:55400/0 (socket says 172.21.15.203:55400) 2021-01-06T22:23:20.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b5952700 1 -- 172.21.15.203:0/3286386537 learned_addr learned my addr 172.21.15.203:0/3286386537 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:23:20.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b5151700 1 --2- 172.21.15.203:0/3286386537 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49b01517a0 0x7f49b00bb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:20.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b6153700 1 --2- 172.21.15.203:0/3286386537 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49b0151b80 0x7f49b0063840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:20.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b5952700 1 -- 172.21.15.203:0/3286386537 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49b01517a0 msgr2=0x7f49b00bb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:20.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b5952700 1 --2- 172.21.15.203:0/3286386537 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49b01517a0 0x7f49b00bb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:20.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b5952700 1 -- 172.21.15.203:0/3286386537 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49b0151b80 msgr2=0x7f49b0063840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:20.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b5952700 1 --2- 172.21.15.203:0/3286386537 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49b0151b80 0x7f49b0063840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:20.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b5952700 1 -- 172.21.15.203:0/3286386537 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b0065430 con 0x7f49b01505a0 2021-01-06T22:23:20.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b5952700 1 --2- 172.21.15.203:0/3286386537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49b01505a0 0x7f49b015e280 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7f49ac006e20 tx=0x7f49ac016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:20.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f499dffb700 1 -- 172.21.15.203:0/3286386537 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac012c50 con 0x7f49b01505a0 2021-01-06T22:23:20.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f499dffb700 1 -- 172.21.15.203:0/3286386537 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac012db0 con 0x7f49b01505a0 2021-01-06T22:23:20.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f499dffb700 1 -- 172.21.15.203:0/3286386537 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac01c650 con 0x7f49b01505a0 2021-01-06T22:23:20.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b7bb6700 1 -- 172.21.15.203:0/3286386537 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b00a3060 con 0x7f49b01505a0 2021-01-06T22:23:20.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b7bb6700 1 -- 172.21.15.203:0/3286386537 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b00a35a0 con 0x7f49b01505a0 2021-01-06T22:23:20.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f49b7bb6700 1 -- 172.21.15.203:0/3286386537 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b005d600 con 0x7f49b01505a0 2021-01-06T22:23:20.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.429+0000 7f499dffb700 1 -- 172.21.15.203:0/3286386537 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f49ac00f070 con 0x7f49b01505a0 2021-01-06T22:23:20.442 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.433+0000 7f499dffb700 1 --2- 172.21.15.203:0/3286386537 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f49900563d0 0x7f4990058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:20.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.433+0000 7f499dffb700 1 -- 172.21.15.203:0/3286386537 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7105+0+0 (secure 0 0 0) 0x7f49ac070ee0 con 0x7f49b01505a0 2021-01-06T22:23:20.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.433+0000 7f49b5151700 1 --2- 172.21.15.203:0/3286386537 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f49900563d0 0x7f4990058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:20.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.433+0000 7f49b5151700 1 --2- 172.21.15.203:0/3286386537 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f49900563d0 0x7f4990058890 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f49b00bbb30 tx=0x7f49a0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:20.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.441+0000 7f499dffb700 1 -- 172.21.15.203:0/3286386537 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f49ac046350 con 0x7f49b01505a0 2021-01-06T22:23:20.647 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.641+0000 7f49b7bb6700 1 -- 172.21.15.203:0/3286386537 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49b00636b0 con 0x7f49b01505a0 2021-01-06T22:23:20.648 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.645+0000 7f499dffb700 1 -- 172.21.15.203:0/3286386537 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+3212 (secure 0 0 0) 0x7f49ac049970 con 0x7f49b01505a0 2021-01-06T22:23:20.648 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:23:20.648 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:23:19.744484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:23:20.650 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.645+0000 7f49b7bb6700 1 -- 172.21.15.203:0/3286386537 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f49900563d0 msgr2=0x7f4990058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:20.650 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.645+0000 7f49b7bb6700 1 --2- 172.21.15.203:0/3286386537 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f49900563d0 0x7f4990058890 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f49b00bbb30 tx=0x7f49a0009040).stop 2021-01-06T22:23:20.650 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.645+0000 7f49b7bb6700 1 -- 172.21.15.203:0/3286386537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49b01505a0 msgr2=0x7f49b015e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:20.650 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.645+0000 7f49b7bb6700 1 --2- 172.21.15.203:0/3286386537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49b01505a0 0x7f49b015e280 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7f49ac006e20 tx=0x7f49ac016000).stop 2021-01-06T22:23:20.651 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.645+0000 7f49b7bb6700 1 -- 172.21.15.203:0/3286386537 shutdown_connections 2021-01-06T22:23:20.651 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.645+0000 7f49b7bb6700 1 --2- 172.21.15.203:0/3286386537 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f49900563d0 0x7f4990058890 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:20.651 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.645+0000 7f49b7bb6700 1 --2- 172.21.15.203:0/3286386537 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49b0151b80 0x7f49b0063840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:20.651 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.645+0000 7f49b7bb6700 1 --2- 172.21.15.203:0/3286386537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49b01505a0 0x7f49b015e280 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:20.651 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.645+0000 7f49b7bb6700 1 --2- 172.21.15.203:0/3286386537 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49b01517a0 0x7f49b00bb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:20.652 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.645+0000 7f49b7bb6700 1 -- 172.21.15.203:0/3286386537 >> 172.21.15.203:0/3286386537 conn(0x7f49b0060d80 msgr2=0x7f49b00c2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:20.652 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.645+0000 7f49b7bb6700 1 -- 172.21.15.203:0/3286386537 shutdown_connections 2021-01-06T22:23:20.652 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:20.645+0000 7f49b7bb6700 1 -- 172.21.15.203:0/3286386537 wait complete. 2021-01-06T22:23:20.652 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 98 2021-01-06T22:23:20.663 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:23:21.474 INFO:tasks.ceph.mds.c.smithi203.stderr:2021-01-06T22:23:21.469+0000 7f5acf83b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2021-01-06T22:23:22.591 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:23:22.855 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:23:22.858 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:23:23.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:22.997+0000 7f21dccd4700 1 Processor -- start 2021-01-06T22:23:23.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:22.997+0000 7f21dccd4700 1 -- start start 2021-01-06T22:23:23.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21dccd4700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f21d80c0160 0x7f21d80a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:23.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21dccd4700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21d80c0540 0x7f21d81521a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:23.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21dccd4700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d80bac00 con 0x7f21d8068420 2021-01-06T22:23:23.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21dccd4700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d80a6340 con 0x7f21d80c0160 2021-01-06T22:23:23.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21dccd4700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80668d0 con 0x7f21d80c0540 2021-01-06T22:23:23.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21d7fff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f21d8068420 0x7f21d814f350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:54066/0 (socket says 172.21.15.203:54066) 2021-01-06T22:23:23.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21d7fff700 1 -- 172.21.15.203:0/1657453479 learned_addr learned my addr 172.21.15.203:0/1657453479 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:23:23.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21d6ffd700 1 --2- 172.21.15.203:0/1657453479 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21d80c0540 0x7f21d81521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:23.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21d77fe700 1 --2- 172.21.15.203:0/1657453479 >> v2:172.21.15.91:3300/0 conn(0x7f21d80c0160 0x7f21d80a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:23.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21d67fc700 1 -- 172.21.15.203:0/1657453479 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155704293 0 0) 0x7f21d80bac00 con 0x7f21d8068420 2021-01-06T22:23:23.006 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21d67fc700 1 -- 172.21.15.203:0/1657453479 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21c4003230 con 0x7f21d8068420 2021-01-06T22:23:23.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21d77fe700 1 -- 172.21.15.203:0/1657453479 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21d80c0540 msgr2=0x7f21d81521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:23:23.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21d77fe700 1 --2- 172.21.15.203:0/1657453479 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21d80c0540 0x7f21d81521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:23.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21d77fe700 1 -- 172.21.15.203:0/1657453479 >> v1:172.21.15.203:6789/0 conn(0x7f21d8068420 legacy=0x7f21d814f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:23.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21d77fe700 1 -- 172.21.15.203:0/1657453479 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8066b70 con 0x7f21d80c0160 2021-01-06T22:23:23.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21d6ffd700 1 --2- 172.21.15.203:0/1657453479 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21d80c0540 0x7f21d81521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:23:23.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21d77fe700 1 --2- 172.21.15.203:0/1657453479 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21d80c0160 0x7f21d80a5e50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f21d0009c90 tx=0x7f21d00091d0).ready entity=mon.0 client_cookie=8b7e844790452dcc server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:23.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21d67fc700 1 -- 172.21.15.203:0/1657453479 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d000ba50 con 0x7f21d80c0160 2021-01-06T22:23:23.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21d67fc700 1 -- 172.21.15.203:0/1657453479 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d000bbb0 con 0x7f21d80c0160 2021-01-06T22:23:23.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21d67fc700 1 -- 172.21.15.203:0/1657453479 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d0011400 con 0x7f21d80c0160 2021-01-06T22:23:23.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21dccd4700 1 -- 172.21.15.203:0/1657453479 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21d80c0160 msgr2=0x7f21d80a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:23.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21dccd4700 1 --2- 172.21.15.203:0/1657453479 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21d80c0160 0x7f21d80a5e50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f21d0009c90 tx=0x7f21d00091d0).stop 2021-01-06T22:23:23.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21dccd4700 1 -- 172.21.15.203:0/1657453479 shutdown_connections 2021-01-06T22:23:23.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21dccd4700 1 --2- 172.21.15.203:0/1657453479 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21d80c0160 0x7f21d80a5e50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:23.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21dccd4700 1 --2- 172.21.15.203:0/1657453479 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21d80c0540 0x7f21d81521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:23.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21dccd4700 1 -- 172.21.15.203:0/1657453479 >> 172.21.15.203:0/1657453479 conn(0x7f21d8056cd0 msgr2=0x7f21d8061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:23.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21dccd4700 1 -- 172.21.15.203:0/1657453479 shutdown_connections 2021-01-06T22:23:23.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21dccd4700 1 -- 172.21.15.203:0/1657453479 wait complete. 2021-01-06T22:23:23.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21dccd4700 1 Processor -- start 2021-01-06T22:23:23.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.001+0000 7f21dccd4700 1 -- start start 2021-01-06T22:23:23.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21dccd4700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21d8068420 0x7f21d814fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:23.020 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21dccd4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21d80c0160 0x7f21d80bcf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:23.021 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21dccd4700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21d80c0540 0x7f21d8161990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:23.021 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21dccd4700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d8164e10 con 0x7f21d8068420 2021-01-06T22:23:23.021 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21dccd4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d8165040 con 0x7f21d80c0160 2021-01-06T22:23:23.021 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21dccd4700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d81652e0 con 0x7f21d80c0540 2021-01-06T22:23:23.021 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21d7fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21d80c0540 0x7f21d8161990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:23.022 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21d77fe700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21d8068420 0x7f21d814fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:23.022 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21d7fff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21d80c0540 0x7f21d8161990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:60558/0 (socket says 172.21.15.203:60558) 2021-01-06T22:23:23.022 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21d7fff700 1 -- 172.21.15.203:0/3771534854 learned_addr learned my addr 172.21.15.203:0/3771534854 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:23:23.022 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21d6ffd700 1 --2- 172.21.15.203:0/3771534854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21d80c0160 0x7f21d80bcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:23.022 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21d7fff700 1 -- 172.21.15.203:0/3771534854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21d80c0160 msgr2=0x7f21d80bcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:23.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21d7fff700 1 --2- 172.21.15.203:0/3771534854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21d80c0160 0x7f21d80bcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:23.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21d7fff700 1 -- 172.21.15.203:0/3771534854 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21d8068420 msgr2=0x7f21d814fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:23.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21d7fff700 1 --2- 172.21.15.203:0/3771534854 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21d8068420 0x7f21d814fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:23.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21d7fff700 1 -- 172.21.15.203:0/3771534854 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8165580 con 0x7f21d80c0540 2021-01-06T22:23:23.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21d77fe700 1 --2- 172.21.15.203:0/3771534854 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21d8068420 0x7f21d814fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:23:23.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21d7fff700 1 --2- 172.21.15.203:0/3771534854 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21d80c0540 0x7f21d8161990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f21cc00c6e0 tx=0x7f21cc007a90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:23.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21bf7fe700 1 -- 172.21.15.203:0/3771534854 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0072e0 con 0x7f21d80c0540 2021-01-06T22:23:23.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21bf7fe700 1 -- 172.21.15.203:0/3771534854 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc007440 con 0x7f21d80c0540 2021-01-06T22:23:23.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21dccd4700 1 -- 172.21.15.203:0/3771534854 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d8165820 con 0x7f21d80c0540 2021-01-06T22:23:23.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21bf7fe700 1 -- 172.21.15.203:0/3771534854 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0057f0 con 0x7f21d80c0540 2021-01-06T22:23:23.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.005+0000 7f21dccd4700 1 -- 172.21.15.203:0/3771534854 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d8153550 con 0x7f21d80c0540 2021-01-06T22:23:23.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.009+0000 7f21bf7fe700 1 -- 172.21.15.203:0/3771534854 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f21cc010070 con 0x7f21d80c0540 2021-01-06T22:23:23.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.009+0000 7f21bf7fe700 1 --2- 172.21.15.203:0/3771534854 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f21b80563d0 0x7f21b8058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:23.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.009+0000 7f21dccd4700 1 -- 172.21.15.203:0/3771534854 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d80450a0 con 0x7f21d80c0540 2021-01-06T22:23:23.026 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.009+0000 7f21bf7fe700 1 -- 172.21.15.203:0/3771534854 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7105+0+0 (secure 0 0 0) 0x7f21cc06f090 con 0x7f21d80c0540 2021-01-06T22:23:23.026 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.009+0000 7f21d77fe700 1 --2- 172.21.15.203:0/3771534854 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f21b80563d0 0x7f21b8058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:23.026 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.009+0000 7f21d77fe700 1 --2- 172.21.15.203:0/3771534854 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f21b80563d0 0x7f21b8058890 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f21d0009210 tx=0x7f21d0017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:23.026 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.021+0000 7f21bf7fe700 1 -- 172.21.15.203:0/3771534854 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f21cc044480 con 0x7f21d80c0540 2021-01-06T22:23:23.269 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.261+0000 7f21dccd4700 1 -- 172.21.15.203:0/3771534854 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21d80b7200 con 0x7f21d80c0540 2021-01-06T22:23:23.270 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.261+0000 7f21bf7fe700 1 -- 172.21.15.203:0/3771534854 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+3212 (secure 0 0 0) 0x7f21cc007bd0 con 0x7f21d80c0540 2021-01-06T22:23:23.271 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:23:23.272 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":99,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:23:20.754099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:23:23.276 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.265+0000 7f21dccd4700 1 -- 172.21.15.203:0/3771534854 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f21b80563d0 msgr2=0x7f21b8058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:23.278 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.265+0000 7f21dccd4700 1 --2- 172.21.15.203:0/3771534854 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f21b80563d0 0x7f21b8058890 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f21d0009210 tx=0x7f21d0017000).stop 2021-01-06T22:23:23.278 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.265+0000 7f21dccd4700 1 -- 172.21.15.203:0/3771534854 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21d80c0540 msgr2=0x7f21d8161990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:23.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.265+0000 7f21dccd4700 1 --2- 172.21.15.203:0/3771534854 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21d80c0540 0x7f21d8161990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f21cc00c6e0 tx=0x7f21cc007a90).stop 2021-01-06T22:23:23.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.265+0000 7f21dccd4700 1 -- 172.21.15.203:0/3771534854 shutdown_connections 2021-01-06T22:23:23.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.265+0000 7f21dccd4700 1 --2- 172.21.15.203:0/3771534854 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f21b80563d0 0x7f21b8058890 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:23.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.265+0000 7f21dccd4700 1 --2- 172.21.15.203:0/3771534854 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21d80c0540 0x7f21d8161990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:23.279 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.265+0000 7f21dccd4700 1 --2- 172.21.15.203:0/3771534854 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21d8068420 0x7f21d814fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:23.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.265+0000 7f21dccd4700 1 --2- 172.21.15.203:0/3771534854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21d80c0160 0x7f21d80bcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:23.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.265+0000 7f21dccd4700 1 -- 172.21.15.203:0/3771534854 >> 172.21.15.203:0/3771534854 conn(0x7f21d8056cd0 msgr2=0x7f21d80bbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:23.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.265+0000 7f21dccd4700 1 -- 172.21.15.203:0/3771534854 shutdown_connections 2021-01-06T22:23:23.280 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:23.265+0000 7f21dccd4700 1 -- 172.21.15.203:0/3771534854 wait complete. 2021-01-06T22:23:23.280 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 99 2021-01-06T22:23:23.283 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=1, replacing gid=5632 2021-01-06T22:23:23.284 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.a 2021-01-06T22:23:28.284 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2021-01-06T22:23:28.285 INFO:tasks.ceph.mds.a:Restarting daemon 2021-01-06T22:23:28.285 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-01-06T22:23:28.287 INFO:tasks.ceph.mds.a:Started 2021-01-06T22:23:28.288 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:23:28.387 INFO:tasks.ceph.mds.a.smithi203.stdout:starting mds.a at 2021-01-06T22:23:28.433 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f98ab9b5700 1 Processor -- start 2021-01-06T22:23:28.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f98ab9b5700 1 -- start start 2021-01-06T22:23:28.434 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f98ab9b5700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98a415eba0 0x7f98a415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:28.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f98ab9b5700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f98a40b7850 0x7f98a40c03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:28.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f98ab9b5700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a415ef80 con 0x7f98a40c0890 2021-01-06T22:23:28.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f98ab9b5700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a4064220 con 0x7f98a40b7850 2021-01-06T22:23:28.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f98ab9b5700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40644c0 con 0x7f98a415eba0 2021-01-06T22:23:28.435 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f98a9751700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98a415eba0 0x7f98a415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:28.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f98a9f52700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f98a40c0890 0x7f98a40803a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:54244/0 (socket says 172.21.15.203:54244) 2021-01-06T22:23:28.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f98a9751700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98a415eba0 0x7f98a415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:60728/0 (socket says 172.21.15.203:60728) 2021-01-06T22:23:28.436 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f98a9f52700 1 -- 172.21.15.203:0/2762980792 learned_addr learned my addr 172.21.15.203:0/2762980792 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:23:28.437 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f989bfff700 1 -- 172.21.15.203:0/2762980792 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350656056 0 0) 0x7f98a415ef80 con 0x7f98a40c0890 2021-01-06T22:23:28.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f989bfff700 1 -- 172.21.15.203:0/2762980792 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9888003230 con 0x7f98a40c0890 2021-01-06T22:23:28.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f98a9751700 1 -- 172.21.15.203:0/2762980792 >> v2:172.21.15.203:3300/0 conn(0x7f98a40b7850 msgr2=0x7f98a40c03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:23:28.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f98a9751700 1 --2- 172.21.15.203:0/2762980792 >> v2:172.21.15.203:3300/0 conn(0x7f98a40b7850 0x7f98a40c03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:23:28.438 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f98a9751700 1 -- 172.21.15.203:0/2762980792 >> v1:172.21.15.203:6789/0 conn(0x7f98a40c0890 legacy=0x7f98a40803a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:28.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f98a9751700 1 -- 172.21.15.203:0/2762980792 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a415e1c0 con 0x7f98a415eba0 2021-01-06T22:23:28.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f98a9751700 1 --2- 172.21.15.203:0/2762980792 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98a415eba0 0x7f98a415abd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f98a00073c0 tx=0x7f98a0006e70).ready entity=mon.2 client_cookie=d986d1ccde57fbe0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:28.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f989bfff700 1 -- 172.21.15.203:0/2762980792 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a00106e0 con 0x7f98a415eba0 2021-01-06T22:23:28.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f989bfff700 1 -- 172.21.15.203:0/2762980792 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a0010840 con 0x7f98a415eba0 2021-01-06T22:23:28.439 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.429+0000 7f989bfff700 1 -- 172.21.15.203:0/2762980792 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0010bc0 con 0x7f98a415eba0 2021-01-06T22:23:28.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.433+0000 7f98ab9b5700 1 -- 172.21.15.203:0/2762980792 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98a415eba0 msgr2=0x7f98a415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:28.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.433+0000 7f98ab9b5700 1 --2- 172.21.15.203:0/2762980792 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98a415eba0 0x7f98a415abd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f98a00073c0 tx=0x7f98a0006e70).stop 2021-01-06T22:23:28.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.433+0000 7f98ab9b5700 1 -- 172.21.15.203:0/2762980792 shutdown_connections 2021-01-06T22:23:28.440 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.433+0000 7f98ab9b5700 1 --2- 172.21.15.203:0/2762980792 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98a415eba0 0x7f98a415abd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:28.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.433+0000 7f98ab9b5700 1 --2- 172.21.15.203:0/2762980792 >> v2:172.21.15.203:3300/0 conn(0x7f98a40b7850 0x7f98a40c03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-01-06T22:23:28.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.433+0000 7f98ab9b5700 1 -- 172.21.15.203:0/2762980792 >> 172.21.15.203:0/2762980792 conn(0x7f98a4056cd0 msgr2=0x7f98a40626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:28.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.433+0000 7f98ab9b5700 1 -- 172.21.15.203:0/2762980792 shutdown_connections 2021-01-06T22:23:28.441 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.433+0000 7f98ab9b5700 1 -- 172.21.15.203:0/2762980792 wait complete. 2021-01-06T22:23:28.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98ab9b5700 1 Processor -- start 2021-01-06T22:23:28.443 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98ab9b5700 1 -- start start 2021-01-06T22:23:28.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98ab9b5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98a40b7850 0x7f98a40b8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:28.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98ab9b5700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f98a40c0890 0x7f98a4171e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:28.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98ab9b5700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98a415eba0 0x7f98a41529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:28.444 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98ab9b5700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a41559e0 con 0x7f98a40c0890 2021-01-06T22:23:28.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98ab9b5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a41592c0 con 0x7f98a40b7850 2021-01-06T22:23:28.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98ab9b5700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a41727a0 con 0x7f98a415eba0 2021-01-06T22:23:28.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98a9751700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98a40b7850 0x7f98a40b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:28.445 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98a9751700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98a40b7850 0x7f98a40b8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:55652/0 (socket says 172.21.15.203:55652) 2021-01-06T22:23:28.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98a9751700 1 -- 172.21.15.203:0/3854440 learned_addr learned my addr 172.21.15.203:0/3854440 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:23:28.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98a9f52700 1 --2- 172.21.15.203:0/3854440 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98a415eba0 0x7f98a41529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:28.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98a8f50700 1 --2- 172.21.15.203:0/3854440 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f98a40c0890 0x7f98a4171e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:28.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98a9751700 1 -- 172.21.15.203:0/3854440 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98a415eba0 msgr2=0x7f98a41529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:28.446 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98a9751700 1 --2- 172.21.15.203:0/3854440 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98a415eba0 0x7f98a41529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:28.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98a9751700 1 -- 172.21.15.203:0/3854440 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f98a40c0890 msgr2=0x7f98a4171e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:28.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98a9751700 1 --2- 172.21.15.203:0/3854440 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f98a40c0890 0x7f98a4171e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:28.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98a9751700 1 -- 172.21.15.203:0/3854440 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a4172a40 con 0x7f98a40b7850 2021-01-06T22:23:28.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98a9751700 1 --2- 172.21.15.203:0/3854440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98a40b7850 0x7f98a40b8010 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f98a0010ce0 tx=0x7f98a0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:28.447 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98997fa700 1 -- 172.21.15.203:0/3854440 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a000a930 con 0x7f98a40b7850 2021-01-06T22:23:28.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98997fa700 1 -- 172.21.15.203:0/3854440 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a0012480 con 0x7f98a40b7850 2021-01-06T22:23:28.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98997fa700 1 -- 172.21.15.203:0/3854440 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0012800 con 0x7f98a40b7850 2021-01-06T22:23:28.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98ab9b5700 1 -- 172.21.15.203:0/3854440 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a416f1c0 con 0x7f98a40b7850 2021-01-06T22:23:28.448 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.437+0000 7f98ab9b5700 1 -- 172.21.15.203:0/3854440 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a4172320 con 0x7f98a40b7850 2021-01-06T22:23:28.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.441+0000 7f98ab9b5700 1 -- 172.21.15.203:0/3854440 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a4172510 con 0x7f98a40b7850 2021-01-06T22:23:28.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.441+0000 7f98997fa700 1 -- 172.21.15.203:0/3854440 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f98a0012c30 con 0x7f98a40b7850 2021-01-06T22:23:28.449 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.441+0000 7f98997fa700 1 --2- 172.21.15.203:0/3854440 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9884056420 0x7f98840588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:28.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.441+0000 7f98997fa700 1 -- 172.21.15.203:0/3854440 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7105+0+0 (secure 0 0 0) 0x7f98a0071460 con 0x7f98a40b7850 2021-01-06T22:23:28.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.441+0000 7f98a8f50700 1 --2- 172.21.15.203:0/3854440 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9884056420 0x7f98840588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:28.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.441+0000 7f98a8f50700 1 --2- 172.21.15.203:0/3854440 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9884056420 0x7f98840588e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f98a4172d00 tx=0x7f9894008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:28.450 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.445+0000 7f98997fa700 1 -- 172.21.15.203:0/3854440 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f98a0046950 con 0x7f98a40b7850 2021-01-06T22:23:28.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.713+0000 7f98ab9b5700 1 -- 172.21.15.203:0/3854440 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98a405cfe0 con 0x7f98a40b7850 2021-01-06T22:23:28.719 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.713+0000 7f98997fa700 1 -- 172.21.15.203:0/3854440 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+3212 (secure 0 0 0) 0x7f98a0017020 con 0x7f98a40b7850 2021-01-06T22:23:28.719 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:23:28.719 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":99,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:23:20.754099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:23:28.720 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.717+0000 7f98ab9b5700 1 -- 172.21.15.203:0/3854440 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9884056420 msgr2=0x7f98840588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:28.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.717+0000 7f98ab9b5700 1 --2- 172.21.15.203:0/3854440 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9884056420 0x7f98840588e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f98a4172d00 tx=0x7f9894008040).stop 2021-01-06T22:23:28.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.717+0000 7f98ab9b5700 1 -- 172.21.15.203:0/3854440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98a40b7850 msgr2=0x7f98a40b8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:28.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.717+0000 7f98ab9b5700 1 --2- 172.21.15.203:0/3854440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98a40b7850 0x7f98a40b8010 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f98a0010ce0 tx=0x7f98a0015000).stop 2021-01-06T22:23:28.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.717+0000 7f98ab9b5700 1 -- 172.21.15.203:0/3854440 shutdown_connections 2021-01-06T22:23:28.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.717+0000 7f98ab9b5700 1 --2- 172.21.15.203:0/3854440 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f9884056420 0x7f98840588e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:28.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.717+0000 7f98ab9b5700 1 --2- 172.21.15.203:0/3854440 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98a415eba0 0x7f98a41529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:28.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.717+0000 7f98ab9b5700 1 --2- 172.21.15.203:0/3854440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98a40b7850 0x7f98a40b8010 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:28.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.717+0000 7f98ab9b5700 1 --2- 172.21.15.203:0/3854440 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f98a40c0890 0x7f98a4171e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:28.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.717+0000 7f98ab9b5700 1 -- 172.21.15.203:0/3854440 >> 172.21.15.203:0/3854440 conn(0x7f98a4056cd0 msgr2=0x7f98a40bd430 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:28.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.717+0000 7f98ab9b5700 1 -- 172.21.15.203:0/3854440 shutdown_connections 2021-01-06T22:23:28.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:28.717+0000 7f98ab9b5700 1 -- 172.21.15.203:0/3854440 wait complete. 2021-01-06T22:23:28.723 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 99 2021-01-06T22:23:28.733 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2021-01-06T22:23:30.734 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:23:30.940 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.933+0000 7f79595c4700 1 Processor -- start 2021-01-06T22:23:30.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.937+0000 7f79595c4700 1 -- start start 2021-01-06T22:23:30.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.937+0000 7f79595c4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f79540bb3c0 0x7f7954155970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:30.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.937+0000 7f79595c4700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79540bbb80 con 0x7f795414f530 2021-01-06T22:23:30.944 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.937+0000 7f79595c4700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79540bbe20 con 0x7f795414f910 2021-01-06T22:23:30.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.937+0000 7f79595c4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79540bc0c0 con 0x7f79540bb3c0 2021-01-06T22:23:30.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.937+0000 7f7952ffd700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f795414f530 0x7f79540660b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:48980/0 (socket says 172.21.15.203:48980) 2021-01-06T22:23:30.945 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.937+0000 7f7952ffd700 1 -- 172.21.15.203:0/1252143267 learned_addr learned my addr 172.21.15.203:0/1252143267 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:23:30.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.941+0000 7f79537fe700 1 --2- 172.21.15.203:0/1252143267 >> v2:172.21.15.203:3300/0 conn(0x7f79540bb3c0 0x7f7954155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:30.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.941+0000 7f79537fe700 1 -- 172.21.15.203:0/1252143267 >> v1:172.21.15.203:6789/0 conn(0x7f795414f910 legacy=0x7f79540bffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:30.947 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.941+0000 7f79537fe700 1 -- 172.21.15.203:0/1252143267 >> v1:172.21.15.91:6789/0 conn(0x7f795414f530 legacy=0x7f79540660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:30.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.941+0000 7f79537fe700 1 -- 172.21.15.203:0/1252143267 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79540bc360 con 0x7f79540bb3c0 2021-01-06T22:23:30.948 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.941+0000 7f79537fe700 1 --2- 172.21.15.203:0/1252143267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79540bb3c0 0x7f7954155970 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f794c009ef0 tx=0x7f794c0091d0).ready entity=mon.1 client_cookie=87a4a44a75abf3b7 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:30.966 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.961+0000 7f79527fc700 1 -- 172.21.15.203:0/1252143267 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c00baa0 con 0x7f79540bb3c0 2021-01-06T22:23:30.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.961+0000 7f79527fc700 1 -- 172.21.15.203:0/1252143267 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c00bc00 con 0x7f79540bb3c0 2021-01-06T22:23:30.967 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.961+0000 7f79527fc700 1 -- 172.21.15.203:0/1252143267 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c011400 con 0x7f79540bb3c0 2021-01-06T22:23:30.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.965+0000 7f79595c4700 1 -- 172.21.15.203:0/1252143267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79540bb3c0 msgr2=0x7f7954155970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:30.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.965+0000 7f79595c4700 1 --2- 172.21.15.203:0/1252143267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79540bb3c0 0x7f7954155970 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f794c009ef0 tx=0x7f794c0091d0).stop 2021-01-06T22:23:30.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.965+0000 7f79595c4700 1 -- 172.21.15.203:0/1252143267 shutdown_connections 2021-01-06T22:23:30.968 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.965+0000 7f79595c4700 1 --2- 172.21.15.203:0/1252143267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79540bb3c0 0x7f7954155970 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:30.969 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.965+0000 7f79595c4700 1 -- 172.21.15.203:0/1252143267 >> 172.21.15.203:0/1252143267 conn(0x7f79540626e0 msgr2=0x7f7954062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:30.969 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.965+0000 7f79595c4700 1 -- 172.21.15.203:0/1252143267 shutdown_connections 2021-01-06T22:23:30.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.965+0000 7f79595c4700 1 -- 172.21.15.203:0/1252143267 wait complete. 2021-01-06T22:23:30.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79595c4700 1 Processor -- start 2021-01-06T22:23:30.971 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79595c4700 1 -- start start 2021-01-06T22:23:30.972 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79595c4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79540bb3c0 0x7f7954162790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:30.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79595c4700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f795414f530 0x7f79541507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:30.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79595c4700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f795414f910 0x7f7954165450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:30.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79595c4700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795414eec0 con 0x7f795414f910 2021-01-06T22:23:30.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79595c4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7954066600 con 0x7f79540bb3c0 2021-01-06T22:23:30.973 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79595c4700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79540668a0 con 0x7f795414f530 2021-01-06T22:23:30.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f7952ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79540bb3c0 0x7f7954162790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:30.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f7952ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79540bb3c0 0x7f7954162790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:55742/0 (socket says 172.21.15.203:55742) 2021-01-06T22:23:30.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f7952ffd700 1 -- 172.21.15.203:0/825101827 learned_addr learned my addr 172.21.15.203:0/825101827 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:23:30.974 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79537fe700 1 --2- 172.21.15.203:0/825101827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f795414f910 0x7f7954165450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:30.975 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79537fe700 1 -- 172.21.15.203:0/825101827 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f795414f530 msgr2=0x7f79541507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:30.976 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79537fe700 1 --2- 172.21.15.203:0/825101827 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f795414f530 0x7f79541507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-01-06T22:23:30.976 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79537fe700 1 -- 172.21.15.203:0/825101827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79540bb3c0 msgr2=0x7f7954162790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:30.976 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79537fe700 1 --2- 172.21.15.203:0/825101827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79540bb3c0 0x7f7954162790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:30.976 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79537fe700 1 -- 172.21.15.203:0/825101827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79540b7540 con 0x7f795414f910 2021-01-06T22:23:30.976 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79537fe700 1 --2- 172.21.15.203:0/825101827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f795414f910 0x7f7954165450 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f794c009ef0 tx=0x7f794c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:30.977 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f794b7fe700 1 -- 172.21.15.203:0/825101827 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c010780 con 0x7f795414f910 2021-01-06T22:23:30.977 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79595c4700 1 -- 172.21.15.203:0/825101827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79540b77e0 con 0x7f795414f910 2021-01-06T22:23:30.977 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f79595c4700 1 -- 172.21.15.203:0/825101827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7954150cc0 con 0x7f795414f910 2021-01-06T22:23:30.977 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f794b7fe700 1 -- 172.21.15.203:0/825101827 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c010af0 con 0x7f795414f910 2021-01-06T22:23:30.978 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.969+0000 7f794b7fe700 1 -- 172.21.15.203:0/825101827 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c010c60 con 0x7f795414f910 2021-01-06T22:23:30.979 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.973+0000 7f794b7fe700 1 -- 172.21.15.203:0/825101827 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f794c00f070 con 0x7f795414f910 2021-01-06T22:23:30.979 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.973+0000 7f794b7fe700 1 --2- 172.21.15.203:0/825101827 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f79380563d0 0x7f7938058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:23:30.979 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.973+0000 7f794b7fe700 1 -- 172.21.15.203:0/825101827 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7105+0+0 (secure 0 0 0) 0x7f794c0720f0 con 0x7f795414f910 2021-01-06T22:23:30.979 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.973+0000 7f7952ffd700 1 --2- 172.21.15.203:0/825101827 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f79380563d0 0x7f7938058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:23:30.980 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.973+0000 7f7952ffd700 1 --2- 172.21.15.203:0/825101827 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f79380563d0 0x7f7938058890 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f793c0016f0 tx=0x7f793c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:23:30.983 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.981+0000 7f79595c4700 1 -- 172.21.15.203:0/825101827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f792c004f60 con 0x7f795414f910 2021-01-06T22:23:30.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:30.985+0000 7f794b7fe700 1 -- 172.21.15.203:0/825101827 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f794c047450 con 0x7f795414f910 2021-01-06T22:23:31.216 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:31.213+0000 7f79595c4700 1 -- 172.21.15.203:0/825101827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f792c005790 con 0x7f795414f910 2021-01-06T22:23:31.217 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:31.213+0000 7f794b7fe700 1 -- 172.21.15.203:0/825101827 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+3570 (secure 0 0 0) 0x7f794c04aa70 con 0x7f795414f910 2021-01-06T22:23:31.217 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:23:31.217 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":6052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3262828783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3262828783},{"type":"v1","addr":"172.21.15.203:6837","nonce":3262828783}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:23:20.754099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:23:31.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:31.213+0000 7f79595c4700 1 -- 172.21.15.203:0/825101827 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f79380563d0 msgr2=0x7f7938058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:31.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:31.213+0000 7f79595c4700 1 --2- 172.21.15.203:0/825101827 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f79380563d0 0x7f7938058890 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f793c0016f0 tx=0x7f793c009040).stop 2021-01-06T22:23:31.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:31.213+0000 7f79595c4700 1 -- 172.21.15.203:0/825101827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f795414f910 msgr2=0x7f7954165450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:23:31.219 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:31.213+0000 7f79595c4700 1 --2- 172.21.15.203:0/825101827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f795414f910 0x7f7954165450 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f794c009ef0 tx=0x7f794c017000).stop 2021-01-06T22:23:31.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:31.213+0000 7f79595c4700 1 -- 172.21.15.203:0/825101827 shutdown_connections 2021-01-06T22:23:31.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:31.213+0000 7f79595c4700 1 --2- 172.21.15.203:0/825101827 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f79380563d0 0x7f7938058890 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:31.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:31.213+0000 7f79595c4700 1 --2- 172.21.15.203:0/825101827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f795414f910 0x7f7954165450 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:31.220 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:31.213+0000 7f79595c4700 1 --2- 172.21.15.203:0/825101827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79540bb3c0 0x7f7954162790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:23:31.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:31.213+0000 7f79595c4700 1 --2- 172.21.15.203:0/825101827 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f795414f530 0x7f79541507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-01-06T22:23:31.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:31.213+0000 7f79595c4700 1 -- 172.21.15.203:0/825101827 >> 172.21.15.203:0/825101827 conn(0x7f79540626e0 msgr2=0x7f7954063c50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:23:31.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:31.213+0000 7f79595c4700 1 -- 172.21.15.203:0/825101827 shutdown_connections 2021-01-06T22:23:31.221 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:23:31.213+0000 7f79595c4700 1 -- 172.21.15.203:0/825101827 wait complete. 2021-01-06T22:23:31.222 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 100 2021-01-06T22:23:31.230 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2021-01-06T22:23:31.230 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 38 secs before thrashing 2021-01-06T22:23:55.642 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:23:55.653 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:24:09.231 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:24:09.367 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70ebfff700 1 Processor -- start 2021-01-06T22:24:09.368 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70ebfff700 1 -- start start 2021-01-06T22:24:09.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70ebfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f70ec0633f0 0x7f70ec151160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:09.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70ebfff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70ec0602b0 con 0x7f70ec0be1b0 2021-01-06T22:24:09.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70ebfff700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70ec151710 con 0x7f70ec063010 2021-01-06T22:24:09.369 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70ebfff700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70ec14b760 con 0x7f70ec0633f0 2021-01-06T22:24:09.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70e27fc700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f70ec0633f0 0x7f70ec151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:09.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70e27fc700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f70ec0633f0 0x7f70ec151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:56844/0 (socket says 172.21.15.203:56844) 2021-01-06T22:24:09.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70e27fc700 1 -- 172.21.15.203:0/4096364395 learned_addr learned my addr 172.21.15.203:0/4096364395 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:09.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70e27fc700 1 -- 172.21.15.203:0/4096364395 >> v1:172.21.15.203:6789/0 conn(0x7f70ec063010 legacy=0x7f70ec14e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:09.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70e27fc700 1 -- 172.21.15.203:0/4096364395 >> v1:172.21.15.91:6789/0 conn(0x7f70ec0be1b0 legacy=0x7f70ec060750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:24:09.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70e27fc700 1 -- 172.21.15.203:0/4096364395 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec1519b0 con 0x7f70ec0633f0 2021-01-06T22:24:09.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70e27fc700 1 --2- 172.21.15.203:0/4096364395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70ec0633f0 0x7f70ec151160 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7f70d400b9e0 tx=0x7f70d4006ea0).ready entity=mon.1 client_cookie=ab2c12c7885c40f0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:09.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70ea7fc700 1 -- 172.21.15.203:0/4096364395 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4009920 con 0x7f70ec0633f0 2021-01-06T22:24:09.375 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70ea7fc700 1 -- 172.21.15.203:0/4096364395 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d4009a80 con 0x7f70ec0633f0 2021-01-06T22:24:09.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70ea7fc700 1 -- 172.21.15.203:0/4096364395 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4009e00 con 0x7f70ec0633f0 2021-01-06T22:24:09.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70ebfff700 1 -- 172.21.15.203:0/4096364395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70ec0633f0 msgr2=0x7f70ec151160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:09.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70ebfff700 1 --2- 172.21.15.203:0/4096364395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70ec0633f0 0x7f70ec151160 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7f70d400b9e0 tx=0x7f70d4006ea0).stop 2021-01-06T22:24:09.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70ebfff700 1 -- 172.21.15.203:0/4096364395 shutdown_connections 2021-01-06T22:24:09.376 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70ebfff700 1 --2- 172.21.15.203:0/4096364395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70ec0633f0 0x7f70ec151160 secure :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 rx=0x7f70d400b9e0 tx=0x7f70d4006ea0).stop 2021-01-06T22:24:09.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70ebfff700 1 -- 172.21.15.203:0/4096364395 >> 172.21.15.203:0/4096364395 conn(0x7f70ec067e20 msgr2=0x7f70ec068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:09.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70ebfff700 1 -- 172.21.15.203:0/4096364395 shutdown_connections 2021-01-06T22:24:09.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.364+0000 7f70ebfff700 1 -- 172.21.15.203:0/4096364395 wait complete. 2021-01-06T22:24:09.377 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70ebfff700 1 Processor -- start 2021-01-06T22:24:09.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70ebfff700 1 -- start start 2021-01-06T22:24:09.378 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70ebfff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70ec063010 0x7f70ec15d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:09.379 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70ebfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70ec0be1b0 0x7f70ec150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:09.379 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70ebfff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70ec161a40 0x7f70ec146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:09.379 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70ebfff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec160e30 con 0x7f70ec063010 2021-01-06T22:24:09.379 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70ebfff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec0b2bb0 con 0x7f70ec0be1b0 2021-01-06T22:24:09.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70ebfff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec0b2e50 con 0x7f70ec161a40 2021-01-06T22:24:09.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70e27fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70ec0be1b0 0x7f70ec150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:09.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70e27fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70ec0be1b0 0x7f70ec150850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:56852/0 (socket says 172.21.15.203:56852) 2021-01-06T22:24:09.380 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70e27fc700 1 -- 172.21.15.203:0/430739724 learned_addr learned my addr 172.21.15.203:0/430739724 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:09.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70eb7fe700 1 --2- 172.21.15.203:0/430739724 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70ec161a40 0x7f70ec146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:09.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70eaffd700 1 --2- 172.21.15.203:0/430739724 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70ec063010 0x7f70ec15d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:09.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70e27fc700 1 -- 172.21.15.203:0/430739724 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70ec161a40 msgr2=0x7f70ec146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:09.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70e27fc700 1 --2- 172.21.15.203:0/430739724 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70ec161a40 0x7f70ec146de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:09.381 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70e27fc700 1 -- 172.21.15.203:0/430739724 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70ec063010 msgr2=0x7f70ec15d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:09.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70e27fc700 1 --2- 172.21.15.203:0/430739724 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70ec063010 0x7f70ec15d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:09.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70e27fc700 1 -- 172.21.15.203:0/430739724 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec161060 con 0x7f70ec0be1b0 2021-01-06T22:24:09.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70eaffd700 1 --2- 172.21.15.203:0/430739724 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70ec063010 0x7f70ec15d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:24:09.382 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70eb7fe700 1 --2- 172.21.15.203:0/430739724 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70ec161a40 0x7f70ec146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:24:09.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70e27fc700 1 --2- 172.21.15.203:0/430739724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70ec0be1b0 0x7f70ec150850 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f70ec05dc10 tx=0x7f70d4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:09.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70e37fe700 1 -- 172.21.15.203:0/430739724 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d401d430 con 0x7f70ec0be1b0 2021-01-06T22:24:09.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70ebfff700 1 -- 172.21.15.203:0/430739724 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ec161300 con 0x7f70ec0be1b0 2021-01-06T22:24:09.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70e37fe700 1 -- 172.21.15.203:0/430739724 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d401d590 con 0x7f70ec0be1b0 2021-01-06T22:24:09.383 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70ebfff700 1 -- 172.21.15.203:0/430739724 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ec15ddd0 con 0x7f70ec0be1b0 2021-01-06T22:24:09.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.368+0000 7f70e37fe700 1 -- 172.21.15.203:0/430739724 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d401d910 con 0x7f70ec0be1b0 2021-01-06T22:24:09.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.372+0000 7f70ebfff700 1 -- 172.21.15.203:0/430739724 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ec0450a0 con 0x7f70ec0be1b0 2021-01-06T22:24:09.384 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.372+0000 7f70e37fe700 1 -- 172.21.15.203:0/430739724 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f70d4024030 con 0x7f70ec0be1b0 2021-01-06T22:24:09.385 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.372+0000 7f70e37fe700 1 --2- 172.21.15.203:0/430739724 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f70d00563d0 0x7f70d0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:09.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.372+0000 7f70e37fe700 1 -- 172.21.15.203:0/430739724 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7105+0+0 (secure 0 0 0) 0x7f70d4070cb0 con 0x7f70ec0be1b0 2021-01-06T22:24:09.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.372+0000 7f70eaffd700 1 --2- 172.21.15.203:0/430739724 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f70d00563d0 0x7f70d0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:09.386 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.372+0000 7f70eaffd700 1 --2- 172.21.15.203:0/430739724 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f70d00563d0 0x7f70d0058890 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f70dc001590 tx=0x7f70dc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:09.387 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.384+0000 7f70e37fe700 1 -- 172.21.15.203:0/430739724 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f70d40461d0 con 0x7f70ec0be1b0 2021-01-06T22:24:09.664 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.660+0000 7f70ebfff700 1 -- 172.21.15.203:0/430739724 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70ec0be990 con 0x7f70ec0be1b0 2021-01-06T22:24:10.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.660+0000 7f70e37fe700 1 -- 172.21.15.203:0/430739724 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+3570 (secure 0 0 0) 0x7f70d4022030 con 0x7f70ec0be1b0 2021-01-06T22:24:10.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.664+0000 7f70ebfff700 1 -- 172.21.15.203:0/430739724 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f70d00563d0 msgr2=0x7f70d0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:10.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.664+0000 7f70ebfff700 1 --2- 172.21.15.203:0/430739724 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f70d00563d0 0x7f70d0058890 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f70dc001590 tx=0x7f70dc009040).stop 2021-01-06T22:24:10.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.664+0000 7f70ebfff700 1 -- 172.21.15.203:0/430739724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70ec0be1b0 msgr2=0x7f70ec150850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:10.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.664+0000 7f70ebfff700 1 --2- 172.21.15.203:0/430739724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70ec0be1b0 0x7f70ec150850 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f70ec05dc10 tx=0x7f70d4016000).stop 2021-01-06T22:24:10.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.664+0000 7f70ebfff700 1 -- 172.21.15.203:0/430739724 shutdown_connections 2021-01-06T22:24:10.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.664+0000 7f70ebfff700 1 --2- 172.21.15.203:0/430739724 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f70d00563d0 0x7f70d0058890 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:10.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.664+0000 7f70ebfff700 1 --2- 172.21.15.203:0/430739724 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70ec161a40 0x7f70ec146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:10.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.664+0000 7f70ebfff700 1 --2- 172.21.15.203:0/430739724 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70ec063010 0x7f70ec15d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:10.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.664+0000 7f70ebfff700 1 --2- 172.21.15.203:0/430739724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70ec0be1b0 0x7f70ec150850 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:10.250 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.664+0000 7f70ebfff700 1 -- 172.21.15.203:0/430739724 >> 172.21.15.203:0/430739724 conn(0x7f70ec067e20 msgr2=0x7f70ec14ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:10.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.664+0000 7f70ebfff700 1 -- 172.21.15.203:0/430739724 shutdown_connections 2021-01-06T22:24:10.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:09.664+0000 7f70ebfff700 1 -- 172.21.15.203:0/430739724 wait complete. 2021-01-06T22:24:10.251 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 100 2021-01-06T22:24:10.252 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:24:10.252 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":6052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3262828783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3262828783},{"type":"v1","addr":"172.21.15.203:6837","nonce":3262828783}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:23:20.754099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:24:10.253 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=0) 2021-01-06T22:24:10.254 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-01-06T22:24:10.254 INFO:teuthology.orchestra.run:waiting for 300 2021-01-06T22:24:10.266 INFO:tasks.ceph.mds.c:Stopped 2021-01-06T22:24:10.267 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:24:10.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.460+0000 7f3d5b402700 1 Processor -- start 2021-01-06T22:24:10.467 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.460+0000 7f3d5b402700 1 -- start start 2021-01-06T22:24:10.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d5b402700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d5414e310 0x7f3d541471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:10.472 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d5b402700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d540c16b0 con 0x7f3d54064380 2021-01-06T22:24:10.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d5b402700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d540bb2d0 con 0x7f3d54064760 2021-01-06T22:24:10.474 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d5b402700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5414a560 con 0x7f3d5414e310 2021-01-06T22:24:10.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d59bff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d5414e310 0x7f3d541471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:10.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d5ac01700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f3d54064760 0x7f3d540bb730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:55500/0 (socket says 172.21.15.203:55500) 2021-01-06T22:24:10.475 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d5ac01700 1 -- 172.21.15.203:0/4108902191 learned_addr learned my addr 172.21.15.203:0/4108902191 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:24:10.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d59bff700 1 -- 172.21.15.203:0/4108902191 >> v1:172.21.15.203:6789/0 conn(0x7f3d54064760 legacy=0x7f3d540bb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:10.476 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d59bff700 1 -- 172.21.15.203:0/4108902191 >> v1:172.21.15.91:6789/0 conn(0x7f3d54064380 legacy=0x7f3d540c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:10.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d593fe700 1 -- 172.21.15.203:0/4108902191 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204060133 0 0) 0x7f3d500010d0 con 0x7f3d54064380 2021-01-06T22:24:10.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d59bff700 1 -- 172.21.15.203:0/4108902191 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5414a800 con 0x7f3d5414e310 2021-01-06T22:24:10.477 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d59bff700 1 --2- 172.21.15.203:0/4108902191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d5414e310 0x7f3d541471b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f3d4400b900 tx=0x7f3d44006de0).ready entity=mon.1 client_cookie=a8f9fecf45aa4f5a server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:10.478 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d593fe700 1 -- 172.21.15.203:0/4108902191 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d44009730 con 0x7f3d5414e310 2021-01-06T22:24:10.478 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d593fe700 1 -- 172.21.15.203:0/4108902191 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d44009890 con 0x7f3d5414e310 2021-01-06T22:24:10.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d593fe700 1 -- 172.21.15.203:0/4108902191 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d44009c10 con 0x7f3d5414e310 2021-01-06T22:24:10.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d5b402700 1 -- 172.21.15.203:0/4108902191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d5414e310 msgr2=0x7f3d541471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:10.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d5b402700 1 --2- 172.21.15.203:0/4108902191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d5414e310 0x7f3d541471b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f3d4400b900 tx=0x7f3d44006de0).stop 2021-01-06T22:24:10.480 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d5b402700 1 -- 172.21.15.203:0/4108902191 shutdown_connections 2021-01-06T22:24:10.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d5b402700 1 --2- 172.21.15.203:0/4108902191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d5414e310 0x7f3d541471b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:10.481 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d5b402700 1 -- 172.21.15.203:0/4108902191 >> 172.21.15.203:0/4108902191 conn(0x7f3d540667b0 msgr2=0x7f3d540c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:10.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d5b402700 1 -- 172.21.15.203:0/4108902191 shutdown_connections 2021-01-06T22:24:10.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.464+0000 7f3d5b402700 1 -- 172.21.15.203:0/4108902191 wait complete. 2021-01-06T22:24:10.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5b402700 1 Processor -- start 2021-01-06T22:24:10.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5b402700 1 -- start start 2021-01-06T22:24:10.488 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5b402700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d54064380 0x7f3d5413e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:10.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5b402700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d54064760 0x7f3d5413a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:10.489 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5b402700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d5414e310 0x7f3d5413d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:10.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5b402700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d54148850 con 0x7f3d54064760 2021-01-06T22:24:10.490 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5b402700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d54148070 con 0x7f3d5414e310 2021-01-06T22:24:10.491 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5b402700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d54148310 con 0x7f3d54064380 2021-01-06T22:24:10.492 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5ac01700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d5414e310 0x7f3d5413d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:10.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5ac01700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d5414e310 0x7f3d5413d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:56912/0 (socket says 172.21.15.203:56912) 2021-01-06T22:24:10.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5ac01700 1 -- 172.21.15.203:0/3359422753 learned_addr learned my addr 172.21.15.203:0/3359422753 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:10.493 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d59bff700 1 --2- 172.21.15.203:0/3359422753 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d54064760 0x7f3d5413a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:10.494 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5a400700 1 --2- 172.21.15.203:0/3359422753 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d54064380 0x7f3d5413e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:10.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5ac01700 1 -- 172.21.15.203:0/3359422753 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d54064380 msgr2=0x7f3d5413e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:10.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5ac01700 1 --2- 172.21.15.203:0/3359422753 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d54064380 0x7f3d5413e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:10.495 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5ac01700 1 -- 172.21.15.203:0/3359422753 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d54064760 msgr2=0x7f3d5413a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:10.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5ac01700 1 --2- 172.21.15.203:0/3359422753 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d54064760 0x7f3d5413a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:10.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5ac01700 1 -- 172.21.15.203:0/3359422753 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d541485b0 con 0x7f3d5414e310 2021-01-06T22:24:10.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5ac01700 1 --2- 172.21.15.203:0/3359422753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d5414e310 0x7f3d5413d180 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f3d4c00cb30 tx=0x7f3d4c007eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:10.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d427fc700 1 -- 172.21.15.203:0/3359422753 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c005b70 con 0x7f3d5414e310 2021-01-06T22:24:10.496 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d427fc700 1 -- 172.21.15.203:0/3359422753 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c005cd0 con 0x7f3d5414e310 2021-01-06T22:24:10.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5b402700 1 -- 172.21.15.203:0/3359422753 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5413ca40 con 0x7f3d5414e310 2021-01-06T22:24:10.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d5b402700 1 -- 172.21.15.203:0/3359422753 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d540b7f00 con 0x7f3d5414e310 2021-01-06T22:24:10.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.468+0000 7f3d427fc700 1 -- 172.21.15.203:0/3359422753 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c007320 con 0x7f3d5414e310 2021-01-06T22:24:10.497 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.472+0000 7f3d5b402700 1 -- 172.21.15.203:0/3359422753 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2c004f60 con 0x7f3d5414e310 2021-01-06T22:24:10.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.472+0000 7f3d427fc700 1 -- 172.21.15.203:0/3359422753 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f3d4c010030 con 0x7f3d5414e310 2021-01-06T22:24:10.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.472+0000 7f3d427fc700 1 --2- 172.21.15.203:0/3359422753 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3d3c056500 0x7f3d3c0589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:10.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.472+0000 7f3d427fc700 1 -- 172.21.15.203:0/3359422753 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7105+0+0 (secure 0 0 0) 0x7f3d4c06f9b0 con 0x7f3d5414e310 2021-01-06T22:24:10.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.472+0000 7f3d5a400700 1 --2- 172.21.15.203:0/3359422753 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3d3c056500 0x7f3d3c0589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:10.498 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.472+0000 7f3d5a400700 1 --2- 172.21.15.203:0/3359422753 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3d3c056500 0x7f3d3c0589c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f3d50003ed0 tx=0x7f3d50009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:10.499 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.480+0000 7f3d427fc700 1 -- 172.21.15.203:0/3359422753 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f3d4c044dc0 con 0x7f3d5414e310 2021-01-06T22:24:10.706 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.700+0000 7f3d5b402700 1 -- 172.21.15.203:0/3359422753 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d2c005790 con 0x7f3d5414e310 2021-01-06T22:24:10.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.704+0000 7f3d427fc700 1 -- 172.21.15.203:0/3359422753 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+3570 (secure 0 0 0) 0x7f3d4c0483e0 con 0x7f3d5414e310 2021-01-06T22:24:10.708 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:24:10.708 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":6052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3262828783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3262828783},{"type":"v1","addr":"172.21.15.203:6837","nonce":3262828783}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:23:20.754099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:24:10.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.708+0000 7f3d5b402700 1 -- 172.21.15.203:0/3359422753 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3d3c056500 msgr2=0x7f3d3c0589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:10.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.708+0000 7f3d5b402700 1 --2- 172.21.15.203:0/3359422753 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3d3c056500 0x7f3d3c0589c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f3d50003ed0 tx=0x7f3d50009040).stop 2021-01-06T22:24:10.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.708+0000 7f3d5b402700 1 -- 172.21.15.203:0/3359422753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d5414e310 msgr2=0x7f3d5413d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:10.712 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.708+0000 7f3d5b402700 1 --2- 172.21.15.203:0/3359422753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d5414e310 0x7f3d5413d180 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f3d4c00cb30 tx=0x7f3d4c007eb0).stop 2021-01-06T22:24:10.713 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.708+0000 7f3d5b402700 1 -- 172.21.15.203:0/3359422753 shutdown_connections 2021-01-06T22:24:10.713 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.708+0000 7f3d5b402700 1 --2- 172.21.15.203:0/3359422753 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f3d3c056500 0x7f3d3c0589c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:10.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.708+0000 7f3d5b402700 1 --2- 172.21.15.203:0/3359422753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d5414e310 0x7f3d5413d180 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:10.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.708+0000 7f3d5b402700 1 --2- 172.21.15.203:0/3359422753 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d54064380 0x7f3d5413e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:10.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.708+0000 7f3d5b402700 1 --2- 172.21.15.203:0/3359422753 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d54064760 0x7f3d5413a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:10.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.708+0000 7f3d5b402700 1 -- 172.21.15.203:0/3359422753 >> 172.21.15.203:0/3359422753 conn(0x7f3d540667b0 msgr2=0x7f3d540ba280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:10.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.708+0000 7f3d5b402700 1 -- 172.21.15.203:0/3359422753 shutdown_connections 2021-01-06T22:24:10.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:10.708+0000 7f3d5b402700 1 -- 172.21.15.203:0/3359422753 wait complete. 2021-01-06T22:24:10.715 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 100 2021-01-06T22:24:10.730 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:24:12.734 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:24:12.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864ffff700 1 Processor -- start 2021-01-06T22:24:12.876 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864ffff700 1 -- start start 2021-01-06T22:24:12.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864ffff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f86500c0fe0 0x7f86500aa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:12.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864ffff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86500a9ca0 con 0x7f86500c03f0 2021-01-06T22:24:12.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864ffff700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8650066a50 con 0x7f86500c13c0 2021-01-06T22:24:12.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864ffff700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8650066cf0 con 0x7f86500c0fe0 2021-01-06T22:24:12.889 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864f7fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f86500c13c0 0x7f8650150110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:55590/0 (socket says 172.21.15.203:55590) 2021-01-06T22:24:12.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864f7fe700 1 -- 172.21.15.203:0/3304110277 learned_addr learned my addr 172.21.15.203:0/3304110277 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:24:12.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864effd700 1 --2- 172.21.15.203:0/3304110277 >> v2:172.21.15.203:3300/0 conn(0x7f86500c0fe0 0x7f86500aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:12.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864effd700 1 -- 172.21.15.203:0/3304110277 >> v1:172.21.15.203:6789/0 conn(0x7f86500c13c0 legacy=0x7f8650150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:12.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864effd700 1 -- 172.21.15.203:0/3304110277 >> v1:172.21.15.91:6789/0 conn(0x7f86500c03f0 legacy=0x7f8650060d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:12.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864effd700 1 -- 172.21.15.203:0/3304110277 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8650068460 con 0x7f86500c0fe0 2021-01-06T22:24:12.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864effd700 1 --2- 172.21.15.203:0/3304110277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86500c0fe0 0x7f86500aa030 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f864000b980 tx=0x7f8640006de0).ready entity=mon.1 client_cookie=1f188aed0c3012b4 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:12.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864e7fc700 1 -- 172.21.15.203:0/3304110277 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86400097b0 con 0x7f86500c0fe0 2021-01-06T22:24:12.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864e7fc700 1 -- 172.21.15.203:0/3304110277 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8640009910 con 0x7f86500c0fe0 2021-01-06T22:24:12.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864e7fc700 1 -- 172.21.15.203:0/3304110277 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8640009c90 con 0x7f86500c0fe0 2021-01-06T22:24:12.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864ffff700 1 -- 172.21.15.203:0/3304110277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86500c0fe0 msgr2=0x7f86500aa030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:12.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864ffff700 1 --2- 172.21.15.203:0/3304110277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86500c0fe0 0x7f86500aa030 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f864000b980 tx=0x7f8640006de0).stop 2021-01-06T22:24:12.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864ffff700 1 -- 172.21.15.203:0/3304110277 shutdown_connections 2021-01-06T22:24:12.895 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864ffff700 1 --2- 172.21.15.203:0/3304110277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86500c0fe0 0x7f86500aa030 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:12.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864ffff700 1 -- 172.21.15.203:0/3304110277 >> 172.21.15.203:0/3304110277 conn(0x7f8650061530 msgr2=0x7f8650061910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:12.896 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864ffff700 1 -- 172.21.15.203:0/3304110277 shutdown_connections 2021-01-06T22:24:12.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.872+0000 7f864ffff700 1 -- 172.21.15.203:0/3304110277 wait complete. 2021-01-06T22:24:12.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864ffff700 1 Processor -- start 2021-01-06T22:24:12.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864ffff700 1 -- start start 2021-01-06T22:24:12.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864ffff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86500c03f0 0x7f865014b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:12.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864ffff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86500c0fe0 0x7f86501512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:12.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86500c13c0 0x7f8650146d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:12.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864ffff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865014a9b0 con 0x7f86500c0fe0 2021-01-06T22:24:12.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864ffff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86501618a0 con 0x7f86500c13c0 2021-01-06T22:24:12.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864ffff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8650161ad0 con 0x7f86500c03f0 2021-01-06T22:24:12.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86500c13c0 0x7f8650146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:12.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86500c13c0 0x7f8650146d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57002/0 (socket says 172.21.15.203:57002) 2021-01-06T22:24:12.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864f7fe700 1 -- 172.21.15.203:0/233747033 learned_addr learned my addr 172.21.15.203:0/233747033 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:12.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864effd700 1 --2- 172.21.15.203:0/233747033 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86500c03f0 0x7f865014b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:12.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f8647fff700 1 --2- 172.21.15.203:0/233747033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86500c0fe0 0x7f86501512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:12.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864f7fe700 1 -- 172.21.15.203:0/233747033 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86500c03f0 msgr2=0x7f865014b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:12.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864f7fe700 1 --2- 172.21.15.203:0/233747033 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86500c03f0 0x7f865014b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:12.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864f7fe700 1 -- 172.21.15.203:0/233747033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86500c0fe0 msgr2=0x7f86501512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:12.904 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864f7fe700 1 --2- 172.21.15.203:0/233747033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86500c0fe0 0x7f86501512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:12.905 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864f7fe700 1 -- 172.21.15.203:0/233747033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86500bbf90 con 0x7f86500c13c0 2021-01-06T22:24:12.905 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864f7fe700 1 --2- 172.21.15.203:0/233747033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86500c13c0 0x7f8650146d10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f864800cac0 tx=0x7f8648007f70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:12.905 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f8646ffd700 1 -- 172.21.15.203:0/233747033 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864800ce90 con 0x7f86500c13c0 2021-01-06T22:24:12.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f8646ffd700 1 -- 172.21.15.203:0/233747033 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8648005b70 con 0x7f86500c13c0 2021-01-06T22:24:12.906 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f8646ffd700 1 -- 172.21.15.203:0/233747033 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648007110 con 0x7f86500c13c0 2021-01-06T22:24:12.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864ffff700 1 -- 172.21.15.203:0/233747033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86500bc230 con 0x7f86500c13c0 2021-01-06T22:24:12.907 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.876+0000 7f864ffff700 1 -- 172.21.15.203:0/233747033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86501494d0 con 0x7f86500c13c0 2021-01-06T22:24:12.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.880+0000 7f8646ffd700 1 -- 172.21.15.203:0/233747033 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f8648010030 con 0x7f86500c13c0 2021-01-06T22:24:12.908 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.880+0000 7f864ffff700 1 -- 172.21.15.203:0/233747033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86500662b0 con 0x7f86500c13c0 2021-01-06T22:24:12.909 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.880+0000 7f8646ffd700 1 --2- 172.21.15.203:0/233747033 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8634056400 0x7f86340588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:12.909 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.880+0000 7f8646ffd700 1 -- 172.21.15.203:0/233747033 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7105+0+0 (secure 0 0 0) 0x7f864806f6e0 con 0x7f86500c13c0 2021-01-06T22:24:12.909 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.880+0000 7f864effd700 1 --2- 172.21.15.203:0/233747033 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8634056400 0x7f86340588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:12.910 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.880+0000 7f864effd700 1 --2- 172.21.15.203:0/233747033 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8634056400 0x7f86340588c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f8640006e20 tx=0x7f8640016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:12.910 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:12.884+0000 7f8646ffd700 1 -- 172.21.15.203:0/233747033 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f86500662b0 con 0x7f86500c13c0 2021-01-06T22:24:13.169 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:13.164+0000 7f864ffff700 1 -- 172.21.15.203:0/233747033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86500662b0 con 0x7f86500c13c0 2021-01-06T22:24:13.171 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:13.164+0000 7f8646ffd700 1 -- 172.21.15.203:0/233747033 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+3570 (secure 0 0 0) 0x7f86500662b0 con 0x7f86500c13c0 2021-01-06T22:24:13.171 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:24:13.172 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":6052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3262828783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3262828783},{"type":"v1","addr":"172.21.15.203:6837","nonce":3262828783}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:23:20.754099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:24:13.173 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:13.168+0000 7f864ffff700 1 -- 172.21.15.203:0/233747033 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8634056400 msgr2=0x7f86340588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:13.174 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:13.168+0000 7f864ffff700 1 --2- 172.21.15.203:0/233747033 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8634056400 0x7f86340588c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f8640006e20 tx=0x7f8640016000).stop 2021-01-06T22:24:13.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:13.168+0000 7f864ffff700 1 -- 172.21.15.203:0/233747033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86500c13c0 msgr2=0x7f8650146d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:13.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:13.168+0000 7f864ffff700 1 --2- 172.21.15.203:0/233747033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86500c13c0 0x7f8650146d10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f864800cac0 tx=0x7f8648007f70).stop 2021-01-06T22:24:13.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:13.168+0000 7f864ffff700 1 -- 172.21.15.203:0/233747033 shutdown_connections 2021-01-06T22:24:13.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:13.168+0000 7f864ffff700 1 --2- 172.21.15.203:0/233747033 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f8634056400 0x7f86340588c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:13.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:13.168+0000 7f864ffff700 1 --2- 172.21.15.203:0/233747033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86500c13c0 0x7f8650146d10 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:13.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:13.168+0000 7f864ffff700 1 --2- 172.21.15.203:0/233747033 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86500c03f0 0x7f865014b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:13.185 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:13.168+0000 7f864ffff700 1 --2- 172.21.15.203:0/233747033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86500c0fe0 0x7f86501512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:13.185 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:13.168+0000 7f864ffff700 1 -- 172.21.15.203:0/233747033 >> 172.21.15.203:0/233747033 conn(0x7f8650061530 msgr2=0x7f8650063a70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:13.185 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:13.168+0000 7f864ffff700 1 -- 172.21.15.203:0/233747033 shutdown_connections 2021-01-06T22:24:13.185 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:13.168+0000 7f864ffff700 1 -- 172.21.15.203:0/233747033 wait complete. 2021-01-06T22:24:13.186 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 100 2021-01-06T22:24:13.190 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:24:15.192 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:24:15.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b18a4700 1 Processor -- start 2021-01-06T22:24:15.319 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b18a4700 1 -- start start 2021-01-06T22:24:15.321 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b18a4700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f22ac0b97a0 0x7f22ac0b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:15.321 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b18a4700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f22ac0bed80 0x7f22ac14c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:15.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b18a4700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ac0be410 con 0x7f22ac0bd420 2021-01-06T22:24:15.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b18a4700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ac066510 con 0x7f22ac0bed80 2021-01-06T22:24:15.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b18a4700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac0667b0 con 0x7f22ac0b97a0 2021-01-06T22:24:15.322 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22abfff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f22ac0bd420 0x7f22ac065c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:55680/0 (socket says 172.21.15.203:55680) 2021-01-06T22:24:15.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22abfff700 1 -- 172.21.15.203:0/1013681639 learned_addr learned my addr 172.21.15.203:0/1013681639 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:24:15.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b10a3700 1 --2- 172.21.15.203:0/1013681639 >> v2:172.21.15.91:3300/0 conn(0x7f22ac0bed80 0x7f22ac14c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:15.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b08a2700 1 --2- 172.21.15.203:0/1013681639 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f22ac0b97a0 0x7f22ac0b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:15.323 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22ab7fe700 1 -- 172.21.15.203:0/1013681639 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268368597 0 0) 0x7f22ac0be410 con 0x7f22ac0bd420 2021-01-06T22:24:15.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22ab7fe700 1 -- 172.21.15.203:0/1013681639 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2290003230 con 0x7f22ac0bd420 2021-01-06T22:24:15.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b10a3700 1 -- 172.21.15.203:0/1013681639 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f22ac0b97a0 msgr2=0x7f22ac0b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:24:15.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b10a3700 1 --2- 172.21.15.203:0/1013681639 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f22ac0b97a0 0x7f22ac0b9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:15.324 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b10a3700 1 -- 172.21.15.203:0/1013681639 >> v1:172.21.15.203:6789/0 conn(0x7f22ac0bd420 legacy=0x7f22ac065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:15.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b10a3700 1 -- 172.21.15.203:0/1013681639 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac066a50 con 0x7f22ac0bed80 2021-01-06T22:24:15.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b10a3700 1 --2- 172.21.15.203:0/1013681639 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22ac0bed80 0x7f22ac14c030 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f22a4009c90 tx=0x7f22a40091d0).ready entity=mon.0 client_cookie=dfbaf670d82a85a5 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:15.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22ab7fe700 1 -- 172.21.15.203:0/1013681639 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a400ba50 con 0x7f22ac0bed80 2021-01-06T22:24:15.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22ab7fe700 1 -- 172.21.15.203:0/1013681639 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a400bbb0 con 0x7f22ac0bed80 2021-01-06T22:24:15.325 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22ab7fe700 1 -- 172.21.15.203:0/1013681639 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a4011400 con 0x7f22ac0bed80 2021-01-06T22:24:15.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b18a4700 1 -- 172.21.15.203:0/1013681639 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22ac0bed80 msgr2=0x7f22ac14c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:15.328 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b18a4700 1 --2- 172.21.15.203:0/1013681639 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22ac0bed80 0x7f22ac14c030 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f22a4009c90 tx=0x7f22a40091d0).stop 2021-01-06T22:24:15.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b18a4700 1 -- 172.21.15.203:0/1013681639 shutdown_connections 2021-01-06T22:24:15.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b18a4700 1 --2- 172.21.15.203:0/1013681639 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22ac0bed80 0x7f22ac14c030 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:15.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b18a4700 1 --2- 172.21.15.203:0/1013681639 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f22ac0b97a0 0x7f22ac0b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:15.329 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b18a4700 1 -- 172.21.15.203:0/1013681639 >> 172.21.15.203:0/1013681639 conn(0x7f22ac0648f0 msgr2=0x7f22ac064cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:15.330 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b18a4700 1 -- 172.21.15.203:0/1013681639 shutdown_connections 2021-01-06T22:24:15.331 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.316+0000 7f22b18a4700 1 -- 172.21.15.203:0/1013681639 wait complete. 2021-01-06T22:24:15.331 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22b18a4700 1 Processor -- start 2021-01-06T22:24:15.331 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22b18a4700 1 -- start start 2021-01-06T22:24:15.331 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22b18a4700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22ac0b97a0 0x7f22ac14b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:15.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22b18a4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac0bd420 0x7f22ac14ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:15.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22b18a4700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f22ac0bed80 0x7f22ac148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:15.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22b18a4700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac149d30 con 0x7f22ac0b97a0 2021-01-06T22:24:15.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22b18a4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac14fda0 con 0x7f22ac0bd420 2021-01-06T22:24:15.332 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22b18a4700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac09e4c0 con 0x7f22ac0bed80 2021-01-06T22:24:15.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22abfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac0bd420 0x7f22ac14ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:15.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22abfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac0bd420 0x7f22ac14ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57092/0 (socket says 172.21.15.203:57092) 2021-01-06T22:24:15.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22abfff700 1 -- 172.21.15.203:0/2470781107 learned_addr learned my addr 172.21.15.203:0/2470781107 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:15.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22b10a3700 1 --2- 172.21.15.203:0/2470781107 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f22ac0bed80 0x7f22ac148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:15.333 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22b08a2700 1 --2- 172.21.15.203:0/2470781107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22ac0b97a0 0x7f22ac14b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:15.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22abfff700 1 -- 172.21.15.203:0/2470781107 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f22ac0bed80 msgr2=0x7f22ac148eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:15.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22abfff700 1 --2- 172.21.15.203:0/2470781107 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f22ac0bed80 0x7f22ac148eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:15.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22abfff700 1 -- 172.21.15.203:0/2470781107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22ac0b97a0 msgr2=0x7f22ac14b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:15.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22abfff700 1 --2- 172.21.15.203:0/2470781107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22ac0b97a0 0x7f22ac14b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:15.334 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22abfff700 1 -- 172.21.15.203:0/2470781107 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac09e760 con 0x7f22ac0bd420 2021-01-06T22:24:15.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22abfff700 1 --2- 172.21.15.203:0/2470781107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac0bd420 0x7f22ac14ef00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f229800c740 tx=0x7f2298007a90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:15.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22a8ff9700 1 -- 172.21.15.203:0/2470781107 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980072e0 con 0x7f22ac0bd420 2021-01-06T22:24:15.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22a8ff9700 1 -- 172.21.15.203:0/2470781107 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2298007440 con 0x7f22ac0bd420 2021-01-06T22:24:15.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22a8ff9700 1 -- 172.21.15.203:0/2470781107 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980057f0 con 0x7f22ac0bd420 2021-01-06T22:24:15.335 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22b18a4700 1 -- 172.21.15.203:0/2470781107 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ac15eed0 con 0x7f22ac0bd420 2021-01-06T22:24:15.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.320+0000 7f22b18a4700 1 -- 172.21.15.203:0/2470781107 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ac14f3f0 con 0x7f22ac0bd420 2021-01-06T22:24:15.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.324+0000 7f22b18a4700 1 -- 172.21.15.203:0/2470781107 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ac15ed00 con 0x7f22ac0bd420 2021-01-06T22:24:15.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.324+0000 7f22a8ff9700 1 -- 172.21.15.203:0/2470781107 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f2298010070 con 0x7f22ac0bd420 2021-01-06T22:24:15.336 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.324+0000 7f22a8ff9700 1 --2- 172.21.15.203:0/2470781107 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2290056830 0x7f2290058cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:15.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.324+0000 7f22a8ff9700 1 -- 172.21.15.203:0/2470781107 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7105+0+0 (secure 0 0 0) 0x7f229806ef10 con 0x7f22ac0bd420 2021-01-06T22:24:15.339 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.328+0000 7f22b08a2700 1 --2- 172.21.15.203:0/2470781107 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2290056830 0x7f2290058cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:15.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.328+0000 7f22b08a2700 1 --2- 172.21.15.203:0/2470781107 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2290056830 0x7f2290058cf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f22a000bb50 tx=0x7f22a0006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:15.340 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.332+0000 7f22a8ff9700 1 -- 172.21.15.203:0/2470781107 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f2298044380 con 0x7f22ac0bd420 2021-01-06T22:24:15.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.520+0000 7f22b18a4700 1 -- 172.21.15.203:0/2470781107 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22ac05ca60 con 0x7f22ac0bd420 2021-01-06T22:24:15.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.520+0000 7f22a8ff9700 1 -- 172.21.15.203:0/2470781107 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+3570 (secure 0 0 0) 0x7f2298018030 con 0x7f22ac0bd420 2021-01-06T22:24:15.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.520+0000 7f22b18a4700 1 -- 172.21.15.203:0/2470781107 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2290056830 msgr2=0x7f2290058cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:15.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.520+0000 7f22b18a4700 1 --2- 172.21.15.203:0/2470781107 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2290056830 0x7f2290058cf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f22a000bb50 tx=0x7f22a0006de0).stop 2021-01-06T22:24:15.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.520+0000 7f22b18a4700 1 -- 172.21.15.203:0/2470781107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac0bd420 msgr2=0x7f22ac14ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:15.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.520+0000 7f22b18a4700 1 --2- 172.21.15.203:0/2470781107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac0bd420 0x7f22ac14ef00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f229800c740 tx=0x7f2298007a90).stop 2021-01-06T22:24:15.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.520+0000 7f22b18a4700 1 -- 172.21.15.203:0/2470781107 shutdown_connections 2021-01-06T22:24:15.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.520+0000 7f22b18a4700 1 --2- 172.21.15.203:0/2470781107 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2290056830 0x7f2290058cf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:15.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.520+0000 7f22b18a4700 1 --2- 172.21.15.203:0/2470781107 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f22ac0bed80 0x7f22ac148eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:15.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.520+0000 7f22b18a4700 1 --2- 172.21.15.203:0/2470781107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22ac0b97a0 0x7f22ac14b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:15.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.520+0000 7f22b18a4700 1 --2- 172.21.15.203:0/2470781107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac0bd420 0x7f22ac14ef00 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:15.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.520+0000 7f22b18a4700 1 -- 172.21.15.203:0/2470781107 >> 172.21.15.203:0/2470781107 conn(0x7f22ac0648f0 msgr2=0x7f22ac0bf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:15.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.520+0000 7f22b18a4700 1 -- 172.21.15.203:0/2470781107 shutdown_connections 2021-01-06T22:24:15.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:15.520+0000 7f22b18a4700 1 -- 172.21.15.203:0/2470781107 wait complete. 2021-01-06T22:24:15.530 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:24:15.530 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":6052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3262828783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3262828783},{"type":"v1","addr":"172.21.15.203:6837","nonce":3262828783}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:23:20.754099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:24:15.531 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 100 2021-01-06T22:24:15.538 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:24:17.540 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:24:17.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.700+0000 7f72a4396700 1 Processor -- start 2021-01-06T22:24:17.705 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.700+0000 7f72a4396700 1 -- start start 2021-01-06T22:24:17.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.700+0000 7f72a4396700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f729c1517a0 0x7f729c066a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:17.707 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.700+0000 7f72a4396700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729c066f30 con 0x7f729c151b80 2021-01-06T22:24:17.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.700+0000 7f72a4396700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729c14f680 con 0x7f729c1505a0 2021-01-06T22:24:17.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.700+0000 7f72a4396700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729c14bc00 con 0x7f729c1517a0 2021-01-06T22:24:17.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.700+0000 7f72a2132700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f729c1517a0 0x7f729c066a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:17.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.700+0000 7f72a2132700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f729c1517a0 0x7f729c066a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57144/0 (socket says 172.21.15.203:57144) 2021-01-06T22:24:17.708 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.700+0000 7f72a2132700 1 -- 172.21.15.203:0/3020260768 learned_addr learned my addr 172.21.15.203:0/3020260768 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:17.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.700+0000 7f72a2132700 1 -- 172.21.15.203:0/3020260768 >> v1:172.21.15.203:6789/0 conn(0x7f729c1505a0 legacy=0x7f729c0bf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:17.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.700+0000 7f72a2132700 1 -- 172.21.15.203:0/3020260768 >> v1:172.21.15.91:6789/0 conn(0x7f729c151b80 legacy=0x7f729c0b8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:17.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.700+0000 7f72a2132700 1 -- 172.21.15.203:0/3020260768 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c14f920 con 0x7f729c1517a0 2021-01-06T22:24:17.709 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.700+0000 7f72a2132700 1 --2- 172.21.15.203:0/3020260768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c1517a0 0x7f729c066a40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f7294009d50 tx=0x7f7294009290).ready entity=mon.1 client_cookie=ef9aa5806568608e server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:17.710 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.704+0000 7f72a1130700 1 -- 172.21.15.203:0/3020260768 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729400bd10 con 0x7f729c1517a0 2021-01-06T22:24:17.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.704+0000 7f72a1130700 1 -- 172.21.15.203:0/3020260768 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729400be70 con 0x7f729c1517a0 2021-01-06T22:24:17.711 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.704+0000 7f72a1130700 1 -- 172.21.15.203:0/3020260768 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294011610 con 0x7f729c1517a0 2021-01-06T22:24:17.713 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.704+0000 7f72a4396700 1 -- 172.21.15.203:0/3020260768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c1517a0 msgr2=0x7f729c066a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:17.713 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.704+0000 7f72a4396700 1 --2- 172.21.15.203:0/3020260768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c1517a0 0x7f729c066a40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f7294009d50 tx=0x7f7294009290).stop 2021-01-06T22:24:17.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.704+0000 7f72a4396700 1 -- 172.21.15.203:0/3020260768 shutdown_connections 2021-01-06T22:24:17.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.704+0000 7f72a4396700 1 --2- 172.21.15.203:0/3020260768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c1517a0 0x7f729c066a40 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:17.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.704+0000 7f72a4396700 1 -- 172.21.15.203:0/3020260768 >> 172.21.15.203:0/3020260768 conn(0x7f729c060d80 msgr2=0x7f729c0661a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:17.714 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.704+0000 7f72a4396700 1 -- 172.21.15.203:0/3020260768 shutdown_connections 2021-01-06T22:24:17.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.704+0000 7f72a4396700 1 -- 172.21.15.203:0/3020260768 wait complete. 2021-01-06T22:24:17.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.704+0000 7f72a4396700 1 Processor -- start 2021-01-06T22:24:17.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.704+0000 7f72a4396700 1 -- start start 2021-01-06T22:24:17.715 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a4396700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f729c1505a0 0x7f729c1461a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:17.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a4396700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c151b80 0x7f729c148c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:17.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a4396700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c15d8e0 0x7f729c149130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:17.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a4396700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c15dd70 con 0x7f729c15d8e0 2021-01-06T22:24:17.716 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a4396700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c15e010 con 0x7f729c151b80 2021-01-06T22:24:17.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a4396700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729c15e2b0 con 0x7f729c1505a0 2021-01-06T22:24:17.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a1931700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c151b80 0x7f729c148c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:17.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a2132700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f729c1505a0 0x7f729c1461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:17.717 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a2933700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c15d8e0 0x7f729c149130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:17.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a2132700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f729c1505a0 0x7f729c1461a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:34000/0 (socket says 172.21.15.203:34000) 2021-01-06T22:24:17.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a1931700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c151b80 0x7f729c148c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57152/0 (socket says 172.21.15.203:57152) 2021-01-06T22:24:17.718 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a2132700 1 -- 172.21.15.203:0/347044703 learned_addr learned my addr 172.21.15.203:0/347044703 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:17.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a1931700 1 -- 172.21.15.203:0/347044703 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f729c1505a0 msgr2=0x7f729c1461a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:17.721 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a1931700 1 --2- 172.21.15.203:0/347044703 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f729c1505a0 0x7f729c1461a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:17.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a1931700 1 -- 172.21.15.203:0/347044703 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c15d8e0 msgr2=0x7f729c149130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:17.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a1931700 1 --2- 172.21.15.203:0/347044703 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c15d8e0 0x7f729c149130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:17.722 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a1931700 1 -- 172.21.15.203:0/347044703 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c15e550 con 0x7f729c151b80 2021-01-06T22:24:17.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a2933700 1 --2- 172.21.15.203:0/347044703 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c15d8e0 0x7f729c149130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:24:17.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a1931700 1 --2- 172.21.15.203:0/347044703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c151b80 0x7f729c148c40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f729800c710 tx=0x7f7298007b80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:17.723 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f728e7fc700 1 -- 172.21.15.203:0/347044703 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298007020 con 0x7f729c151b80 2021-01-06T22:24:17.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f728e7fc700 1 -- 172.21.15.203:0/347044703 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7298007180 con 0x7f729c151b80 2021-01-06T22:24:17.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a4396700 1 -- 172.21.15.203:0/347044703 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729c15e7f0 con 0x7f729c151b80 2021-01-06T22:24:17.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f728e7fc700 1 -- 172.21.15.203:0/347044703 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980064e0 con 0x7f729c151b80 2021-01-06T22:24:17.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.708+0000 7f72a4396700 1 -- 172.21.15.203:0/347044703 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729c0a8370 con 0x7f729c151b80 2021-01-06T22:24:17.724 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.712+0000 7f728e7fc700 1 -- 172.21.15.203:0/347044703 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f7298010030 con 0x7f729c151b80 2021-01-06T22:24:17.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.712+0000 7f72a4396700 1 -- 172.21.15.203:0/347044703 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729c05d600 con 0x7f729c151b80 2021-01-06T22:24:17.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.712+0000 7f728e7fc700 1 --2- 172.21.15.203:0/347044703 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f727c0563d0 0x7f727c058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:17.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.712+0000 7f728e7fc700 1 -- 172.21.15.203:0/347044703 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7105+0+0 (secure 0 0 0) 0x7f729806eec0 con 0x7f729c151b80 2021-01-06T22:24:17.725 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.712+0000 7f72a2132700 1 --2- 172.21.15.203:0/347044703 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f727c0563d0 0x7f727c058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:17.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.712+0000 7f72a2132700 1 --2- 172.21.15.203:0/347044703 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f727c0563d0 0x7f727c058890 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f7294009500 tx=0x7f7294017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:17.726 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.716+0000 7f728e7fc700 1 -- 172.21.15.203:0/347044703 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f7298044330 con 0x7f729c151b80 2021-01-06T22:24:17.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.944+0000 7f72a4396700 1 -- 172.21.15.203:0/347044703 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f729c0a7f90 con 0x7f729c151b80 2021-01-06T22:24:17.952 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.944+0000 7f728e7fc700 1 -- 172.21.15.203:0/347044703 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+3570 (secure 0 0 0) 0x7f7298001e60 con 0x7f729c151b80 2021-01-06T22:24:17.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.948+0000 7f72a4396700 1 -- 172.21.15.203:0/347044703 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f727c0563d0 msgr2=0x7f727c058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:17.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.948+0000 7f72a4396700 1 --2- 172.21.15.203:0/347044703 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f727c0563d0 0x7f727c058890 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f7294009500 tx=0x7f7294017000).stop 2021-01-06T22:24:17.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.948+0000 7f72a4396700 1 -- 172.21.15.203:0/347044703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c151b80 msgr2=0x7f729c148c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:17.953 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.948+0000 7f72a4396700 1 --2- 172.21.15.203:0/347044703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c151b80 0x7f729c148c40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f729800c710 tx=0x7f7298007b80).stop 2021-01-06T22:24:17.954 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.948+0000 7f72a4396700 1 -- 172.21.15.203:0/347044703 shutdown_connections 2021-01-06T22:24:17.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.948+0000 7f72a4396700 1 --2- 172.21.15.203:0/347044703 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f727c0563d0 0x7f727c058890 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:17.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.948+0000 7f72a4396700 1 --2- 172.21.15.203:0/347044703 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c15d8e0 0x7f729c149130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:17.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.948+0000 7f72a4396700 1 --2- 172.21.15.203:0/347044703 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f729c1505a0 0x7f729c1461a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:17.956 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.948+0000 7f72a4396700 1 --2- 172.21.15.203:0/347044703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c151b80 0x7f729c148c40 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:17.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.948+0000 7f72a4396700 1 -- 172.21.15.203:0/347044703 >> 172.21.15.203:0/347044703 conn(0x7f729c060d80 msgr2=0x7f729c153a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:17.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.948+0000 7f72a4396700 1 -- 172.21.15.203:0/347044703 shutdown_connections 2021-01-06T22:24:17.957 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:17.948+0000 7f72a4396700 1 -- 172.21.15.203:0/347044703 wait complete. 2021-01-06T22:24:17.958 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:24:17.958 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":6052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3262828783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3262828783},{"type":"v1","addr":"172.21.15.203:6837","nonce":3262828783}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:23:20.754099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:24:17.959 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 100 2021-01-06T22:24:17.968 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:24:19.974 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:24:20.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.100+0000 7f82441c4700 1 Processor -- start 2021-01-06T22:24:20.104 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.100+0000 7f82441c4700 1 -- start start 2021-01-06T22:24:20.106 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.100+0000 7f82441c4700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f823c067f20 0x7f823c14e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:20.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.100+0000 7f82441c4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f823c1521b0 0x7f823c0bddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:20.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.100+0000 7f82441c4700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823c0beb40 con 0x7f823c067b40 2021-01-06T22:24:20.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.100+0000 7f82441c4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823c152630 con 0x7f823c1521b0 2021-01-06T22:24:20.107 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.100+0000 7f82441c4700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823c0be600 con 0x7f823c067f20 2021-01-06T22:24:20.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.100+0000 7f8242761700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f823c1521b0 0x7f823c0bddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:20.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.100+0000 7f8242761700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f823c1521b0 0x7f823c0bddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57232/0 (socket says 172.21.15.203:57232) 2021-01-06T22:24:20.108 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.100+0000 7f8242761700 1 -- 172.21.15.203:0/1580246814 learned_addr learned my addr 172.21.15.203:0/1580246814 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:20.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.100+0000 7f824175f700 1 --2- 172.21.15.203:0/1580246814 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f823c067f20 0x7f823c14e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:20.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.100+0000 7f8242761700 1 -- 172.21.15.203:0/1580246814 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f823c067f20 msgr2=0x7f823c14e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:24:20.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.100+0000 7f8242761700 1 --2- 172.21.15.203:0/1580246814 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f823c067f20 0x7f823c14e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:20.109 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.100+0000 7f8242761700 1 -- 172.21.15.203:0/1580246814 >> v1:172.21.15.91:6789/0 conn(0x7f823c067b40 legacy=0x7f823c0b8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:20.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.100+0000 7f8242761700 1 -- 172.21.15.203:0/1580246814 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c0be8a0 con 0x7f823c1521b0 2021-01-06T22:24:20.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.100+0000 7f8242761700 1 --2- 172.21.15.203:0/1580246814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f823c1521b0 0x7f823c0bddb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f822c00b9e0 tx=0x7f822c006ea0).ready entity=mon.1 client_cookie=67f6ee73e967fc6b server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:20.114 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.104+0000 7f8240f5e700 1 -- 172.21.15.203:0/1580246814 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c009810 con 0x7f823c1521b0 2021-01-06T22:24:20.115 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.104+0000 7f8240f5e700 1 -- 172.21.15.203:0/1580246814 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c009970 con 0x7f823c1521b0 2021-01-06T22:24:20.115 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.104+0000 7f8240f5e700 1 -- 172.21.15.203:0/1580246814 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c009cf0 con 0x7f823c1521b0 2021-01-06T22:24:20.115 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.104+0000 7f82441c4700 1 -- 172.21.15.203:0/1580246814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f823c1521b0 msgr2=0x7f823c0bddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:20.115 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.104+0000 7f82441c4700 1 --2- 172.21.15.203:0/1580246814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f823c1521b0 0x7f823c0bddb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f822c00b9e0 tx=0x7f822c006ea0).stop 2021-01-06T22:24:20.115 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.104+0000 7f82441c4700 1 -- 172.21.15.203:0/1580246814 shutdown_connections 2021-01-06T22:24:20.116 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.104+0000 7f82441c4700 1 --2- 172.21.15.203:0/1580246814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f823c1521b0 0x7f823c0bddb0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:20.116 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.104+0000 7f82441c4700 1 --2- 172.21.15.203:0/1580246814 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f823c067f20 0x7f823c14e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:20.117 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.104+0000 7f82441c4700 1 -- 172.21.15.203:0/1580246814 >> 172.21.15.203:0/1580246814 conn(0x7f823c056cd0 msgr2=0x7f823c0570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:20.117 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.104+0000 7f82441c4700 1 -- 172.21.15.203:0/1580246814 shutdown_connections 2021-01-06T22:24:20.117 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.104+0000 7f82441c4700 1 -- 172.21.15.203:0/1580246814 wait complete. 2021-01-06T22:24:20.118 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.104+0000 7f82441c4700 1 Processor -- start 2021-01-06T22:24:20.118 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.104+0000 7f82441c4700 1 -- start start 2021-01-06T22:24:20.118 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f82441c4700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f823c067b40 0x7f823c0c1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:20.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f82441c4700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f823c067f20 0x7f823c146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:20.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f82441c4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f823c1521b0 0x7f823c147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:20.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f82441c4700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c147510 con 0x7f823c067f20 2021-01-06T22:24:20.119 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f82441c4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c15ec60 con 0x7f823c1521b0 2021-01-06T22:24:20.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f82441c4700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823c147d30 con 0x7f823c067b40 2021-01-06T22:24:20.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f8242761700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f823c1521b0 0x7f823c147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:20.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f8242761700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f823c1521b0 0x7f823c147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57238/0 (socket says 172.21.15.203:57238) 2021-01-06T22:24:20.120 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f8242761700 1 -- 172.21.15.203:0/3235211627 learned_addr learned my addr 172.21.15.203:0/3235211627 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:20.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f824175f700 1 --2- 172.21.15.203:0/3235211627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f823c067f20 0x7f823c146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:20.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f8241f60700 1 --2- 172.21.15.203:0/3235211627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f823c067b40 0x7f823c0c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:20.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f8242761700 1 -- 172.21.15.203:0/3235211627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f823c067b40 msgr2=0x7f823c0c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:20.121 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f8242761700 1 --2- 172.21.15.203:0/3235211627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f823c067b40 0x7f823c0c1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:20.122 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f8242761700 1 -- 172.21.15.203:0/3235211627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f823c067f20 msgr2=0x7f823c146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:20.122 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f8242761700 1 --2- 172.21.15.203:0/3235211627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f823c067f20 0x7f823c146b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:20.122 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f8242761700 1 -- 172.21.15.203:0/3235211627 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c147fd0 con 0x7f823c1521b0 2021-01-06T22:24:20.123 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f8242761700 1 --2- 172.21.15.203:0/3235211627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f823c1521b0 0x7f823c147020 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f822c0069e0 tx=0x7f822c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:20.123 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f8229ffb700 1 -- 172.21.15.203:0/3235211627 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c009810 con 0x7f823c1521b0 2021-01-06T22:24:20.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f8229ffb700 1 -- 172.21.15.203:0/3235211627 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c01c420 con 0x7f823c1521b0 2021-01-06T22:24:20.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f8229ffb700 1 -- 172.21.15.203:0/3235211627 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c01c7c0 con 0x7f823c1521b0 2021-01-06T22:24:20.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f82441c4700 1 -- 172.21.15.203:0/3235211627 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823c148270 con 0x7f823c1521b0 2021-01-06T22:24:20.125 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f82441c4700 1 -- 172.21.15.203:0/3235211627 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823c057930 con 0x7f823c1521b0 2021-01-06T22:24:20.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f8229ffb700 1 -- 172.21.15.203:0/3235211627 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f822c00f070 con 0x7f823c1521b0 2021-01-06T22:24:20.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f821b7fe700 1 -- 172.21.15.203:0/3235211627 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823c148400 con 0x7f823c1521b0 2021-01-06T22:24:20.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f8229ffb700 1 --2- 172.21.15.203:0/3235211627 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f821c056400 0x7f821c0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:20.126 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.108+0000 7f8229ffb700 1 -- 172.21.15.203:0/3235211627 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7105+0+0 (secure 0 0 0) 0x7f822c071100 con 0x7f823c1521b0 2021-01-06T22:24:20.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.116+0000 7f8241f60700 1 --2- 172.21.15.203:0/3235211627 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f821c056400 0x7f821c0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:20.127 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.116+0000 7f8241f60700 1 --2- 172.21.15.203:0/3235211627 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f821c056400 0x7f821c0588c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f8234002bd0 tx=0x7f8234007f90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:20.128 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.116+0000 7f8229ffb700 1 -- 172.21.15.203:0/3235211627 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f822c046650 con 0x7f823c1521b0 2021-01-06T22:24:20.367 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.364+0000 7f821b7fe700 1 -- 172.21.15.203:0/3235211627 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f823c05d600 con 0x7f823c1521b0 2021-01-06T22:24:20.368 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.364+0000 7f8229ffb700 1 -- 172.21.15.203:0/3235211627 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+3570 (secure 0 0 0) 0x7f822c049c70 con 0x7f823c1521b0 2021-01-06T22:24:20.368 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:24:20.368 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":6052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3262828783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3262828783},{"type":"v1","addr":"172.21.15.203:6837","nonce":3262828783}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:23:20.754099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:24:20.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.364+0000 7f821b7fe700 1 -- 172.21.15.203:0/3235211627 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f821c056400 msgr2=0x7f821c0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:20.370 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.364+0000 7f821b7fe700 1 --2- 172.21.15.203:0/3235211627 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f821c056400 0x7f821c0588c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f8234002bd0 tx=0x7f8234007f90).stop 2021-01-06T22:24:20.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.364+0000 7f821b7fe700 1 -- 172.21.15.203:0/3235211627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f823c1521b0 msgr2=0x7f823c147020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:20.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.364+0000 7f821b7fe700 1 --2- 172.21.15.203:0/3235211627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f823c1521b0 0x7f823c147020 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f822c0069e0 tx=0x7f822c016000).stop 2021-01-06T22:24:20.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.364+0000 7f821b7fe700 1 -- 172.21.15.203:0/3235211627 shutdown_connections 2021-01-06T22:24:20.371 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.364+0000 7f821b7fe700 1 --2- 172.21.15.203:0/3235211627 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f821c056400 0x7f821c0588c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:20.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.364+0000 7f821b7fe700 1 --2- 172.21.15.203:0/3235211627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f823c1521b0 0x7f823c147020 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:20.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.364+0000 7f821b7fe700 1 --2- 172.21.15.203:0/3235211627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f823c067b40 0x7f823c0c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:20.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.364+0000 7f821b7fe700 1 --2- 172.21.15.203:0/3235211627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f823c067f20 0x7f823c146b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:20.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.364+0000 7f821b7fe700 1 -- 172.21.15.203:0/3235211627 >> 172.21.15.203:0/3235211627 conn(0x7f823c056cd0 msgr2=0x7f823c0bdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:20.372 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.364+0000 7f821b7fe700 1 -- 172.21.15.203:0/3235211627 shutdown_connections 2021-01-06T22:24:20.373 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:20.364+0000 7f821b7fe700 1 -- 172.21.15.203:0/3235211627 wait complete. 2021-01-06T22:24:20.373 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 100 2021-01-06T22:24:20.387 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:24:22.388 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:24:22.543 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 Processor -- start 2021-01-06T22:24:22.544 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 -- start start 2021-01-06T22:24:22.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6a84063010 0x7f6a8414e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:22.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a840633f0 0x7f6a84064b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:22.546 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6a840be1b0 0x7f6a84147790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:22.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a8414ec60 con 0x7f6a840be1b0 2021-01-06T22:24:22.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a840be990 con 0x7f6a84063010 2021-01-06T22:24:22.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a840be6f0 con 0x7f6a840633f0 2021-01-06T22:24:22.547 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a893a6700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6a84063010 0x7f6a8414e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:22.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a893a6700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6a84063010 0x7f6a8414e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57306/0 (socket says 172.21.15.203:57306) 2021-01-06T22:24:22.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a893a6700 1 -- 172.21.15.203:0/1054775292 learned_addr learned my addr 172.21.15.203:0/1054775292 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:22.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a88ba5700 1 --2- 172.21.15.203:0/1054775292 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a840633f0 0x7f6a84064b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:22.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a89ba7700 1 --2- 172.21.15.203:0/1054775292 >> v2:172.21.15.91:3300/0 conn(0x7f6a840be1b0 0x7f6a84147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:22.548 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a893a6700 1 -- 172.21.15.203:0/1054775292 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a840633f0 msgr2=0x7f6a84064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:24:22.549 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a893a6700 1 --2- 172.21.15.203:0/1054775292 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a840633f0 0x7f6a84064b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:22.549 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a893a6700 1 -- 172.21.15.203:0/1054775292 >> v2:172.21.15.91:3300/0 conn(0x7f6a840be1b0 msgr2=0x7f6a84147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:24:22.549 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a893a6700 1 --2- 172.21.15.203:0/1054775292 >> v2:172.21.15.91:3300/0 conn(0x7f6a840be1b0 0x7f6a84147790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:22.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a893a6700 1 -- 172.21.15.203:0/1054775292 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a840c1050 con 0x7f6a84063010 2021-01-06T22:24:22.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a89ba7700 1 --2- 172.21.15.203:0/1054775292 >> v2:172.21.15.91:3300/0 conn(0x7f6a840be1b0 0x7f6a84147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:24:22.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a88ba5700 1 --2- 172.21.15.203:0/1054775292 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a840633f0 0x7f6a84064b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:24:22.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a893a6700 1 --2- 172.21.15.203:0/1054775292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a84063010 0x7f6a8414e770 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f6a840667e0 tx=0x7f6a80006ea0).ready entity=mon.1 client_cookie=3c745c9a050653d3 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:22.550 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a73fff700 1 -- 172.21.15.203:0/1054775292 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800098d0 con 0x7f6a84063010 2021-01-06T22:24:22.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a73fff700 1 -- 172.21.15.203:0/1054775292 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a80009a30 con 0x7f6a84063010 2021-01-06T22:24:22.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a73fff700 1 -- 172.21.15.203:0/1054775292 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a80009db0 con 0x7f6a84063010 2021-01-06T22:24:22.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 -- 172.21.15.203:0/1054775292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a84063010 msgr2=0x7f6a8414e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:22.551 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 --2- 172.21.15.203:0/1054775292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a84063010 0x7f6a8414e770 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f6a840667e0 tx=0x7f6a80006ea0).stop 2021-01-06T22:24:22.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 -- 172.21.15.203:0/1054775292 shutdown_connections 2021-01-06T22:24:22.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 --2- 172.21.15.203:0/1054775292 >> v2:172.21.15.91:3300/0 conn(0x7f6a840be1b0 0x7f6a84147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:22.552 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 --2- 172.21.15.203:0/1054775292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a84063010 0x7f6a8414e770 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:22.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 --2- 172.21.15.203:0/1054775292 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a840633f0 0x7f6a84064b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:22.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 -- 172.21.15.203:0/1054775292 >> 172.21.15.203:0/1054775292 conn(0x7f6a84067e20 msgr2=0x7f6a84068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:22.553 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 -- 172.21.15.203:0/1054775292 shutdown_connections 2021-01-06T22:24:22.554 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 -- 172.21.15.203:0/1054775292 wait complete. 2021-01-06T22:24:22.556 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 Processor -- start 2021-01-06T22:24:22.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.540+0000 7f6a8a3a8700 1 -- start start 2021-01-06T22:24:22.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a8a3a8700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a84063010 0x7f6a8415dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:22.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a8a3a8700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a840633f0 0x7f6a841612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:22.557 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a8a3a8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a840be1b0 0x7f6a84144e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:22.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a8a3a8700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a840b7e90 con 0x7f6a84063010 2021-01-06T22:24:22.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a8a3a8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8415e390 con 0x7f6a840be1b0 2021-01-06T22:24:22.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a8a3a8700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a840b9870 con 0x7f6a840633f0 2021-01-06T22:24:22.558 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a89ba7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a840be1b0 0x7f6a84144e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:22.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a89ba7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a840be1b0 0x7f6a84144e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57316/0 (socket says 172.21.15.203:57316) 2021-01-06T22:24:22.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a89ba7700 1 -- 172.21.15.203:0/1763917987 learned_addr learned my addr 172.21.15.203:0/1763917987 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:22.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a893a6700 1 --2- 172.21.15.203:0/1763917987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a84063010 0x7f6a8415dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:22.559 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a88ba5700 1 --2- 172.21.15.203:0/1763917987 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a840633f0 0x7f6a841612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:22.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a89ba7700 1 -- 172.21.15.203:0/1763917987 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a840633f0 msgr2=0x7f6a841612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:22.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a89ba7700 1 --2- 172.21.15.203:0/1763917987 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a840633f0 0x7f6a841612a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:22.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a89ba7700 1 -- 172.21.15.203:0/1763917987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a84063010 msgr2=0x7f6a8415dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:22.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a89ba7700 1 --2- 172.21.15.203:0/1763917987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a84063010 0x7f6a8415dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:22.560 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a89ba7700 1 -- 172.21.15.203:0/1763917987 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8413e7d0 con 0x7f6a840be1b0 2021-01-06T22:24:22.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a89ba7700 1 --2- 172.21.15.203:0/1763917987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a840be1b0 0x7f6a84144e80 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f6a7c00db30 tx=0x7f6a7c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:22.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a717fa700 1 -- 172.21.15.203:0/1763917987 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00b900 con 0x7f6a840be1b0 2021-01-06T22:24:22.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a717fa700 1 -- 172.21.15.203:0/1763917987 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7c00ba60 con 0x7f6a840be1b0 2021-01-06T22:24:22.561 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a8a3a8700 1 -- 172.21.15.203:0/1763917987 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8413ea70 con 0x7f6a840be1b0 2021-01-06T22:24:22.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a717fa700 1 -- 172.21.15.203:0/1763917987 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00be00 con 0x7f6a840be1b0 2021-01-06T22:24:22.562 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a8a3a8700 1 -- 172.21.15.203:0/1763917987 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a84160b60 con 0x7f6a840be1b0 2021-01-06T22:24:22.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a8a3a8700 1 -- 172.21.15.203:0/1763917987 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8413ef10 con 0x7f6a840be1b0 2021-01-06T22:24:22.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a717fa700 1 -- 172.21.15.203:0/1763917987 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f6a7c012630 con 0x7f6a840be1b0 2021-01-06T22:24:22.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a717fa700 1 --2- 172.21.15.203:0/1763917987 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a64056400 0x7f6a640588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:22.563 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a717fa700 1 -- 172.21.15.203:0/1763917987 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7105+0+0 (secure 0 0 0) 0x7f6a7c06fe50 con 0x7f6a840be1b0 2021-01-06T22:24:22.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.544+0000 7f6a893a6700 1 --2- 172.21.15.203:0/1763917987 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a64056400 0x7f6a640588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:22.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.548+0000 7f6a893a6700 1 --2- 172.21.15.203:0/1763917987 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a64056400 0x7f6a640588c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f6a8000b190 tx=0x7f6a80016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:22.564 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.548+0000 7f6a717fa700 1 -- 172.21.15.203:0/1763917987 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f6a7c0452c0 con 0x7f6a840be1b0 2021-01-06T22:24:22.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.752+0000 7f6a8a3a8700 1 -- 172.21.15.203:0/1763917987 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a840450a0 con 0x7f6a840be1b0 2021-01-06T22:24:22.759 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.752+0000 7f6a717fa700 1 -- 172.21.15.203:0/1763917987 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+3570 (secure 0 0 0) 0x7f6a7c0102e0 con 0x7f6a840be1b0 2021-01-06T22:24:22.760 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:24:22.760 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":6052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3262828783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3262828783},{"type":"v1","addr":"172.21.15.203:6837","nonce":3262828783}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:23:20.754099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:24:22.761 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.756+0000 7f6a8a3a8700 1 -- 172.21.15.203:0/1763917987 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a64056400 msgr2=0x7f6a640588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:22.762 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.756+0000 7f6a8a3a8700 1 --2- 172.21.15.203:0/1763917987 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a64056400 0x7f6a640588c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f6a8000b190 tx=0x7f6a80016000).stop 2021-01-06T22:24:22.764 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.756+0000 7f6a8a3a8700 1 -- 172.21.15.203:0/1763917987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a840be1b0 msgr2=0x7f6a84144e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:22.765 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.756+0000 7f6a8a3a8700 1 --2- 172.21.15.203:0/1763917987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a840be1b0 0x7f6a84144e80 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f6a7c00db30 tx=0x7f6a7c008040).stop 2021-01-06T22:24:22.765 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.756+0000 7f6a8a3a8700 1 -- 172.21.15.203:0/1763917987 shutdown_connections 2021-01-06T22:24:22.765 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.756+0000 7f6a8a3a8700 1 --2- 172.21.15.203:0/1763917987 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f6a64056400 0x7f6a640588c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:22.766 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.756+0000 7f6a8a3a8700 1 --2- 172.21.15.203:0/1763917987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a840be1b0 0x7f6a84144e80 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:22.766 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.756+0000 7f6a8a3a8700 1 --2- 172.21.15.203:0/1763917987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a84063010 0x7f6a8415dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:22.767 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.756+0000 7f6a8a3a8700 1 --2- 172.21.15.203:0/1763917987 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a840633f0 0x7f6a841612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:22.767 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.756+0000 7f6a8a3a8700 1 -- 172.21.15.203:0/1763917987 >> 172.21.15.203:0/1763917987 conn(0x7f6a84067e20 msgr2=0x7f6a84147560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:22.767 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.756+0000 7f6a8a3a8700 1 -- 172.21.15.203:0/1763917987 shutdown_connections 2021-01-06T22:24:22.767 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:22.756+0000 7f6a8a3a8700 1 -- 172.21.15.203:0/1763917987 wait complete. 2021-01-06T22:24:22.768 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 100 2021-01-06T22:24:22.776 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:24:24.778 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:24:25.374 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.368+0000 7fe50cfff700 1 Processor -- start 2021-01-06T22:24:25.978 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.368+0000 7fe50cfff700 1 -- start start 2021-01-06T22:24:25.984 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:24:25.985 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.368+0000 7fe50cfff700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe508063010 0x7fe50814e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:25.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.368+0000 7fe50cfff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5080633f0 0x7fe508064b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:25.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.368+0000 7fe50cfff700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5080be990 con 0x7fe5080be1b0 2021-01-06T22:24:25.986 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.368+0000 7fe50cfff700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe50814ec60 con 0x7fe508063010 2021-01-06T22:24:25.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.368+0000 7fe50cfff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5080be6f0 con 0x7fe5080633f0 2021-01-06T22:24:25.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.368+0000 7fe5077fe700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe508063010 0x7fe50814e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:25.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.368+0000 7fe4fffff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5080633f0 0x7fe508064b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:25.987 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.368+0000 7fe5077fe700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe508063010 0x7fe50814e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:37018/0 (socket says 172.21.15.203:37018) 2021-01-06T22:24:25.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe5077fe700 1 -- 172.21.15.203:0/3839618308 learned_addr learned my addr 172.21.15.203:0/3839618308 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:25.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.368+0000 7fe507fff700 1 --1- 172.21.15.203:0/3839618308 >> v1:172.21.15.91:6789/0 conn(0x7fe5080be1b0 0x7fe508147790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:50654/0 (socket says 172.21.15.203:50654) 2021-01-06T22:24:25.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe5077fe700 1 -- 172.21.15.203:0/3839618308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5080633f0 msgr2=0x7fe508064b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:24:25.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe5077fe700 1 --2- 172.21.15.203:0/3839618308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5080633f0 0x7fe508064b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:25.988 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe5077fe700 1 -- 172.21.15.203:0/3839618308 >> v1:172.21.15.91:6789/0 conn(0x7fe5080be1b0 legacy=0x7fe508147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:25.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe5077fe700 1 -- 172.21.15.203:0/3839618308 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5080c1050 con 0x7fe508063010 2021-01-06T22:24:25.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe5077fe700 1 --2- 172.21.15.203:0/3839618308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe508063010 0x7fe50814e770 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7fe4f800b9e0 tx=0x7fe4f8006ea0).ready entity=mon.0 client_cookie=6d72f97f78ea15de server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:25.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe506ffd700 1 -- 172.21.15.203:0/3839618308 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8009710 con 0x7fe508063010 2021-01-06T22:24:25.989 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe506ffd700 1 -- 172.21.15.203:0/3839618308 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f8009870 con 0x7fe508063010 2021-01-06T22:24:25.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe506ffd700 1 -- 172.21.15.203:0/3839618308 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8009bf0 con 0x7fe508063010 2021-01-06T22:24:25.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe50cfff700 1 -- 172.21.15.203:0/3839618308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe508063010 msgr2=0x7fe50814e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:25.990 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe50cfff700 1 --2- 172.21.15.203:0/3839618308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe508063010 0x7fe50814e770 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7fe4f800b9e0 tx=0x7fe4f8006ea0).stop 2021-01-06T22:24:25.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe50cfff700 1 -- 172.21.15.203:0/3839618308 shutdown_connections 2021-01-06T22:24:25.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe50cfff700 1 --2- 172.21.15.203:0/3839618308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe508063010 0x7fe50814e770 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:25.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe50cfff700 1 --2- 172.21.15.203:0/3839618308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5080633f0 0x7fe508064b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:25.991 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe50cfff700 1 -- 172.21.15.203:0/3839618308 >> 172.21.15.203:0/3839618308 conn(0x7fe508067e20 msgr2=0x7fe508068230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:25.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe50cfff700 1 -- 172.21.15.203:0/3839618308 shutdown_connections 2021-01-06T22:24:25.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe50cfff700 1 -- 172.21.15.203:0/3839618308 wait complete. 2021-01-06T22:24:25.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe50cfff700 1 Processor -- start 2021-01-06T22:24:25.992 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.372+0000 7fe50cfff700 1 -- start start 2021-01-06T22:24:25.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe50cfff700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe508063010 0x7fe50814ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:25.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe50cfff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5080633f0 0x7fe5081426b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:25.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe50cfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5080be1b0 0x7fe50814aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:25.993 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe50cfff700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5080b7880 con 0x7fe5080633f0 2021-01-06T22:24:25.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe50cfff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50814ffa0 con 0x7fe5080be1b0 2021-01-06T22:24:25.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe50cfff700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe508150660 con 0x7fe508063010 2021-01-06T22:24:25.994 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe5077fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe508063010 0x7fe50814ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:25.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe507fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5080be1b0 0x7fe50814aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:25.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe4fffff700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5080633f0 0x7fe5081426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:25.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe5077fe700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe508063010 0x7fe50814ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.203:34258/0 (socket says 172.21.15.203:34258) 2021-01-06T22:24:25.995 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe507fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5080be1b0 0x7fe50814aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57412/0 (socket says 172.21.15.203:57412) 2021-01-06T22:24:25.996 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe5077fe700 1 -- 172.21.15.203:0/1358504977 learned_addr learned my addr 172.21.15.203:0/1358504977 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:25.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe5077fe700 1 -- 172.21.15.203:0/1358504977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5080be1b0 msgr2=0x7fe50814aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:25.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe5077fe700 1 --2- 172.21.15.203:0/1358504977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5080be1b0 0x7fe50814aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:25.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe5077fe700 1 -- 172.21.15.203:0/1358504977 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5080633f0 msgr2=0x7fe5081426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:25.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe5077fe700 1 --2- 172.21.15.203:0/1358504977 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5080633f0 0x7fe5081426b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:25.997 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe5077fe700 1 -- 172.21.15.203:0/1358504977 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe508142ba0 con 0x7fe508063010 2021-01-06T22:24:25.998 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe4fffff700 1 --2- 172.21.15.203:0/1358504977 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5080633f0 0x7fe5081426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:24:25.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe5077fe700 1 --2- 172.21.15.203:0/1358504977 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe508063010 0x7fe50814ddb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fe508068a30 tx=0x7fe4f8016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:25.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe4ff7fe700 1 -- 172.21.15.203:0/1358504977 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8011b40 con 0x7fe508063010 2021-01-06T22:24:25.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe4ff7fe700 1 -- 172.21.15.203:0/1358504977 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f8011ca0 con 0x7fe508063010 2021-01-06T22:24:25.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe50cfff700 1 -- 172.21.15.203:0/1358504977 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5081501d0 con 0x7fe508063010 2021-01-06T22:24:25.999 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe4ff7fe700 1 -- 172.21.15.203:0/1358504977 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f801d660 con 0x7fe508063010 2021-01-06T22:24:26.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe50cfff700 1 -- 172.21.15.203:0/1358504977 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe508150b30 con 0x7fe508063010 2021-01-06T22:24:26.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe4ff7fe700 1 -- 172.21.15.203:0/1358504977 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fe4f800f070 con 0x7fe508063010 2021-01-06T22:24:26.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe4ff7fe700 1 --2- 172.21.15.203:0/1358504977 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe4ec056400 0x7fe4ec0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:26.000 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe4ff7fe700 1 -- 172.21.15.203:0/1358504977 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7105+0+0 (secure 0 0 0) 0x7fe4f8071020 con 0x7fe508063010 2021-01-06T22:24:26.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe4fffff700 1 --2- 172.21.15.203:0/1358504977 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe4ec056400 0x7fe4ec0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:26.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.376+0000 7fe4fffff700 1 --2- 172.21.15.203:0/1358504977 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe4ec056400 0x7fe4ec0588c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7fe50814ec30 tx=0x7fe4f0006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:26.001 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.380+0000 7fe50cfff700 1 -- 172.21.15.203:0/1358504977 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe508142ba0 con 0x7fe508063010 2021-01-06T22:24:26.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.388+0000 7fe4ff7fe700 1 -- 172.21.15.203:0/1358504977 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fe508142ba0 con 0x7fe508063010 2021-01-06T22:24:26.002 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.588+0000 7fe50cfff700 1 -- 172.21.15.203:0/1358504977 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe508142ba0 con 0x7fe508063010 2021-01-06T22:24:26.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.588+0000 7fe4ff7fe700 1 -- 172.21.15.203:0/1358504977 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+3570 (secure 0 0 0) 0x7fe508142ba0 con 0x7fe508063010 2021-01-06T22:24:26.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.592+0000 7fe50cfff700 1 -- 172.21.15.203:0/1358504977 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe4ec056400 msgr2=0x7fe4ec0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:26.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.592+0000 7fe50cfff700 1 --2- 172.21.15.203:0/1358504977 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe4ec056400 0x7fe4ec0588c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7fe50814ec30 tx=0x7fe4f0006de0).stop 2021-01-06T22:24:26.003 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.592+0000 7fe50cfff700 1 -- 172.21.15.203:0/1358504977 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe508063010 msgr2=0x7fe50814ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:26.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.592+0000 7fe50cfff700 1 --2- 172.21.15.203:0/1358504977 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe508063010 0x7fe50814ddb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fe508068a30 tx=0x7fe4f8016000).stop 2021-01-06T22:24:26.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.592+0000 7fe50cfff700 1 -- 172.21.15.203:0/1358504977 shutdown_connections 2021-01-06T22:24:26.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.592+0000 7fe50cfff700 1 --2- 172.21.15.203:0/1358504977 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fe4ec056400 0x7fe4ec0588c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:26.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.592+0000 7fe50cfff700 1 --2- 172.21.15.203:0/1358504977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5080be1b0 0x7fe50814aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:26.004 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.592+0000 7fe50cfff700 1 --2- 172.21.15.203:0/1358504977 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe508063010 0x7fe50814ddb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:26.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.592+0000 7fe50cfff700 1 --2- 172.21.15.203:0/1358504977 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5080633f0 0x7fe5081426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:26.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.592+0000 7fe50cfff700 1 -- 172.21.15.203:0/1358504977 >> 172.21.15.203:0/1358504977 conn(0x7fe508067e20 msgr2=0x7fe508146750 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:26.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.592+0000 7fe50cfff700 1 -- 172.21.15.203:0/1358504977 shutdown_connections 2021-01-06T22:24:26.005 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:25.592+0000 7fe50cfff700 1 -- 172.21.15.203:0/1358504977 wait complete. 2021-01-06T22:24:26.006 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 100 2021-01-06T22:24:26.006 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:24:26.006 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":6052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3262828783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3262828783},{"type":"v1","addr":"172.21.15.203:6837","nonce":3262828783}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:23:20.754099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5836,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5836":{"gid":5836,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":34,"addr":"172.21.15.203:6835/3960786803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3960786803},{"type":"v1","addr":"172.21.15.203:6835","nonce":3960786803}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0},"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:24:26.007 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2021-01-06T22:24:26.009 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:24:28.008 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:24:28.163 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.160+0000 7f478b7c6700 1 Processor -- start 2021-01-06T22:24:28.164 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.160+0000 7f478b7c6700 1 -- start start 2021-01-06T22:24:28.165 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.160+0000 7f478b7c6700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47840c04c0 0x7f4784066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:28.166 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.160+0000 7f478b7c6700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47840bb860 con 0x7f47840c08a0 2021-01-06T22:24:28.166 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.160+0000 7f478b7c6700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478414b940 con 0x7f478414ab60 2021-01-06T22:24:28.167 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.160+0000 7f478b7c6700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478414b720 con 0x7f47840c04c0 2021-01-06T22:24:28.167 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.160+0000 7f4789562700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47840c04c0 0x7f4784066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:28.167 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.160+0000 7f4788d61700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f47840c08a0 0x7f4784147cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:50740/0 (socket says 172.21.15.203:50740) 2021-01-06T22:24:28.168 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.160+0000 7f4788d61700 1 -- 172.21.15.203:0/3177390351 learned_addr learned my addr 172.21.15.203:0/3177390351 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:24:28.168 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.160+0000 7f4789562700 1 -- 172.21.15.203:0/3177390351 >> v1:172.21.15.203:6789/0 conn(0x7f478414ab60 legacy=0x7f478414b010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-01-06T22:24:28.168 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.160+0000 7f4789562700 1 -- 172.21.15.203:0/3177390351 >> v1:172.21.15.91:6789/0 conn(0x7f47840c08a0 legacy=0x7f4784147cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:28.169 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.160+0000 7f4789562700 1 -- 172.21.15.203:0/3177390351 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47840bba90 con 0x7f47840c04c0 2021-01-06T22:24:28.169 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.160+0000 7f4789562700 1 --2- 172.21.15.203:0/3177390351 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47840c04c0 0x7f4784066cd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f477c00bf50 tx=0x7f477c00b460).ready entity=mon.2 client_cookie=9a962120025b5d5c server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:28.170 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.164+0000 7f4773fff700 1 -- 172.21.15.203:0/3177390351 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c010470 con 0x7f47840c04c0 2021-01-06T22:24:28.171 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.164+0000 7f4773fff700 1 -- 172.21.15.203:0/3177390351 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477c0105d0 con 0x7f47840c04c0 2021-01-06T22:24:28.171 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.164+0000 7f4773fff700 1 -- 172.21.15.203:0/3177390351 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c010950 con 0x7f47840c04c0 2021-01-06T22:24:28.171 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.164+0000 7f478b7c6700 1 -- 172.21.15.203:0/3177390351 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47840c04c0 msgr2=0x7f4784066cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:28.172 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.164+0000 7f478b7c6700 1 --2- 172.21.15.203:0/3177390351 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47840c04c0 0x7f4784066cd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f477c00bf50 tx=0x7f477c00b460).stop 2021-01-06T22:24:28.173 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.168+0000 7f478b7c6700 1 -- 172.21.15.203:0/3177390351 shutdown_connections 2021-01-06T22:24:28.173 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.168+0000 7f478b7c6700 1 --2- 172.21.15.203:0/3177390351 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47840c04c0 0x7f4784066cd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:28.173 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.168+0000 7f478b7c6700 1 -- 172.21.15.203:0/3177390351 >> 172.21.15.203:0/3177390351 conn(0x7f4784056cd0 msgr2=0x7f47840626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:28.174 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.168+0000 7f478b7c6700 1 -- 172.21.15.203:0/3177390351 shutdown_connections 2021-01-06T22:24:28.174 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.168+0000 7f478b7c6700 1 -- 172.21.15.203:0/3177390351 wait complete. 2021-01-06T22:24:28.175 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.168+0000 7f478b7c6700 1 Processor -- start 2021-01-06T22:24:28.176 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.168+0000 7f478b7c6700 1 -- start start 2021-01-06T22:24:28.176 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f478b7c6700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47840c04c0 0x7f47840be2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:28.176 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f478b7c6700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f47840c08a0 0x7f47840bf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:28.177 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f478b7c6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f478414ab60 0x7f478413ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:28.177 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f478b7c6700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4784068320 con 0x7f47840c08a0 2021-01-06T22:24:28.177 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f478b7c6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47840bec00 con 0x7f478414ab60 2021-01-06T22:24:28.178 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f478b7c6700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4784063e60 con 0x7f47840c04c0 2021-01-06T22:24:28.178 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f4789562700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47840c04c0 0x7f47840be2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:28.180 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f4789d63700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f478414ab60 0x7f478413ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:28.180 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f4789d63700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f478414ab60 0x7f478413ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57496/0 (socket says 172.21.15.203:57496) 2021-01-06T22:24:28.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f4789d63700 1 -- 172.21.15.203:0/4045968675 learned_addr learned my addr 172.21.15.203:0/4045968675 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:28.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f4789d63700 1 -- 172.21.15.203:0/4045968675 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47840c04c0 msgr2=0x7f47840be2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:28.181 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f4789d63700 1 --2- 172.21.15.203:0/4045968675 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47840c04c0 0x7f47840be2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:28.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f4789d63700 1 -- 172.21.15.203:0/4045968675 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f47840c08a0 msgr2=0x7f47840bf2d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2021-01-06T22:24:28.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f4788d61700 1 --2- 172.21.15.203:0/4045968675 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f47840c08a0 0x7f47840bf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:28.182 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f4789d63700 1 --2- 172.21.15.203:0/4045968675 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f47840c08a0 0x7f47840bf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:28.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f4789d63700 1 -- 172.21.15.203:0/4045968675 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4784149380 con 0x7f478414ab60 2021-01-06T22:24:28.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f4789562700 1 --2- 172.21.15.203:0/4045968675 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47840c04c0 0x7f47840be2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:24:28.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f4789d63700 1 --2- 172.21.15.203:0/4045968675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f478414ab60 0x7f478413ff40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f4774009710 tx=0x7f4774006cd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:28.183 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f47717fa700 1 -- 172.21.15.203:0/4045968675 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774000ea0 con 0x7f478414ab60 2021-01-06T22:24:28.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f47717fa700 1 -- 172.21.15.203:0/4045968675 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477400d560 con 0x7f478414ab60 2021-01-06T22:24:28.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f478b7c6700 1 -- 172.21.15.203:0/4045968675 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4784149620 con 0x7f478414ab60 2021-01-06T22:24:28.184 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f47717fa700 1 -- 172.21.15.203:0/4045968675 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477400d920 con 0x7f478414ab60 2021-01-06T22:24:28.185 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.172+0000 7f478b7c6700 1 -- 172.21.15.203:0/4045968675 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47840bee30 con 0x7f478414ab60 2021-01-06T22:24:28.187 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.176+0000 7f47717fa700 1 -- 172.21.15.203:0/4045968675 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f477400f030 con 0x7f478414ab60 2021-01-06T22:24:28.188 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.176+0000 7f47717fa700 1 --2- 172.21.15.203:0/4045968675 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f47640563d0 0x7f4764058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:28.188 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.176+0000 7f47717fa700 1 -- 172.21.15.203:0/4045968675 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7191+0+0 (secure 0 0 0) 0x7f477406ede0 con 0x7f478414ab60 2021-01-06T22:24:28.188 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.180+0000 7f478b7c6700 1 -- 172.21.15.203:0/4045968675 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47840beff0 con 0x7f478414ab60 2021-01-06T22:24:28.189 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.180+0000 7f4789562700 1 --2- 172.21.15.203:0/4045968675 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f47640563d0 0x7f4764058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:28.189 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.180+0000 7f4789562700 1 --2- 172.21.15.203:0/4045968675 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f47640563d0 0x7f4764058890 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f477c00dbf0 tx=0x7f477c00b460).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:28.189 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.184+0000 7f47717fa700 1 -- 172.21.15.203:0/4045968675 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f47740440c0 con 0x7f478414ab60 2021-01-06T22:24:28.453 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.448+0000 7f478b7c6700 1 -- 172.21.15.203:0/4045968675 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f478405cfe0 con 0x7f478414ab60 2021-01-06T22:24:28.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.448+0000 7f47717fa700 1 -- 172.21.15.203:0/4045968675 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+3215 (secure 0 0 0) 0x7f4774043090 con 0x7f478414ab60 2021-01-06T22:24:28.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.452+0000 7f478b7c6700 1 -- 172.21.15.203:0/4045968675 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f47640563d0 msgr2=0x7f4764058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:28.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.452+0000 7f478b7c6700 1 --2- 172.21.15.203:0/4045968675 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f47640563d0 0x7f4764058890 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f477c00dbf0 tx=0x7f477c00b460).stop 2021-01-06T22:24:28.459 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.452+0000 7f478b7c6700 1 -- 172.21.15.203:0/4045968675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f478414ab60 msgr2=0x7f478413ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:28.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.452+0000 7f478b7c6700 1 --2- 172.21.15.203:0/4045968675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f478414ab60 0x7f478413ff40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f4774009710 tx=0x7f4774006cd0).stop 2021-01-06T22:24:28.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.452+0000 7f478b7c6700 1 -- 172.21.15.203:0/4045968675 shutdown_connections 2021-01-06T22:24:28.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.452+0000 7f478b7c6700 1 --2- 172.21.15.203:0/4045968675 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f47640563d0 0x7f4764058890 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:28.460 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.452+0000 7f478b7c6700 1 --2- 172.21.15.203:0/4045968675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f478414ab60 0x7f478413ff40 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:28.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.452+0000 7f478b7c6700 1 --2- 172.21.15.203:0/4045968675 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47840c04c0 0x7f47840be2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:28.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.452+0000 7f478b7c6700 1 --2- 172.21.15.203:0/4045968675 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f47840c08a0 0x7f47840bf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:28.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.452+0000 7f478b7c6700 1 -- 172.21.15.203:0/4045968675 >> 172.21.15.203:0/4045968675 conn(0x7f4784056cd0 msgr2=0x7f47840626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:28.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.452+0000 7f478b7c6700 1 -- 172.21.15.203:0/4045968675 shutdown_connections 2021-01-06T22:24:28.461 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.452+0000 7f478b7c6700 1 -- 172.21.15.203:0/4045968675 wait complete. 2021-01-06T22:24:28.462 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 102 2021-01-06T22:24:28.462 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:24:28.462 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":102,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:24:28.338455+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6052,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6052":{"gid":6052,"name":"a","rank":0,"incarnation":101,"state":"up:resolve","state_seq":16,"addr":"172.21.15.203:6837/3262828783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3262828783},{"type":"v1","addr":"172.21.15.203:6837","nonce":3262828783}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:24:28.468 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2021-01-06T22:24:28.469 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2021-01-06T22:24:28.469 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:24:28.611 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5d4c2c700 1 Processor -- start 2021-01-06T22:24:28.611 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5d4c2c700 1 -- start start 2021-01-06T22:24:28.613 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5d4c2c700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff5d00bcbd0 0x7ff5d00baf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:28.613 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5d4c2c700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d00a4610 con 0x7ff5d00bc450 2021-01-06T22:24:28.613 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5d4c2c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d014ef80 con 0x7ff5d00bcfb0 2021-01-06T22:24:28.614 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5d4c2c700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d0064730 con 0x7ff5d00bcbd0 2021-01-06T22:24:28.614 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5cf7fe700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff5d00bcbd0 0x7ff5d00baf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:28.614 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5cffff700 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff5d00bc450 0x7ff5d014e870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.203:50762/0 (socket says 172.21.15.203:50762) 2021-01-06T22:24:28.614 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5cf7fe700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff5d00bcbd0 0x7ff5d00baf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.203:37128/0 (socket says 172.21.15.203:37128) 2021-01-06T22:24:28.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5cffff700 1 -- 172.21.15.203:0/2065075217 learned_addr learned my addr 172.21.15.203:0/2065075217 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:24:28.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5ceffd700 1 --1- 172.21.15.203:0/2065075217 >> v1:172.21.15.203:6789/0 conn(0x7ff5d00bcfb0 0x7ff5d014e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:56110/0 (socket says 172.21.15.203:56110) 2021-01-06T22:24:28.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5ce7fc700 1 -- 172.21.15.203:0/2065075217 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501344591 0 0) 0x7ff5d014ef80 con 0x7ff5d00bcfb0 2021-01-06T22:24:28.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5cf7fe700 1 -- 172.21.15.203:0/2065075217 >> v1:172.21.15.203:6789/0 conn(0x7ff5d00bcfb0 legacy=0x7ff5d014e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:28.615 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5cf7fe700 1 -- 172.21.15.203:0/2065075217 >> v1:172.21.15.91:6789/0 conn(0x7ff5d00bc450 legacy=0x7ff5d014e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:28.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5cf7fe700 1 -- 172.21.15.203:0/2065075217 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d00649d0 con 0x7ff5d00bcbd0 2021-01-06T22:24:28.616 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5cf7fe700 1 --2- 172.21.15.203:0/2065075217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5d00bcbd0 0x7ff5d00baf80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7ff5c400b980 tx=0x7ff5c4006de0).ready entity=mon.0 client_cookie=101ea43bd6e67f88 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:28.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5ce7fc700 1 -- 172.21.15.203:0/2065075217 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40096b0 con 0x7ff5d00bcbd0 2021-01-06T22:24:28.617 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5ce7fc700 1 -- 172.21.15.203:0/2065075217 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c4009810 con 0x7ff5d00bcbd0 2021-01-06T22:24:28.618 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5ce7fc700 1 -- 172.21.15.203:0/2065075217 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4009b90 con 0x7ff5d00bcbd0 2021-01-06T22:24:28.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5d4c2c700 1 -- 172.21.15.203:0/2065075217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5d00bcbd0 msgr2=0x7ff5d00baf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:28.620 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5d4c2c700 1 --2- 172.21.15.203:0/2065075217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5d00bcbd0 0x7ff5d00baf80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7ff5c400b980 tx=0x7ff5c4006de0).stop 2021-01-06T22:24:28.622 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5d4c2c700 1 -- 172.21.15.203:0/2065075217 shutdown_connections 2021-01-06T22:24:28.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5d4c2c700 1 --2- 172.21.15.203:0/2065075217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5d00bcbd0 0x7ff5d00baf80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:28.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5d4c2c700 1 -- 172.21.15.203:0/2065075217 >> 172.21.15.203:0/2065075217 conn(0x7ff5d0061530 msgr2=0x7ff5d0063c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:28.623 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5d4c2c700 1 -- 172.21.15.203:0/2065075217 shutdown_connections 2021-01-06T22:24:28.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.608+0000 7ff5d4c2c700 1 -- 172.21.15.203:0/2065075217 wait complete. 2021-01-06T22:24:28.624 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.612+0000 7ff5d4c2c700 1 Processor -- start 2021-01-06T22:24:28.632 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.612+0000 7ff5d4c2c700 1 -- start start 2021-01-06T22:24:28.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5d4c2c700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff5d00bc450 0x7ff5d00b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:28.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5d4c2c700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5d00bcbd0 0x7ff5d0153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:28.633 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5d4c2c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5d00bcfb0 0x7ff5d014b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:28.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5d4c2c700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d00ba4c0 con 0x7ff5d00bcbd0 2021-01-06T22:24:28.634 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5d4c2c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d00b75a0 con 0x7ff5d00bcfb0 2021-01-06T22:24:28.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5d4c2c700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d014c9a0 con 0x7ff5d00bc450 2021-01-06T22:24:28.635 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5cffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5d00bcfb0 0x7ff5d014b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:28.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5cffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5d00bcfb0 0x7ff5d014b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57520/0 (socket says 172.21.15.203:57520) 2021-01-06T22:24:28.636 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5cffff700 1 -- 172.21.15.203:0/401692228 learned_addr learned my addr 172.21.15.203:0/401692228 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:28.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5cf7fe700 1 --2- 172.21.15.203:0/401692228 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff5d00bc450 0x7ff5d00b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:28.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5ceffd700 1 --2- 172.21.15.203:0/401692228 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5d00bcbd0 0x7ff5d0153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:28.637 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5cf7fe700 1 -- 172.21.15.203:0/401692228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5d00bcfb0 msgr2=0x7ff5d014b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:28.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5cf7fe700 1 --2- 172.21.15.203:0/401692228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5d00bcfb0 0x7ff5d014b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:28.638 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5cf7fe700 1 -- 172.21.15.203:0/401692228 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5d00bcbd0 msgr2=0x7ff5d0153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:28.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5cf7fe700 1 --2- 172.21.15.203:0/401692228 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5d00bcbd0 0x7ff5d0153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:28.639 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5cf7fe700 1 -- 172.21.15.203:0/401692228 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d00ba220 con 0x7ff5d00bc450 2021-01-06T22:24:28.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5cffff700 1 --2- 172.21.15.203:0/401692228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5d00bcfb0 0x7ff5d014b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-01-06T22:24:28.640 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5ceffd700 1 --2- 172.21.15.203:0/401692228 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5d00bcbd0 0x7ff5d0153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-01-06T22:24:28.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5cf7fe700 1 --2- 172.21.15.203:0/401692228 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff5d00bc450 0x7ff5d00b8ea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7ff5d00619f0 tx=0x7ff5c4016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:28.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5b77fe700 1 -- 172.21.15.203:0/401692228 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4011d30 con 0x7ff5d00bc450 2021-01-06T22:24:28.641 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5b77fe700 1 -- 172.21.15.203:0/401692228 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c4011e90 con 0x7ff5d00bc450 2021-01-06T22:24:28.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5b77fe700 1 -- 172.21.15.203:0/401692228 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c401d660 con 0x7ff5d00bc450 2021-01-06T22:24:28.642 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5d4c2c700 1 -- 172.21.15.203:0/401692228 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d00b7800 con 0x7ff5d00bc450 2021-01-06T22:24:28.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.616+0000 7ff5d4c2c700 1 -- 172.21.15.203:0/401692228 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d00b9390 con 0x7ff5d00bc450 2021-01-06T22:24:28.643 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.620+0000 7ff5d4c2c700 1 -- 172.21.15.203:0/401692228 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d00450a0 con 0x7ff5d00bc450 2021-01-06T22:24:28.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.620+0000 7ff5b77fe700 1 -- 172.21.15.203:0/401692228 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7ff5c400f070 con 0x7ff5d00bc450 2021-01-06T22:24:28.644 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.620+0000 7ff5b77fe700 1 --2- 172.21.15.203:0/401692228 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff5b00563d0 0x7ff5b0058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:28.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.620+0000 7ff5b77fe700 1 -- 172.21.15.203:0/401692228 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7191+0+0 (secure 0 0 0) 0x7ff5c4071360 con 0x7ff5d00bc450 2021-01-06T22:24:28.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.620+0000 7ff5ceffd700 1 --2- 172.21.15.203:0/401692228 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff5b00563d0 0x7ff5b0058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:28.645 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.620+0000 7ff5ceffd700 1 --2- 172.21.15.203:0/401692228 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff5b00563d0 0x7ff5b0058890 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7ff5d00a6120 tx=0x7ff5c0007100).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:28.646 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.624+0000 7ff5b77fe700 1 -- 172.21.15.203:0/401692228 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7ff5c4046670 con 0x7ff5d00bc450 2021-01-06T22:24:28.863 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.856+0000 7ff5d4c2c700 1 -- 172.21.15.203:0/401692228 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5d00a2c40 con 0x7ff5d00bc450 2021-01-06T22:24:28.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.860+0000 7ff5b77fe700 1 -- 172.21.15.203:0/401692228 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+3215 (secure 0 0 0) 0x7ff5c4049c90 con 0x7ff5d00bc450 2021-01-06T22:24:28.869 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.860+0000 7ff5d4c2c700 1 -- 172.21.15.203:0/401692228 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff5b00563d0 msgr2=0x7ff5b0058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:28.870 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.860+0000 7ff5d4c2c700 1 --2- 172.21.15.203:0/401692228 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff5b00563d0 0x7ff5b0058890 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7ff5d00a6120 tx=0x7ff5c0007100).stop 2021-01-06T22:24:28.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.860+0000 7ff5d4c2c700 1 -- 172.21.15.203:0/401692228 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff5d00bc450 msgr2=0x7ff5d00b8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:28.871 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.860+0000 7ff5d4c2c700 1 --2- 172.21.15.203:0/401692228 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff5d00bc450 0x7ff5d00b8ea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7ff5d00619f0 tx=0x7ff5c4016000).stop 2021-01-06T22:24:28.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.860+0000 7ff5d4c2c700 1 -- 172.21.15.203:0/401692228 shutdown_connections 2021-01-06T22:24:28.872 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.860+0000 7ff5d4c2c700 1 --2- 172.21.15.203:0/401692228 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7ff5b00563d0 0x7ff5b0058890 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:28.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.860+0000 7ff5d4c2c700 1 --2- 172.21.15.203:0/401692228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5d00bcfb0 0x7ff5d014b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:28.873 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.860+0000 7ff5d4c2c700 1 --2- 172.21.15.203:0/401692228 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff5d00bc450 0x7ff5d00b8ea0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:28.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.860+0000 7ff5d4c2c700 1 --2- 172.21.15.203:0/401692228 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5d00bcbd0 0x7ff5d0153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:28.874 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.860+0000 7ff5d4c2c700 1 -- 172.21.15.203:0/401692228 >> 172.21.15.203:0/401692228 conn(0x7ff5d0061530 msgr2=0x7ff5d00bb350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:28.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.860+0000 7ff5d4c2c700 1 -- 172.21.15.203:0/401692228 shutdown_connections 2021-01-06T22:24:28.875 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:28.860+0000 7ff5d4c2c700 1 -- 172.21.15.203:0/401692228 wait complete. 2021-01-06T22:24:28.876 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 102 2021-01-06T22:24:28.876 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:24:28.877 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":102,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:24:28.338455+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6052,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6052":{"gid":6052,"name":"a","rank":0,"incarnation":101,"state":"up:resolve","state_seq":16,"addr":"172.21.15.203:6837/3262828783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3262828783},{"type":"v1","addr":"172.21.15.203:6837","nonce":3262828783}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:24:28.882 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 102, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2021-01-06T22:03:39.741327+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 102, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5874": { "addr": "172.21.15.91:6836/1921314962", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6834", "nonce": 1921314962, "type": "v2" }, { "addr": "172.21.15.91:6836", "nonce": 1921314962, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5874, "incarnation": 95, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 29 }, "gid_6052": { "addr": "172.21.15.203:6837/3262828783", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6836", "nonce": 3262828783, "type": "v2" }, { "addr": "172.21.15.203:6837", "nonce": 3262828783, "type": "v1" } ] }, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 6052, "incarnation": 101, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:resolve", "state_seq": 16 } }, "last_failure": 0, "last_failure_osd_epoch": 36, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2021-01-06T22:24:28.338455+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6052, "mds_1": 5874 } } } ], "standbys": [ { "addr": "172.21.15.91:6833/3811290068", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6832", "nonce": 3811290068, "type": "v2" }, { "addr": "172.21.15.91:6833", "nonce": 3811290068, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4248, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6839/453324421", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6838", "nonce": 453324421, "type": "v2" }, { "addr": "172.21.15.203:6839", "nonce": 453324421, "type": "v1" } ] }, "epoch": 3, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 4251, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.91:6837/2972061575", "addrs": { "addrvec": [ { "addr": "172.21.15.91:6835", "nonce": 2972061575, "type": "v2" }, { "addr": "172.21.15.91:6837", "nonce": 2972061575, "type": "v1" } ] }, "epoch": 87, "export_targets": [], "features": 4540138297136906239, "flags": 0, "gid": 5384, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 2 } ] } 2021-01-06T22:24:30.883 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:24:31.011 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f518aac7700 1 Processor -- start 2021-01-06T22:24:31.012 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f518aac7700 1 -- start start 2021-01-06T22:24:31.013 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f518aac7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5184068440 0x7f518414b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:31.014 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f518aac7700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f51840570f0 0x7f518414b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:31.015 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f518aac7700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5184067530 con 0x7f51840bb790 2021-01-06T22:24:31.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f518aac7700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51841524e0 con 0x7f51840570f0 2021-01-06T22:24:31.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f518aac7700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5184067760 con 0x7f5184068440 2021-01-06T22:24:31.016 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f5183fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5184068440 0x7f518414b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:31.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f5183fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5184068440 0x7f518414b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57592/0 (socket says 172.21.15.203:57592) 2021-01-06T22:24:31.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f5183fff700 1 -- 172.21.15.203:0/1481067434 learned_addr learned my addr 172.21.15.203:0/1481067434 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:31.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f5189064700 1 --2- 172.21.15.203:0/1481067434 >> v2:172.21.15.91:3300/0 conn(0x7f51840570f0 0x7f518414b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:31.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f5183fff700 1 -- 172.21.15.203:0/1481067434 >> v2:172.21.15.91:3300/0 conn(0x7f51840570f0 msgr2=0x7f518414b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:24:31.017 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f5183fff700 1 --2- 172.21.15.203:0/1481067434 >> v2:172.21.15.91:3300/0 conn(0x7f51840570f0 0x7f518414b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:31.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f5183fff700 1 -- 172.21.15.203:0/1481067434 >> v1:172.21.15.91:6789/0 conn(0x7f51840bb790 legacy=0x7f51840bbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:31.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f5183fff700 1 -- 172.21.15.203:0/1481067434 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5184067a00 con 0x7f5184068440 2021-01-06T22:24:31.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f5183fff700 1 --2- 172.21.15.203:0/1481067434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5184068440 0x7f518414b300 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f51840c1800 tx=0x7f5170006ea0).ready entity=mon.1 client_cookie=7c2c9859b6dbdfa2 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:31.018 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f51837fe700 1 -- 172.21.15.203:0/1481067434 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51700098a0 con 0x7f5184068440 2021-01-06T22:24:31.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f51837fe700 1 -- 172.21.15.203:0/1481067434 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5170009a00 con 0x7f5184068440 2021-01-06T22:24:31.019 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f51837fe700 1 -- 172.21.15.203:0/1481067434 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170009d80 con 0x7f5184068440 2021-01-06T22:24:31.022 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f518aac7700 1 -- 172.21.15.203:0/1481067434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5184068440 msgr2=0x7f518414b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:31.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f518aac7700 1 --2- 172.21.15.203:0/1481067434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5184068440 0x7f518414b300 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f51840c1800 tx=0x7f5170006ea0).stop 2021-01-06T22:24:31.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f518aac7700 1 -- 172.21.15.203:0/1481067434 shutdown_connections 2021-01-06T22:24:31.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f518aac7700 1 --2- 172.21.15.203:0/1481067434 >> v2:172.21.15.91:3300/0 conn(0x7f51840570f0 0x7f518414b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:31.023 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f518aac7700 1 --2- 172.21.15.203:0/1481067434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5184068440 0x7f518414b300 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:31.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f518aac7700 1 -- 172.21.15.203:0/1481067434 >> 172.21.15.203:0/1481067434 conn(0x7f51840c0c50 msgr2=0x7f51840c1030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:31.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f518aac7700 1 -- 172.21.15.203:0/1481067434 shutdown_connections 2021-01-06T22:24:31.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.012+0000 7f518aac7700 1 -- 172.21.15.203:0/1481067434 wait complete. 2021-01-06T22:24:31.024 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f518aac7700 1 Processor -- start 2021-01-06T22:24:31.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f518aac7700 1 -- start start 2021-01-06T22:24:31.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f518aac7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51840570f0 0x7f51840bff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:31.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f518aac7700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5184068440 0x7f518414f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:31.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f518aac7700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f51840bb790 0x7f5184062880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:31.025 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f518aac7700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51840c0400 con 0x7f51840bb790 2021-01-06T22:24:31.026 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f518aac7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518415d950 con 0x7f51840570f0 2021-01-06T22:24:31.026 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f518aac7700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518415dbf0 con 0x7f5184068440 2021-01-06T22:24:31.026 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f5188863700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51840570f0 0x7f51840bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:31.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f5188863700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51840570f0 0x7f51840bff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57596/0 (socket says 172.21.15.203:57596) 2021-01-06T22:24:31.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f5188863700 1 -- 172.21.15.203:0/2322861343 learned_addr learned my addr 172.21.15.203:0/2322861343 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:31.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f5189064700 1 --2- 172.21.15.203:0/2322861343 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f51840bb790 0x7f5184062880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:31.027 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f5183fff700 1 --2- 172.21.15.203:0/2322861343 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5184068440 0x7f518414f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:31.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f5188863700 1 -- 172.21.15.203:0/2322861343 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5184068440 msgr2=0x7f518414f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:31.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f5188863700 1 --2- 172.21.15.203:0/2322861343 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5184068440 0x7f518414f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:31.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f5188863700 1 -- 172.21.15.203:0/2322861343 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f51840bb790 msgr2=0x7f5184062880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:31.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f5188863700 1 --2- 172.21.15.203:0/2322861343 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f51840bb790 0x7f5184062880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:31.028 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f5188863700 1 -- 172.21.15.203:0/2322861343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518415de90 con 0x7f51840570f0 2021-01-06T22:24:31.029 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f5188863700 1 --2- 172.21.15.203:0/2322861343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51840570f0 0x7f51840bff10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f517800c710 tx=0x7f5178007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:31.029 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f5180ff9700 1 -- 172.21.15.203:0/2322861343 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178007020 con 0x7f51840570f0 2021-01-06T22:24:31.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f5180ff9700 1 -- 172.21.15.203:0/2322861343 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5178007180 con 0x7f51840570f0 2021-01-06T22:24:31.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f518aac7700 1 -- 172.21.15.203:0/2322861343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518415e130 con 0x7f51840570f0 2021-01-06T22:24:31.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f5180ff9700 1 -- 172.21.15.203:0/2322861343 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517800e560 con 0x7f51840570f0 2021-01-06T22:24:31.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.016+0000 7f518aac7700 1 -- 172.21.15.203:0/2322861343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518415e670 con 0x7f51840570f0 2021-01-06T22:24:31.030 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.020+0000 7f518aac7700 1 -- 172.21.15.203:0/2322861343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518415e830 con 0x7f51840570f0 2021-01-06T22:24:31.031 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.020+0000 7f5180ff9700 1 -- 172.21.15.203:0/2322861343 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f517801a060 con 0x7f51840570f0 2021-01-06T22:24:31.031 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.020+0000 7f5180ff9700 1 --2- 172.21.15.203:0/2322861343 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f51640563d0 0x7f5164058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:31.031 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.020+0000 7f5180ff9700 1 -- 172.21.15.203:0/2322861343 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7191+0+0 (secure 0 0 0) 0x7f517806ef20 con 0x7f51840570f0 2021-01-06T22:24:31.031 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.020+0000 7f5183fff700 1 --2- 172.21.15.203:0/2322861343 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f51640563d0 0x7f5164058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:31.032 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.020+0000 7f5183fff700 1 --2- 172.21.15.203:0/2322861343 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f51640563d0 0x7f5164058890 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f51840b7f50 tx=0x7f5170016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:31.032 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.028+0000 7f5180ff9700 1 -- 172.21.15.203:0/2322861343 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f5178044340 con 0x7f51840570f0 2021-01-06T22:24:31.232 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.232+0000 7f518aac7700 1 -- 172.21.15.203:0/2322861343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f518405d600 con 0x7f51840570f0 2021-01-06T22:24:31.232 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:24:31.233 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":104,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:24:30.361362+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6052,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6052":{"gid":6052,"name":"a","rank":0,"incarnation":101,"state":"up:rejoin","state_seq":18,"addr":"172.21.15.203:6837/3262828783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3262828783},{"type":"v1","addr":"172.21.15.203:6837","nonce":3262828783}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:24:31.345 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.232+0000 7f5180ff9700 1 -- 172.21.15.203:0/2322861343 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+3214 (secure 0 0 0) 0x7f5178018070 con 0x7f51840570f0 2021-01-06T22:24:31.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.232+0000 7f518aac7700 1 -- 172.21.15.203:0/2322861343 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f51640563d0 msgr2=0x7f5164058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:31.346 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.232+0000 7f518aac7700 1 --2- 172.21.15.203:0/2322861343 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f51640563d0 0x7f5164058890 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f51840b7f50 tx=0x7f5170016000).stop 2021-01-06T22:24:31.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.232+0000 7f518aac7700 1 -- 172.21.15.203:0/2322861343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51840570f0 msgr2=0x7f51840bff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:31.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.232+0000 7f518aac7700 1 --2- 172.21.15.203:0/2322861343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51840570f0 0x7f51840bff10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f517800c710 tx=0x7f5178007c40).stop 2021-01-06T22:24:31.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.232+0000 7f518aac7700 1 -- 172.21.15.203:0/2322861343 shutdown_connections 2021-01-06T22:24:31.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.232+0000 7f518aac7700 1 --2- 172.21.15.203:0/2322861343 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f51640563d0 0x7f5164058890 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:31.353 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.232+0000 7f518aac7700 1 --2- 172.21.15.203:0/2322861343 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f51840bb790 0x7f5184062880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:31.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.232+0000 7f518aac7700 1 --2- 172.21.15.203:0/2322861343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51840570f0 0x7f51840bff10 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:31.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.232+0000 7f518aac7700 1 --2- 172.21.15.203:0/2322861343 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5184068440 0x7f518414f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:31.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.232+0000 7f518aac7700 1 -- 172.21.15.203:0/2322861343 >> 172.21.15.203:0/2322861343 conn(0x7f51840c0c50 msgr2=0x7f5184142930 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:31.354 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.232+0000 7f518aac7700 1 -- 172.21.15.203:0/2322861343 shutdown_connections 2021-01-06T22:24:31.355 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:31.232+0000 7f518aac7700 1 -- 172.21.15.203:0/2322861343 wait complete. 2021-01-06T22:24:31.355 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 104 2021-01-06T22:24:31.363 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2021-01-06T22:24:33.364 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:24:33.517 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 Processor -- start 2021-01-06T22:24:33.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 -- start start 2021-01-06T22:24:33.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd8480c44f0 0x7fd8480c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:33.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd8480c48d0 0x7fd8480c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:33.518 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd848060fc0 0x7fd848150730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:33.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8480c1d60 con 0x7fd8480c44f0 2021-01-06T22:24:33.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8480c0f50 con 0x7fd8480c48d0 2021-01-06T22:24:33.519 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd848066cf0 con 0x7fd848060fc0 2021-01-06T22:24:33.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd847fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd8480c48d0 0x7fd8480c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:33.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd847fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd8480c48d0 0x7fd8480c1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57678/0 (socket says 172.21.15.203:57678) 2021-01-06T22:24:33.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd847fff700 1 -- 172.21.15.203:0/1361874065 learned_addr learned my addr 172.21.15.203:0/1361874065 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:33.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84d67c700 1 --2- 172.21.15.203:0/1361874065 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd848060fc0 0x7fd848150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:33.520 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84ce7b700 1 --2- 172.21.15.203:0/1361874065 >> v2:172.21.15.91:3300/0 conn(0x7fd8480c44f0 0x7fd8480c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:33.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd847fff700 1 -- 172.21.15.203:0/1361874065 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd848060fc0 msgr2=0x7fd848150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:24:33.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd847fff700 1 --2- 172.21.15.203:0/1361874065 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd848060fc0 0x7fd848150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:33.521 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd847fff700 1 -- 172.21.15.203:0/1361874065 >> v2:172.21.15.91:3300/0 conn(0x7fd8480c44f0 msgr2=0x7fd8480c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:24:33.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd847fff700 1 --2- 172.21.15.203:0/1361874065 >> v2:172.21.15.91:3300/0 conn(0x7fd8480c44f0 0x7fd8480c1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:33.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd847fff700 1 -- 172.21.15.203:0/1361874065 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd848060750 con 0x7fd8480c48d0 2021-01-06T22:24:33.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd847fff700 1 --2- 172.21.15.203:0/1361874065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8480c48d0 0x7fd8480c1870 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7fd848061480 tx=0x7fd834006ea0).ready entity=mon.1 client_cookie=283644df64eef3a7 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:33.523 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd8477fe700 1 -- 172.21.15.203:0/1361874065 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8340096e0 con 0x7fd8480c48d0 2021-01-06T22:24:33.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd8477fe700 1 -- 172.21.15.203:0/1361874065 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd834009840 con 0x7fd8480c48d0 2021-01-06T22:24:33.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd8477fe700 1 -- 172.21.15.203:0/1361874065 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834009bc0 con 0x7fd8480c48d0 2021-01-06T22:24:33.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 -- 172.21.15.203:0/1361874065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8480c48d0 msgr2=0x7fd8480c1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:33.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 --2- 172.21.15.203:0/1361874065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8480c48d0 0x7fd8480c1870 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7fd848061480 tx=0x7fd834006ea0).stop 2021-01-06T22:24:33.524 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 -- 172.21.15.203:0/1361874065 shutdown_connections 2021-01-06T22:24:33.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 --2- 172.21.15.203:0/1361874065 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd848060fc0 0x7fd848150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:33.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 --2- 172.21.15.203:0/1361874065 >> v2:172.21.15.91:3300/0 conn(0x7fd8480c44f0 0x7fd8480c1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:33.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 --2- 172.21.15.203:0/1361874065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8480c48d0 0x7fd8480c1870 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:33.525 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 -- 172.21.15.203:0/1361874065 >> 172.21.15.203:0/1361874065 conn(0x7fd848056cd0 msgr2=0x7fd848061530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:33.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 -- 172.21.15.203:0/1361874065 shutdown_connections 2021-01-06T22:24:33.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 -- 172.21.15.203:0/1361874065 wait complete. 2021-01-06T22:24:33.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 Processor -- start 2021-01-06T22:24:33.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.499+0000 7fd84de7d700 1 -- start start 2021-01-06T22:24:33.526 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd84de7d700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd848060fc0 0x7fd8480ba730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:33.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd84de7d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8480c44f0 0x7fd848161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:33.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd84de7d700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd8480c48d0 0x7fd8481615f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:33.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd84de7d700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd848140270 con 0x7fd848060fc0 2021-01-06T22:24:33.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd84de7d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84814e100 con 0x7fd8480c44f0 2021-01-06T22:24:33.527 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd84de7d700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd848161b90 con 0x7fd8480c48d0 2021-01-06T22:24:33.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd847fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8480c44f0 0x7fd848161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:33.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd847fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8480c44f0 0x7fd848161100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57682/0 (socket says 172.21.15.203:57682) 2021-01-06T22:24:33.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd847fff700 1 -- 172.21.15.203:0/2679622594 learned_addr learned my addr 172.21.15.203:0/2679622594 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:33.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd84d67c700 1 --2- 172.21.15.203:0/2679622594 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd8480c48d0 0x7fd8481615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:33.528 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd84ce7b700 1 --2- 172.21.15.203:0/2679622594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd848060fc0 0x7fd8480ba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:33.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd847fff700 1 -- 172.21.15.203:0/2679622594 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd8480c48d0 msgr2=0x7fd8481615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:33.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd847fff700 1 --2- 172.21.15.203:0/2679622594 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd8480c48d0 0x7fd8481615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:33.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd847fff700 1 -- 172.21.15.203:0/2679622594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd848060fc0 msgr2=0x7fd8480ba730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:33.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd847fff700 1 --2- 172.21.15.203:0/2679622594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd848060fc0 0x7fd8480ba730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:33.529 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd847fff700 1 -- 172.21.15.203:0/2679622594 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8480b8c30 con 0x7fd8480c44f0 2021-01-06T22:24:33.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd847fff700 1 --2- 172.21.15.203:0/2679622594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8480c44f0 0x7fd848161100 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7fd834006ee0 tx=0x7fd834016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:33.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd844ff9700 1 -- 172.21.15.203:0/2679622594 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834011b40 con 0x7fd8480c44f0 2021-01-06T22:24:33.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd844ff9700 1 -- 172.21.15.203:0/2679622594 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd834011ca0 con 0x7fd8480c44f0 2021-01-06T22:24:33.530 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd84de7d700 1 -- 172.21.15.203:0/2679622594 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8480b8ed0 con 0x7fd8480c44f0 2021-01-06T22:24:33.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd844ff9700 1 -- 172.21.15.203:0/2679622594 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83401c650 con 0x7fd8480c44f0 2021-01-06T22:24:33.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd84de7d700 1 -- 172.21.15.203:0/2679622594 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8480b9170 con 0x7fd8480c44f0 2021-01-06T22:24:33.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd844ff9700 1 -- 172.21.15.203:0/2679622594 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7fd83400f070 con 0x7fd8480c44f0 2021-01-06T22:24:33.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd844ff9700 1 --2- 172.21.15.203:0/2679622594 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd8280563d0 0x7fd828058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:33.531 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd844ff9700 1 -- 172.21.15.203:0/2679622594 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7191+0+0 (secure 0 0 0) 0x7fd834070e90 con 0x7fd8480c44f0 2021-01-06T22:24:33.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.503+0000 7fd84ce7b700 1 --2- 172.21.15.203:0/2679622594 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd8280563d0 0x7fd828058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:33.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.507+0000 7fd84ce7b700 1 --2- 172.21.15.203:0/2679622594 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd8280563d0 0x7fd828058890 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fd83c00bb50 tx=0x7fd83c006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:33.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.511+0000 7fd84de7d700 1 -- 172.21.15.203:0/2679622594 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd848060930 con 0x7fd8480c44f0 2021-01-06T22:24:33.532 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.515+0000 7fd844ff9700 1 -- 172.21.15.203:0/2679622594 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7fd8340462b0 con 0x7fd8480c44f0 2021-01-06T22:24:33.736 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.735+0000 7fd84de7d700 1 -- 172.21.15.203:0/2679622594 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8480b8020 con 0x7fd8480c44f0 2021-01-06T22:24:33.737 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.735+0000 7fd844ff9700 1 -- 172.21.15.203:0/2679622594 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+3214 (secure 0 0 0) 0x7fd8340498d0 con 0x7fd8480c44f0 2021-01-06T22:24:33.737 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:24:33.738 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":105,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:24:31.368444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6052,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6052":{"gid":6052,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":19,"addr":"172.21.15.203:6837/3262828783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3262828783},{"type":"v1","addr":"172.21.15.203:6837","nonce":3262828783}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:24:33.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.735+0000 7fd84de7d700 1 -- 172.21.15.203:0/2679622594 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd8280563d0 msgr2=0x7fd828058890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:33.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.735+0000 7fd84de7d700 1 --2- 172.21.15.203:0/2679622594 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd8280563d0 0x7fd828058890 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fd83c00bb50 tx=0x7fd83c006de0).stop 2021-01-06T22:24:33.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.735+0000 7fd84de7d700 1 -- 172.21.15.203:0/2679622594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8480c44f0 msgr2=0x7fd848161100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:33.739 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.735+0000 7fd84de7d700 1 --2- 172.21.15.203:0/2679622594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8480c44f0 0x7fd848161100 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7fd834006ee0 tx=0x7fd834016000).stop 2021-01-06T22:24:33.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.739+0000 7fd84de7d700 1 -- 172.21.15.203:0/2679622594 shutdown_connections 2021-01-06T22:24:33.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.739+0000 7fd84de7d700 1 --2- 172.21.15.203:0/2679622594 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7fd8280563d0 0x7fd828058890 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:33.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.739+0000 7fd84de7d700 1 --2- 172.21.15.203:0/2679622594 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd8480c48d0 0x7fd8481615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:33.740 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.739+0000 7fd84de7d700 1 --2- 172.21.15.203:0/2679622594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd848060fc0 0x7fd8480ba730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:33.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.739+0000 7fd84de7d700 1 --2- 172.21.15.203:0/2679622594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8480c44f0 0x7fd848161100 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:33.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.739+0000 7fd84de7d700 1 -- 172.21.15.203:0/2679622594 >> 172.21.15.203:0/2679622594 conn(0x7fd848056cd0 msgr2=0x7fd8480c0470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:33.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.739+0000 7fd84de7d700 1 -- 172.21.15.203:0/2679622594 shutdown_connections 2021-01-06T22:24:33.741 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:33.739+0000 7fd84de7d700 1 -- 172.21.15.203:0/2679622594 wait complete. 2021-01-06T22:24:33.742 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 105 2021-01-06T22:24:33.751 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=5836 2021-01-06T22:24:33.752 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 3 secs before reviving mds.c 2021-01-06T22:24:33.752 INFO:tasks.ceph.mds.a.smithi203.stderr:2021-01-06T22:24:33.751+0000 7fc456fa5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2021-01-06T22:24:33.753 INFO:tasks.ceph.mds.a.smithi203.stderr:2021-01-06T22:24:33.751+0000 7fc456fa5700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2021-01-06T22:24:36.753 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2021-01-06T22:24:36.753 INFO:tasks.ceph.mds.c:Restarting daemon 2021-01-06T22:24:36.754 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-01-06T22:24:36.756 INFO:tasks.ceph.mds.c:Started 2021-01-06T22:24:36.757 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:24:36.836 INFO:tasks.ceph.mds.c.smithi203.stdout:starting mds.c at 2021-01-06T22:24:36.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.883+0000 7f068327b700 1 Processor -- start 2021-01-06T22:24:36.885 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.883+0000 7f068327b700 1 -- start start 2021-01-06T22:24:36.886 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.883+0000 7f068327b700 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f067c067720 0x7f067c068110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:36.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.883+0000 7f068327b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f067c148080 0x7f067c0602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:36.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.883+0000 7f068327b700 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f067c0bba10 con 0x7f067c067340 2021-01-06T22:24:36.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.883+0000 7f068327b700 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067c0bb230 con 0x7f067c067720 2021-01-06T22:24:36.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.883+0000 7f068327b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067c0bb4d0 con 0x7f067c148080 2021-01-06T22:24:36.887 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.883+0000 7f0682a7a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f067c148080 0x7f067c0602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:36.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.883+0000 7f0682a7a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f067c148080 0x7f067c0602b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57800/0 (socket says 172.21.15.203:57800) 2021-01-06T22:24:36.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.883+0000 7f0682a7a700 1 -- 172.21.15.203:0/1996551106 learned_addr learned my addr 172.21.15.203:0/1996551106 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:36.888 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.883+0000 7f0681a78700 1 --2- 172.21.15.203:0/1996551106 >> v2:172.21.15.91:3300/0 conn(0x7f067c067720 0x7f067c068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:36.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f0682a7a700 1 -- 172.21.15.203:0/1996551106 >> v2:172.21.15.91:3300/0 conn(0x7f067c067720 msgr2=0x7f067c068110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:24:36.890 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f0682a7a700 1 --2- 172.21.15.203:0/1996551106 >> v2:172.21.15.91:3300/0 conn(0x7f067c067720 0x7f067c068110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:36.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f0682a7a700 1 -- 172.21.15.203:0/1996551106 >> v1:172.21.15.91:6789/0 conn(0x7f067c067340 legacy=0x7f067c147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:36.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f0682a7a700 1 -- 172.21.15.203:0/1996551106 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c0bb770 con 0x7f067c148080 2021-01-06T22:24:36.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f0682a7a700 1 --2- 172.21.15.203:0/1996551106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c148080 0x7f067c0602b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f067c066630 tx=0x7f0674009290).ready entity=mon.1 client_cookie=8f0d08ad053725b server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:36.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f0681277700 1 -- 172.21.15.203:0/1996551106 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067400bc50 con 0x7f067c148080 2021-01-06T22:24:36.891 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f0681277700 1 -- 172.21.15.203:0/1996551106 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067400bdb0 con 0x7f067c148080 2021-01-06T22:24:36.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f0681277700 1 -- 172.21.15.203:0/1996551106 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674011610 con 0x7f067c148080 2021-01-06T22:24:36.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f068327b700 1 -- 172.21.15.203:0/1996551106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c148080 msgr2=0x7f067c0602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:36.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f068327b700 1 --2- 172.21.15.203:0/1996551106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c148080 0x7f067c0602b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f067c066630 tx=0x7f0674009290).stop 2021-01-06T22:24:36.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f068327b700 1 -- 172.21.15.203:0/1996551106 shutdown_connections 2021-01-06T22:24:36.892 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f068327b700 1 --2- 172.21.15.203:0/1996551106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c148080 0x7f067c0602b0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:36.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f068327b700 1 --2- 172.21.15.203:0/1996551106 >> v2:172.21.15.91:3300/0 conn(0x7f067c067720 0x7f067c068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:36.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f068327b700 1 -- 172.21.15.203:0/1996551106 >> 172.21.15.203:0/1996551106 conn(0x7f067c0667b0 msgr2=0x7f067c0c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:36.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f068327b700 1 -- 172.21.15.203:0/1996551106 shutdown_connections 2021-01-06T22:24:36.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f068327b700 1 -- 172.21.15.203:0/1996551106 wait complete. 2021-01-06T22:24:36.893 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f068327b700 1 Processor -- start 2021-01-06T22:24:36.894 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.887+0000 7f068327b700 1 -- start start 2021-01-06T22:24:36.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f068327b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c067340 0x7f067c0ba410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:36.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f068327b700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f067c067720 0x7f067c146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:36.897 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f068327b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f067c148080 0x7f067c146660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:36.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f068327b700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c146b50 con 0x7f067c067720 2021-01-06T22:24:36.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f068327b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c14bf10 con 0x7f067c067340 2021-01-06T22:24:36.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f068327b700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067c0c03b0 con 0x7f067c148080 2021-01-06T22:24:36.898 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f0682279700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c067340 0x7f067c0ba410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:36.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f0682279700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c067340 0x7f067c0ba410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57802/0 (socket says 172.21.15.203:57802) 2021-01-06T22:24:36.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f0682279700 1 -- 172.21.15.203:0/224030808 learned_addr learned my addr 172.21.15.203:0/224030808 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:36.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f0681a78700 1 --2- 172.21.15.203:0/224030808 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f067c067720 0x7f067c146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:36.899 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f0682a7a700 1 --2- 172.21.15.203:0/224030808 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f067c148080 0x7f067c146660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:36.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f0682279700 1 -- 172.21.15.203:0/224030808 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f067c148080 msgr2=0x7f067c146660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:36.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f0682279700 1 --2- 172.21.15.203:0/224030808 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f067c148080 0x7f067c146660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:36.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f0682279700 1 -- 172.21.15.203:0/224030808 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f067c067720 msgr2=0x7f067c146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:36.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f0682279700 1 --2- 172.21.15.203:0/224030808 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f067c067720 0x7f067c146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:36.900 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f0682279700 1 -- 172.21.15.203:0/224030808 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c0b90b0 con 0x7f067c067340 2021-01-06T22:24:36.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f0682279700 1 --2- 172.21.15.203:0/224030808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c067340 0x7f067c0ba410 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f067800c740 tx=0x7f0678007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:36.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f066a7fc700 1 -- 172.21.15.203:0/224030808 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678007020 con 0x7f067c067340 2021-01-06T22:24:36.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f066a7fc700 1 -- 172.21.15.203:0/224030808 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0678007180 con 0x7f067c067340 2021-01-06T22:24:36.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f068327b700 1 -- 172.21.15.203:0/224030808 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067c0b9350 con 0x7f067c067340 2021-01-06T22:24:36.901 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f066a7fc700 1 -- 172.21.15.203:0/224030808 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06780064e0 con 0x7f067c067340 2021-01-06T22:24:36.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f068327b700 1 -- 172.21.15.203:0/224030808 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067c13dbb0 con 0x7f067c067340 2021-01-06T22:24:36.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f068327b700 1 -- 172.21.15.203:0/224030808 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067c0b94e0 con 0x7f067c067340 2021-01-06T22:24:36.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f066a7fc700 1 -- 172.21.15.203:0/224030808 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f067801a060 con 0x7f067c067340 2021-01-06T22:24:36.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f066a7fc700 1 --2- 172.21.15.203:0/224030808 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f065c056400 0x7f065c0588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:36.902 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f066a7fc700 1 -- 172.21.15.203:0/224030808 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7191+0+0 (secure 0 0 0) 0x7f067806f060 con 0x7f067c067340 2021-01-06T22:24:36.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.891+0000 7f0681a78700 1 --2- 172.21.15.203:0/224030808 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f065c056400 0x7f065c0588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:36.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.895+0000 7f0681a78700 1 --2- 172.21.15.203:0/224030808 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f065c056400 0x7f065c0588c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f067c149d30 tx=0x7f066c006de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:36.903 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:36.899+0000 7f066a7fc700 1 -- 172.21.15.203:0/224030808 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f0678044480 con 0x7f067c067340 2021-01-06T22:24:37.110 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:37.107+0000 7f068327b700 1 -- 172.21.15.203:0/224030808 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f067c0c0570 con 0x7f067c067340 2021-01-06T22:24:37.111 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:37.107+0000 7f066a7fc700 1 -- 172.21.15.203:0/224030808 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+3214 (secure 0 0 0) 0x7f0678018030 con 0x7f067c067340 2021-01-06T22:24:37.111 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:24:37.111 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":105,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:24:31.368444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6052,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6052":{"gid":6052,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":19,"addr":"172.21.15.203:6837/3262828783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3262828783},{"type":"v1","addr":"172.21.15.203:6837","nonce":3262828783}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:24:37.112 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:37.111+0000 7f068327b700 1 -- 172.21.15.203:0/224030808 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f065c056400 msgr2=0x7f065c0588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:37.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:37.111+0000 7f068327b700 1 --2- 172.21.15.203:0/224030808 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f065c056400 0x7f065c0588c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f067c149d30 tx=0x7f066c006de0).stop 2021-01-06T22:24:37.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:37.111+0000 7f068327b700 1 -- 172.21.15.203:0/224030808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c067340 msgr2=0x7f067c0ba410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:37.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:37.111+0000 7f068327b700 1 --2- 172.21.15.203:0/224030808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c067340 0x7f067c0ba410 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f067800c740 tx=0x7f0678007c40).stop 2021-01-06T22:24:37.113 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:37.111+0000 7f068327b700 1 -- 172.21.15.203:0/224030808 shutdown_connections 2021-01-06T22:24:37.114 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:37.111+0000 7f068327b700 1 --2- 172.21.15.203:0/224030808 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f065c056400 0x7f065c0588c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:37.114 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:37.111+0000 7f068327b700 1 --2- 172.21.15.203:0/224030808 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f067c148080 0x7f067c146660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:37.114 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:37.111+0000 7f068327b700 1 --2- 172.21.15.203:0/224030808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c067340 0x7f067c0ba410 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:37.114 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:37.111+0000 7f068327b700 1 --2- 172.21.15.203:0/224030808 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f067c067720 0x7f067c146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:37.115 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:37.111+0000 7f068327b700 1 -- 172.21.15.203:0/224030808 >> 172.21.15.203:0/224030808 conn(0x7f067c0667b0 msgr2=0x7f067c0bbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:37.115 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:37.111+0000 7f068327b700 1 -- 172.21.15.203:0/224030808 shutdown_connections 2021-01-06T22:24:37.115 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:37.111+0000 7f068327b700 1 -- 172.21.15.203:0/224030808 wait complete. 2021-01-06T22:24:37.116 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 105 2021-01-06T22:24:37.125 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2021-01-06T22:24:39.126 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-01-06T22:24:39.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3d63b700 1 Processor -- start 2021-01-06T22:24:39.239 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3d63b700 1 -- start start 2021-01-06T22:24:39.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3d63b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2a38067340 0x7f2a38068110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:39.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3d63b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a38067720 0x7f2a380602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:39.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3d63b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a380bba10 con 0x7f2a3814b4f0 2021-01-06T22:24:39.243 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3d63b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a380bb230 con 0x7f2a38067340 2021-01-06T22:24:39.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3d63b700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a380bb4d0 con 0x7f2a38067720 2021-01-06T22:24:39.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a37fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2a38067340 0x7f2a38068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:39.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3ce3a700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f2a3814b4f0 0x7f2a38147970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:56472/0 (socket says 172.21.15.203:56472) 2021-01-06T22:24:39.244 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3ce3a700 1 -- 172.21.15.203:0/3912140354 learned_addr learned my addr 172.21.15.203:0/3912140354 (peer_addr_for_me v1:172.21.15.203:0/0) 2021-01-06T22:24:39.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a377fe700 1 --2- 172.21.15.203:0/3912140354 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a38067720 0x7f2a380602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:39.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a36ffd700 1 -- 172.21.15.203:0/3912140354 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709052225 0 0) 0x7f2a380bba10 con 0x7f2a3814b4f0 2021-01-06T22:24:39.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a36ffd700 1 -- 172.21.15.203:0/3912140354 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a24003230 con 0x7f2a3814b4f0 2021-01-06T22:24:39.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a37fff700 1 -- 172.21.15.203:0/3912140354 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a38067720 msgr2=0x7f2a380602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-01-06T22:24:39.245 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a37fff700 1 --2- 172.21.15.203:0/3912140354 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a38067720 0x7f2a380602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:39.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a37fff700 1 -- 172.21.15.203:0/3912140354 >> v1:172.21.15.203:6789/0 conn(0x7f2a3814b4f0 legacy=0x7f2a38147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:39.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a37fff700 1 -- 172.21.15.203:0/3912140354 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a380bb770 con 0x7f2a38067340 2021-01-06T22:24:39.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a37fff700 1 --2- 172.21.15.203:0/3912140354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a38067340 0x7f2a38068110 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f2a2800b980 tx=0x7f2a28006de0).ready entity=mon.1 client_cookie=8426aa94fc79e12b server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:39.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a36ffd700 1 -- 172.21.15.203:0/3912140354 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a280096d0 con 0x7f2a38067340 2021-01-06T22:24:39.246 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a36ffd700 1 -- 172.21.15.203:0/3912140354 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a28009830 con 0x7f2a38067340 2021-01-06T22:24:39.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a36ffd700 1 -- 172.21.15.203:0/3912140354 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28009bb0 con 0x7f2a38067340 2021-01-06T22:24:39.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3d63b700 1 -- 172.21.15.203:0/3912140354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a38067340 msgr2=0x7f2a38068110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:39.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3d63b700 1 --2- 172.21.15.203:0/3912140354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a38067340 0x7f2a38068110 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f2a2800b980 tx=0x7f2a28006de0).stop 2021-01-06T22:24:39.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3d63b700 1 -- 172.21.15.203:0/3912140354 shutdown_connections 2021-01-06T22:24:39.247 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3d63b700 1 --2- 172.21.15.203:0/3912140354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a38067340 0x7f2a38068110 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:39.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3d63b700 1 --2- 172.21.15.203:0/3912140354 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a38067720 0x7f2a380602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:39.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3d63b700 1 -- 172.21.15.203:0/3912140354 >> 172.21.15.203:0/3912140354 conn(0x7f2a380667b0 msgr2=0x7f2a380c0890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:39.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3d63b700 1 -- 172.21.15.203:0/3912140354 shutdown_connections 2021-01-06T22:24:39.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.239+0000 7f2a3d63b700 1 -- 172.21.15.203:0/3912140354 wait complete. 2021-01-06T22:24:39.248 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a3d63b700 1 Processor -- start 2021-01-06T22:24:39.249 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a3d63b700 1 -- start start 2021-01-06T22:24:39.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a3d63b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a38067340 0x7f2a38146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:39.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a3d63b700 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a38067720 0x7f2a3815ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:39.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a3d63b700 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a3814b4f0 0x7f2a3814e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:39.251 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a3d63b700 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a38149790 con 0x7f2a38067720 2021-01-06T22:24:39.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a3d63b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3813f940 con 0x7f2a38067340 2021-01-06T22:24:39.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a3d63b700 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3815ef10 con 0x7f2a3814b4f0 2021-01-06T22:24:39.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a37fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a38067340 0x7f2a38146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:39.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a37fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a38067340 0x7f2a38146720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57882/0 (socket says 172.21.15.203:57882) 2021-01-06T22:24:39.252 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a37fff700 1 -- 172.21.15.203:0/2984025986 learned_addr learned my addr 172.21.15.203:0/2984025986 (peer_addr_for_me v2:172.21.15.203:0/0) 2021-01-06T22:24:39.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a3ce3a700 1 --2- 172.21.15.203:0/2984025986 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a3814b4f0 0x7f2a3814e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:39.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a377fe700 1 --2- 172.21.15.203:0/2984025986 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a38067720 0x7f2a3815ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:39.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a37fff700 1 -- 172.21.15.203:0/2984025986 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a3814b4f0 msgr2=0x7f2a3814e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:39.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a37fff700 1 --2- 172.21.15.203:0/2984025986 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a3814b4f0 0x7f2a3814e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:39.253 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a37fff700 1 -- 172.21.15.203:0/2984025986 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a38067720 msgr2=0x7f2a3815ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:39.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a37fff700 1 --2- 172.21.15.203:0/2984025986 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a38067720 0x7f2a3815ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:39.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a37fff700 1 -- 172.21.15.203:0/2984025986 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3813f6a0 con 0x7f2a38067340 2021-01-06T22:24:39.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a37fff700 1 --2- 172.21.15.203:0/2984025986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a38067340 0x7f2a38146720 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f2a280118e0 tx=0x7f2a28016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:39.254 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a1ffff700 1 -- 172.21.15.203:0/2984025986 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28020070 con 0x7f2a38067340 2021-01-06T22:24:39.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a1ffff700 1 -- 172.21.15.203:0/2984025986 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2801c420 con 0x7f2a38067340 2021-01-06T22:24:39.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a3d63b700 1 -- 172.21.15.203:0/2984025986 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3814a460 con 0x7f2a38067340 2021-01-06T22:24:39.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a1ffff700 1 -- 172.21.15.203:0/2984025986 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2801c7e0 con 0x7f2a38067340 2021-01-06T22:24:39.255 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a3d63b700 1 -- 172.21.15.203:0/2984025986 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a380beac0 con 0x7f2a38067340 2021-01-06T22:24:39.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a3d63b700 1 -- 172.21.15.203:0/2984025986 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a380450a0 con 0x7f2a38067340 2021-01-06T22:24:39.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.243+0000 7f2a1ffff700 1 -- 172.21.15.203:0/2984025986 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 67273+0+0 (secure 0 0 0) 0x7f2a2800f070 con 0x7f2a38067340 2021-01-06T22:24:39.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.247+0000 7f2a1ffff700 1 --2- 172.21.15.203:0/2984025986 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2a18056400 0x7f2a180588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-01-06T22:24:39.256 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.247+0000 7f2a1ffff700 1 -- 172.21.15.203:0/2984025986 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7191+0+0 (secure 0 0 0) 0x7f2a28070d70 con 0x7f2a38067340 2021-01-06T22:24:39.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.247+0000 7f2a377fe700 1 --2- 172.21.15.203:0/2984025986 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2a18056400 0x7f2a180588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-01-06T22:24:39.257 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.247+0000 7f2a377fe700 1 --2- 172.21.15.203:0/2984025986 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2a18056400 0x7f2a180588c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f2a381515b0 tx=0x7f2a2c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-01-06T22:24:39.258 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.251+0000 7f2a1ffff700 1 -- 172.21.15.203:0/2984025986 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+144990 (secure 0 0 0) 0x7f2a28046190 con 0x7f2a38067340 2021-01-06T22:24:39.478 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.475+0000 7f2a3d63b700 1 -- 172.21.15.203:0/2984025986 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a38060360 con 0x7f2a38067340 2021-01-06T22:24:39.479 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.479+0000 7f2a1ffff700 1 -- 172.21.15.203:0/2984025986 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+3571 (secure 0 0 0) 0x7f2a280497b0 con 0x7f2a38067340 2021-01-06T22:24:39.480 INFO:teuthology.orchestra.run.smithi203.stdout: 2021-01-06T22:24:39.480 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6833/3811290068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":3811290068},{"type":"v1","addr":"172.21.15.91:6833","nonce":3811290068}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4251,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/453324421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":453324421},{"type":"v1","addr":"172.21.15.203:6839","nonce":453324421}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":5384,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.91:6837/2972061575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2972061575},{"type":"v1","addr":"172.21.15.91:6837","nonce":2972061575}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":87},{"gid":6151,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1272437650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1272437650},{"type":"v1","addr":"172.21.15.203:6835","nonce":1272437650}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":106}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-01-06T22:03:39.741327+0000","modified":"2021-01-06T22:24:31.368444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6052,"mds_1":5874},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5874":{"gid":5874,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":29,"addr":"172.21.15.91:6836/1921314962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1921314962},{"type":"v1","addr":"172.21.15.91:6836","nonce":1921314962}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6052":{"gid":6052,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":19,"addr":"172.21.15.203:6837/3262828783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3262828783},{"type":"v1","addr":"172.21.15.203:6837","nonce":3262828783}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-01-06T22:24:39.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.479+0000 7f2a3d63b700 1 -- 172.21.15.203:0/2984025986 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2a18056400 msgr2=0x7f2a180588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:39.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.479+0000 7f2a3d63b700 1 --2- 172.21.15.203:0/2984025986 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2a18056400 0x7f2a180588c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f2a381515b0 tx=0x7f2a2c008040).stop 2021-01-06T22:24:39.482 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.479+0000 7f2a3d63b700 1 -- 172.21.15.203:0/2984025986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a38067340 msgr2=0x7f2a38146720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-01-06T22:24:39.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.479+0000 7f2a3d63b700 1 --2- 172.21.15.203:0/2984025986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a38067340 0x7f2a38146720 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f2a280118e0 tx=0x7f2a28016000).stop 2021-01-06T22:24:39.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.479+0000 7f2a3d63b700 1 -- 172.21.15.203:0/2984025986 shutdown_connections 2021-01-06T22:24:39.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.479+0000 7f2a3d63b700 1 --2- 172.21.15.203:0/2984025986 >> [v2:172.21.15.203:6800/18146,v1:172.21.15.203:6801/18146] conn(0x7f2a18056400 0x7f2a180588c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:39.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.479+0000 7f2a3d63b700 1 --2- 172.21.15.203:0/2984025986 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a3814b4f0 0x7f2a3814e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:39.483 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.479+0000 7f2a3d63b700 1 --2- 172.21.15.203:0/2984025986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a38067340 0x7f2a38146720 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:39.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.479+0000 7f2a3d63b700 1 --2- 172.21.15.203:0/2984025986 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a38067720 0x7f2a3815ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-01-06T22:24:39.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.479+0000 7f2a3d63b700 1 -- 172.21.15.203:0/2984025986 >> 172.21.15.203:0/2984025986 conn(0x7f2a380667b0 msgr2=0x7f2a380bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2021-01-06T22:24:39.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.479+0000 7f2a3d63b700 1 -- 172.21.15.203:0/2984025986 shutdown_connections 2021-01-06T22:24:39.484 INFO:teuthology.orchestra.run.smithi203.stderr:2021-01-06T22:24:39.479+0000 7f2a3d63b700 1 -- 172.21.15.203:0/2984025986 wait complete. 2021-01-06T22:24:39.485 INFO:teuthology.orchestra.run.smithi203.stderr:dumped fsmap epoch 106 2021-01-06T22:24:39.495 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2021-01-06T22:24:39.496 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 32 secs before thrashing 2021-01-06T22:24:57.433 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:24:57.435 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-01-06T22:24:57.561 INFO:tasks.ceph.osd.4.smithi091.stderr:2021-01-06T22:24:57.554+0000 7fcf13246700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 22339) UID: 0 2021-01-06T22:24:57.562 INFO:tasks.ceph.osd.5.smithi091.stderr:2021-01-06T22:24:57.554+0000 7fd31d8fe700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 22339) UID: 0 2021-01-06T22:24:57.562 INFO:tasks.ceph.osd.6.smithi091.stderr:2021-01-06T22:24:57.554+0000 7f561c29f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 22339) UID: 0 2021-01-06T22:24:57.562 INFO:tasks.ceph.osd.7.smithi091.stderr:2021-01-06T22:24:57.554+0000 7f7639e2e700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 22339) UID: 0 2021-01-06T22:25:06.856 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2021-01-06T22:25:06.857 DEBUG:teuthology.task.console_log:Killing console logger for smithi203 2021-01-06T22:25:06.857 DEBUG:teuthology.task.console_log:Killing console logger for smithi091 2021-01-06T22:25:06.858 DEBUG:teuthology.task.console_log:Killing console logger for smithi203 2021-01-06T22:25:06.858 DEBUG:teuthology.task.console_log:Killing console logger for smithi091 2021-01-06T22:25:06.859 DEBUG:teuthology.exit:Finished running handlers